feat(analysis): record analysis runs into the model with %record and -record-run - #557
Merged
Merged
Conversation
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…lback Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ed library Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…n-free Co-Authored-By: jason.han <hanhuijun@gmail.com>
…nresolved Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ded conversions Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…r gaps Co-Authored-By: jason.han <hanhuijun@gmail.com>
… was not recorded Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…rries a unit Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…nition Co-Authored-By: jason.han <hanhuijun@gmail.com>
…othing Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…oncrete type Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
An output read that fails because the activation never assigned it now carries which output of which calc could not be read, so a Monte Carlo run can tell an output that depends on a statistic the sample has not supplied — the sample's own — from one that failed for a reason of its own. Co-Authored-By: jason.han <hanhuijun@gmail.com>
The run's record carries the body's verdict as the new AnalysisRun feature 'verdict', and each verdict the body and its subcases produced as a VerdictRecord row of kind 'verification' or 'subcase' — they were run but never recorded. Verifications alone satisfy 'nothing to record'. Co-Authored-By: jason.han <hanhuijun@gmail.com>
The runtime now leaves the sample's outputs out of a run's Unread, so the name adjudication against the conclusion is gone — anything left failed the iteration. A verification case's record passes its verdicts through. And a generated record adding an occurrence of a diagnostic the model already reported is refused, not accepted. Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
A name bound on both sides is one parameter, not an input colliding with an output: the member carries the value the run left in it and a <name>In companion the value it was bound with, emitted for every inout whatever the two values. Same-side duplicates and a member colliding with the companion still error. Co-Authored-By: jason.han <hanhuijun@gmail.com>
When several loaded files open the same top namespace, the record goes to the one already holding the deepest prefix of the generated package nesting, rather than making a second package of that name in the first. Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
… to Real A scalar-valued enumeration literal records as the literal it is, not the scalar it equals, and members supplied as Integer and Real alike settle to Real — Integer literals remaining valid under it, both in a run's own settlement and against a declared Real member; a declared Integer still refuses a Real, which would widen a definition the model owns. Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ows it Co-Authored-By: jason.han <hanhuijun@gmail.com>
…y member A member supplied as Integer or Real and as a quantity, in either order, settles to the quantity shape — Real with the unit companion — the plain row keeping its literal and taking no unit, as a Real-then-quantity row always did. The inout companion check settles the same pair. Co-Authored-By: jason.han <hanhuijun@gmail.com>
-schedule, -clock-step, -engine and -instantiate change what the run computed or which objects it ran on, so the command the record reports carries each as written, not just the sweep and sample flags. Co-Authored-By: jason.han <hanhuijun@gmail.com>
…anions Co-Authored-By: jason.han <hanhuijun@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
-analysis/%analysis,-sweepand-runsprinted their results and discarded them: nothing wrote a run back into the model,%savenever contained it, and-render-documentrefused to run alongside an analysis. This adds a first-class "record this run" step that turns a completed analysis run into ordinary declared model elements, submitted through the normal REPL declaration path, so that%save/-convert sysml -opersist them and document queries table, filter and sort them like any other element.Surface
%record <case>[(<args>)] [<object>] [into <package>]-record-run "<call>"(repeatable; takes the-analysiscall syntax, composes with-sweepand-runs/-seed) and-record-into <package>.-record-runis allowed together with-render-document(-o,-doc-form) and with-convert sysml -o saved.sysml; the records are made first, so the document sees them and the written file holds them.-render-documentwith plain-analysisstill refuses as before;-record-intowithout-record-runrefuses.Generated record shape (bundled library
AnalysisRecords, non-normative, besideDocumentQueries/Simulation)Sweeps and Monte Carlo samples write one record per row/run (
iterationset), sharing the provenance. Inputs are the values the body actually ran with — positional, named, defaulted, and the sweep/sample overlay — carried by a newAnalysisResult.Inputs(SweepRow.Inputs,MonteCarloRun.Inputs/Outputs).Value coverage: Integer/Real/Boolean literals; String (escaped); enumeration literals typed by their enum; quantities as a Real magnitude plus a
<name>Unit : Stringcompanion; objects asrefto their declared usage; null/unset declared but left unredefined (objective stays as the runtime decided it,undecidedwhen the case declares none); everything else (sequences, arrays, complex, functions, unresolvable objects, Infinity) as a String holding the runtime's printed text — documented as the fallback.Atomicity: generation is pure (
internal/exec/analysis/record.Generate) and validated before anything is submitted. A missing case, a calc, a failed run, a case with no outputs, a per-case definition that does not specializeAnalysisRunor whose attributes have incompatible types, a reserved-name collision, or a submission that would drop declarations or raise new diagnostics all report a diagnostic and leave the model byte-identical; a rollback goes through the same rebuild as a submission, so held objects and debugger sessions survive.The record merges into a package that a loaded file declared (the one case where the REPL folds into a file snippet), keeping the file's identity so a later reload still supersedes it.
Limitations (documented): a declared name containing
::is ambiguous in the index's raw-joined FQN form (names needing quotes otherwise work); structured values are recorded as their printed text, not nested attributes; the gRPC service is not extended with a record RPC (follow-up); the hand-writtenexamples/analysis-results-demostays as is and can switch to-record-runin a follow-up — the vocabulary keeps its names (RecordedRun,AnalysisRun,caseName,kind,objective, arefsubject).How it was verified
internal/exec/analysis/record: golden tests (single run with every value kind, trade study, sweep reusing an existing definition) plus error cases; goldens are fixed points of the repository formatter.internal/exec/runtime:AnalysisResult.Inputsorder and provenance (positional/named/default), sweep overlay, Monte Carlo inputs/outputs.internal/frontend/repl:%record→%save→ reload →%recordagain yields_run2; document queries over the records (WhereMetadata('metadata' = "AnalysisRecords::RecordedRun"),WhereFeatureonobjective,Projectof outputs); failure leaves the text unchanged and keeps objects and%actiondebugger sessions; default package placement by the case's owner chain;intoa nested package.cmd/sysml:-record-runalone, with-sweep(one row per value), with-runs/-seed, with-render-document(rendered table shows the recorded values), with-convert sysml -o(re-validates clean, re-recording gives_run2), and both refusals.go build,go vet,gofmt -lempty,make man-check, doc link/ID/changelog checks,mkdocs build --strict, andOPENSYSML_REQUIRE_TRAINING_CORPUS=1 OPENSYSML_REQUIRE_PILOT_CORPORA=1 go test -count=1 ./...green; no corpus ratchet moved. The bundled-library count moved 104 → 105 (self-model figures and architecture doc updated).Checklist
make testandmake lintpass locallychanges/unreleased/<slug>.<section>.md, not as an edit toCHANGELOG.mdmake docs-countsrun if a gate count moved (compliance rows need nothing: the census is counted at docs build)F4,K5) in the body, docs, or changelog