fix(doc): build relationship edge tables once per model, uncharged to the visit budget - #565
Merged
Merged
Conversation
… the visit budget RelatedElements, WhereRelated and relationship-derived columns build a kind's edge table by scanning every workspace declaration, and charged each one to the query's visit budget, so any model past ~100k declarations failed every relationship query before traversing. The scan is now memoized in queryexec.Context.Related, shared across the queries a document evaluates, and left uncharged; traversal still pays one visit per element reached. Co-Authored-By: jason.han <hanhuijun@gmail.com>
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
…uilt from takes an edit Co-Authored-By: jason.han <hanhuijun@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
RelatedElements,WhereRelatedand relationship-derived columns read a per-kind edge table that is built, on first use, by scanning every declaration in the workspace. Each declaration scanned was charged to the query's visit budget (default 100,000), so on any model past ~100k declarations every relationship query failed withvisit-budgetbefore traversing a single edge. The migrated TMT model has 101,014 declarations; its two requirements-mapping documents need about 200 visits for their rows and were the only two of 103 documents that would not render.The scan is a fixed cost of the model, not of the query, so it is no longer charged and is memoized across executions:
executor.scanScopeno longer callsconsumeVisit;relationshipEdges/relatedNeighborscan no longer fail and drop theirexpression/errorplumbing.RelationshipTables.lookup(kind, context)discards entries built against anotherIndex/Model, or against the sameIndexbefore an edit (symbols.Index.Generation(), the write counter its layers already bump andShortNamed/LibraryIdentitymemoize on), so a shared table can never answer for the wrong or a stale model.docir.EvaluateandEvaluateLinkedgive a context without tables its own, so the queries of one document (or a linked set) build each kind once instead of once per query.traverseRelatedstill pays one visit per element reached and still raisesvisit-budget, naming the column for a related column.Default budgets are untouched.
Specification basis
Not a specification behaviour; the executor's own bounded-workload contract. The
RelatedElementsand relationship-column rows ofdocs/project/spec-compliance.mdnow state that the edge tables are built once per model and uncharged, and name the new tests.How it was verified
TestExecuteRelatedLeavesTableConstructionUncharged(a source with no edges succeeds withVisitBudget: 1; two subsetters fit a budget of exactly two),TestExecuteRelatedColumnLeavesTableConstructionUncharged(count0within one visit; count2within two; budget one still fails naming columnrelated), andTestExecuteWhereRelatedChargesTheVisitBudget(a row nothing satisfies is kept withexists = falseunder one visit).TestExecuteRelatedConsumesTheVisitBudgetand the related-columnexhausted visit budgetcase still prove traversal is charged.TestExecuteSharesRelationshipTablesThroughTheContext: two executions under oneContext.Relatedreuse the same table; a context over another index rebuilds it.TestExecuteRebuildsRelationshipTablesAfterAnIndexEdit: oneContext.RelatedfollowsAddDocumentof a further subsetter and itsRemoveDocument.L1 TMT to L2 APS Requirements Mapping DocumentandL3 APT to L2 APS Requirement Mapping Documentrender under the default budget in ~9 s each (previouslyexceeded its visit budget in column Refine (outgoing)); theRefinecells hold the one real refinement the model states, as Cameo's matrix does.go build ./...,go vet ./...,gofmt -l .(empty),go test ./internal/doc/...; fullgo test ./...reported in the checks.Checklist
make testandmake lintpass locallychanges/unreleased/<slug>.<section>.md, not as an edit toCHANGELOG.mdmake docs-countsrun if a gate count moved (compliance rows need nothing: the census is counted at docs build)F4,K5) in the body, docs, or changelog