compute: import published indexes as a shared arrangement - #38390
Draft
antiguru wants to merge 1 commit into
Draft
compute: import published indexes as a shared arrangement#38390antiguru wants to merge 1 commit into
antiguru wants to merge 1 commit into
Conversation
antiguru
force-pushed
the
mh/interactive-05-import
branch
from
August 21, 2026 11:23
cf8f1f6 to
5c374d3
Compare
antiguru
force-pushed
the
mh/interactive-05-import
branch
from
August 21, 2026 13:24
5c374d3 to
1bce309
Compare
antiguru
force-pushed
the
mh/interactive-05-import
branch
from
August 21, 2026 13:42
1bce309 to
35cb94e
Compare
antiguru
force-pushed
the
mh/interactive-05-import
branch
from
August 21, 2026 14:31
35cb94e to
f92f260
Compare
`ArrangementFlavor` gains a `SharedTrace` variant carrying arrangements backed by `TraceFrontier<SharedTraceHandle>`, and `import_index_shared` constructs it on the interactive runtime instead of reading the local `TraceManager`, which holds nothing there. Because the shared handle shares the `RowRow`/`Err` batch and cursor types of a maintenance `TraceAgent`, the variant flows through the same generic bodies as `Trace`, so a downstream `Get` receives a real arrangement with its key and permutation intact rather than a collection the plan has to re-derive. Joins, delta joins, thresholds, and hydration logging gain the corresponding arm. The import is a static snapshot at `as_of`, bounded one step past it rather than by `self.until`. Interactive work is single-time, so the capability must drop once the shared trace seals past `as_of` for the one-shot result to complete, and an unbounded `until` never gets there. An `as_of` at `Timestamp::MAX` yields the empty bound, which is the right reading of "the final state". The read hold is the returned `Arranged`'s own trace rather than a separate token, so consumers can downgrade it as their frontiers advance. A token nobody downgrades would be a floor under every hold that is downgraded, because the publisher forwards the meet. The two export paths reject `SharedTrace` as unreachable. Only the interactive runtime produces the variant, and its exports are freshly rendered query outputs, never a re-export of an import. Both `import_index_shared` and the variant are reachable only when a runtime holds the `Interactive` role, which nothing constructs yet. The two `sharing` tests that build this variant return here, along with the `consolidate_capture` helper they share. `render`'s tests move out of line to `render/tests.rs`, per the convention in `src/compute/AGENTS.md`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
antiguru
force-pushed
the
mh/interactive-05-import
branch
from
August 21, 2026 17:54
f92f260 to
aa7c8eb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fifth of eight PRs splitting #37770. Stacks on #38389. Tracked by CPU-215.
ArrangementFlavorgains aSharedTracevariant carrying arrangements backed byTraceFrontier<SharedTraceHandle>, andimport_index_sharedconstructs it on the interactive runtime instead of reading the localTraceManager, which holds nothing there. The shared handle shares theRowRow/Errbatch and cursor types of a maintenanceTraceAgent, so the variant flows through the same generic bodies asTraceand a downstreamGetreceives a real arrangement with its key and permutation intact, rather than a collection the plan re-derives. Joins, delta joins, thresholds, and hydration logging gain the corresponding arm.The import is a static snapshot at
as_of, bounded one step past it rather than byself.until: interactive work is single-time, so the capability must drop once the shared trace seals pastas_offor the one-shot result to complete, and an unboundeduntilnever gets there.The read hold is the returned
Arranged's own trace rather than a separate token, so consumers can downgrade it as their frontiers advance. A token nobody downgrades would be a floor under every hold that is downgraded, since the publisher forwards the meet.The variant and its constructor land together because a never-constructed variant of an enum inside the private
rendermodule tripsdead_code, and CI lints with-D warnings. Reachable only when a runtime holds theInteractiverole, which nothing constructs yet.