compute: launch a second, interactive compute runtime - #38391
Draft
antiguru wants to merge 1 commit into
Draft
Conversation
antiguru
force-pushed
the
mh/interactive-06-runtime
branch
from
August 21, 2026 10:15
91defbe to
5eecaa4
Compare
antiguru
force-pushed
the
mh/interactive-06-runtime
branch
from
August 21, 2026 11:23
5eecaa4 to
e6d27f4
Compare
antiguru
commented
Aug 21, 2026
| true, | ||
| "Aggressively downgrade input read holds for indexes on zero-replica clusters.", | ||
| ); | ||
| /// Whether to launch compute replicas with a second, interactive compute timely runtime. |
Member
Author
There was a problem hiding this comment.
Fixed. (Posted by Claude Code.)
antiguru
force-pushed
the
mh/interactive-06-runtime
branch
2 times, most recently
from
August 21, 2026 13:24
a985417 to
3cc5402
Compare
antiguru
force-pushed
the
mh/interactive-06-runtime
branch
from
August 21, 2026 13:42
3cc5402 to
15cc33c
Compare
antiguru
force-pushed
the
mh/interactive-06-runtime
branch
from
August 21, 2026 14:31
15cc33c to
9311614
Compare
With `--interactive-compute-timely-config`, clusterd runs two compute runtimes in one process: the first takes the `Maintenance` role, the second `Interactive`, and both share the one sharing registry so a reader on worker `i` of either finds the slot a publisher on worker `i` of the other filled. The two must span an equal number of Timely peers, which the config preparation asserts, because the registry pairs workers by ordinal and reads are sound only if both shard keys across the same peer count. Without the flag the process runs a single `Solo` runtime and is byte-unchanged from a deployment that has no second runtime. One controller endpoint still fronts the replica. With two runtimes a `Multiplexer` serves it, routing each command to the owning runtime and merging responses; with one, the maintenance client builder serves it directly. Shared fate is the read-hold mechanism. Both runtimes' worker and reader threads are covered by the process-global panic hook installed at the top of `main`, so a panic on either aborts the whole process. That bounds an interactive import's read hold to the life of the replica without a lease, because there is no way for one runtime to wedge while the other's holds continue. A subprocess test asserts the abort, which cannot be observed from inside the panicking process. `ClusterSpec::cluster_name` lets one process run two clusters of the same kind with distinguishable tracing spans. The interactive runtime takes `compute-interactive`; solo and maintenance keep the bare `compute` so single-runtime logs are unchanged. `enable_compute_interactive_runtime` is replica-scoped but resolved in `environmentd`, because the controller decides `ServiceConfig::ports` before the replica exists and so cannot read the value from the replica's `worker_config`. `CatalogState::replica_scoped_bool` parses the override through the dyncfg rather than `str::parse`, since a stored bool formats as `on`/`off`, which `str::parse::<bool>()` rejects. Off by default in production and in tests. Flipping it changes how a replica is provisioned, so it is not a live toggle: a running replica keeps the layout it was launched with. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
antiguru
force-pushed
the
mh/interactive-06-runtime
branch
from
August 21, 2026 17:54
9311614 to
eb80f6a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sixth of eight PRs splitting #37770. Stacks on #38390. Tracked by CPU-216.
With
--interactive-compute-timely-config, clusterd runs two compute runtimes in one process: the first takesMaintenance, the secondInteractive, and both share the one sharing registry so a reader on workeriof either finds the slot a publisher on workeriof the other filled. They must span an equal number of Timely peers, which the config preparation asserts, because the registry pairs workers by ordinal. Without the flag the process runs a singleSoloruntime, unchanged from a deployment with no second runtime.One controller endpoint still fronts the replica: with two runtimes a
Multiplexerserves it, with one the maintenance client builder serves it directly.Shared fate is the read-hold mechanism. Both runtimes' threads are covered by the process-global panic hook installed at the top of
main, so a panic on either aborts the process, bounding an interactive import's read hold to the life of the replica without a lease. A subprocess test asserts the abort, which cannot be observed from inside the panicking process.enable_compute_interactive_runtimeis replica-scoped but resolved inenvironmentd, since the controller decidesServiceConfig::portsbefore the replica exists.replica_scoped_boolparses through the dyncfg rather thanstr::parse, because a stored bool formats ason/off.Off in production and in tests. Flipping it changes how a replica is provisioned, so it is not a live toggle. The interactive runtime cannot serve index peeks until the next PR in the stack, which is safe only because the flag is off here. That is also why the flag is registered in
UNINTERESTING_SYSTEM_PARAMETERSand pinned toFALSEin parallel-workload'sFlipFlagsAction:check-test-flagsrequires a new flag to be registered in the PR that introduces it, but nothing may turn this one on yet. The last PR of the stack promotes it to a variable system parameter defaulting on.