Repository navigation
DDIR: count projects its values away; remove value and lift_iter - #1101
Merged
Merged
Conversation
count summed diffs over a key's whole value list, so its reduce arranged and read every distinct value of a key on each change. Lowering now writes map((k, _) -> (k, ())) before the reduce when the value is not unit, as distinct already rekeys by the whole record once shapes are known. With 200,000 values under one key, 200 one-row updates took 0.39 s before and 0.01 s after. value was sugar for map and is gone; its uses are now maps. lift_iter had no users outside one test, and lift gives the round as the time's last coordinate. Explain still writes LinearOp::LiftIter internally, so the IR op stays, with its discipline comment moved onto it. The two half_join projection tests now end in an explicit reduce. With count's new map in between, pushdown narrows nothing there: a map that drops a half join's whole value does not let it mask the half join, while a reduce does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4Gjwwivtq59LaFYbF5
A half join appends its time to the value its lambda gives, and keeps one demand fact for that value. Once the value was masked to unit, the fact stayed dead, so each round masked it again and counted a change. Pushdown hit its 16-round limit and reverted the whole program's projection. That happened whenever a half join's value went unread, as when count now drops its values after one. A unit value is now left alone, and the half_join tests end in count again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4Gjwwivtq59LaFYbF5
Explain's scope exits now fold the round into the chain with `lift` and a map that keeps the value's fields and the time's last coordinate. The op, its shape rule, its projection rule, its corgi kernel and its printer line go. Removing the `lift_iter` spelling had left the printer writing a word the parser rejects. Review nits: the lambda_slots header no longer describes `value`, the bitemporal test drops its repeated column, an identity map and a doubled map go, and `by_record` is now `valued`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H6wB4Gjwwivtq59LaFYbF5
frankmcsherry
force-pushed
the
ddir-prune
branch
from
November 20, 2026 20:41
0c0a333 to
02c1aec
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
countnow drops its values before reducing, andvalueandlift_iterleave the language.count.
countsummed diffs over a key's whole value list, so its reduce arranged and read every distinct value of a key on each change.Lowering now writes
map((k, _) -> (k, ()))before the reduce when the value is not unit, alongside the existing shape-dependent lowering ofdistinct(reducers::expand_by_shape).With 200,000 values under one key, 200 one-row updates took 0.39 s before and 0.01 s after, with the same answer.
No program in
bench/orexamples/counted a non-unit value; only the demonstration programs and two plan-pair cases did.value. It was sugar for
map; its uses (AoC days 1-4,lambda_slots.ddp, two tests, the LDBCrel.pygenerator) are now maps.difftest's generator draws
mapwhere it drewvalue, which changes its random programs.lift_iter. No users outside one test, which now reads the round as
lift's last coordinate.Explain still writes
LinearOp::LiftIterinternally, so the IR op stays, with its discipline comment moved onto it.Projection pushdown fix. A half join whose value went unread never converged: once its value was masked to unit, the dead fact for it was masked again each round, pushdown hit its 16-round limit, and the whole program's projection was reverted.
Count's new map exposed this after a half join; a unit value is now left alone.
Before the fix, no program under
bench/,examples/,tests/programsor the plan-pair cases hit the revert.Plan pairs: only
reduce_masksandshared_demandchange in plans; every parsed view usingcountgains the map, as the parsed view has no shapes.cargo test --release --no-fail-fast -p interactive -p ddir-server: 394 passed.LinearOp::LiftIterremovedExplain's scope exits now write
liftand then a map that keeps the value's fields and the time's last coordinate.The op, its shape rule, its projection rule, its corgi kernel and its printer line are gone.
Without this, the printer would still write
lift_iter, which the parser no longer accepts.🤖 Generated with Claude Code
https://claude.ai/code/session_01H6wB4Gjwwivtq59LaFYbF5