Skip to content

DDIR: count projects its values away; remove value and lift_iter - #1101

Merged
frankmcsherry merged 3 commits into
master-nextfrom
ddir-prune
Nov 20, 2026
Merged

frankmcsherry merged 3 commits into
master-nextfrom
ddir-prune

Conversation

@frankmcsherry

@frankmcsherry frankmcsherry commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

count now drops its values before reducing, and value and lift_iter leave the language.

count. count summed diffs over a key's whole value list, so its reduce arranged and read every distinct value of a key on each change.
Lowering now writes map((k, _) -> (k, ())) before the reduce when the value is not unit, alongside the existing shape-dependent lowering of distinct (reducers::expand_by_shape).
With 200,000 values under one key, 200 one-row updates took 0.39 s before and 0.01 s after, with the same answer.
No program in bench/ or examples/ counted a non-unit value; only the demonstration programs and two plan-pair cases did.

value. It was sugar for map; its uses (AoC days 1-4, lambda_slots.ddp, two tests, the LDBC rel.py generator) are now maps.
difftest's generator draws map where it drew value, which changes its random programs.

lift_iter. No users outside one test, which now reads the round as lift's last coordinate.
Explain still writes LinearOp::LiftIter internally, so the IR op stays, with its discipline comment moved onto it.

Projection pushdown fix. A half join whose value went unread never converged: once its value was masked to unit, the dead fact for it was masked again each round, pushdown hit its 16-round limit, and the whole program's projection was reverted.
Count's new map exposed this after a half join; a unit value is now left alone.
Before the fix, no program under bench/, examples/, tests/programs or the plan-pair cases hit the revert.

Plan pairs: only reduce_masks and shared_demand change in plans; every parsed view using count gains the map, as the parsed view has no shapes.

cargo test --release --no-fail-fast -p interactive -p ddir-server: 394 passed.

LinearOp::LiftIter removed

Explain's scope exits now write lift and then a map that keeps the value's fields and the time's last coordinate.
The op, its shape rule, its projection rule, its corgi kernel and its printer line are gone.
Without this, the printer would still write lift_iter, which the parser no longer accepts.

🤖 Generated with Claude Code

https://claude.ai/code/session_01H6wB4Gjwwivtq59LaFYbF5

frankmcsherry and others added 3 commits November 20, 2026 15:36
count summed diffs over a key's whole value list, so its reduce arranged and read every distinct value of a key on each change.
Lowering now writes map((k, _) -> (k, ())) before the reduce when the value is not unit, as distinct already rekeys by the whole record once shapes are known.
With 200,000 values under one key, 200 one-row updates took 0.39 s before and 0.01 s after.

value was sugar for map and is gone; its uses are now maps.
lift_iter had no users outside one test, and lift gives the round as the time's last coordinate.
Explain still writes LinearOp::LiftIter internally, so the IR op stays, with its discipline comment moved onto it.

The two half_join projection tests now end in an explicit reduce.
With count's new map in between, pushdown narrows nothing there: a map that drops a half join's whole value does not let it mask the half join, while a reduce does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H6wB4Gjwwivtq59LaFYbF5
A half join appends its time to the value its lambda gives, and keeps one demand fact for that value.
Once the value was masked to unit, the fact stayed dead, so each round masked it again and counted a change.
Pushdown hit its 16-round limit and reverted the whole program's projection.
That happened whenever a half join's value went unread, as when count now drops its values after one.
A unit value is now left alone, and the half_join tests end in count again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H6wB4Gjwwivtq59LaFYbF5
Explain's scope exits now fold the round into the chain with `lift` and a map that keeps the value's fields and the time's last coordinate.
The op, its shape rule, its projection rule, its corgi kernel and its printer line go.
Removing the `lift_iter` spelling had left the printer writing a word the parser rejects.

Review nits: the lambda_slots header no longer describes `value`, the bitemporal test drops its repeated column, an identity map and a doubled map go, and `by_record` is now `valued`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H6wB4Gjwwivtq59LaFYbF5
@frankmcsherry
frankmcsherry merged commit 5075b08 into master-next Nov 20, 2026
7 checks passed
@frankmcsherry
frankmcsherry deleted the ddir-prune branch November 20, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant