Skip to content

feat: a layer that keeps out of the way - #194

Open
kixelated wants to merge 33 commits into
mainfrom
quest/m1/auto-place
Open

kixelated wants to merge 33 commits into
mainfrom
quest/m1/auto-place

Conversation

@kixelated

@kixelated kixelated commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Implements most of quest/m1/auto-place.md. With Keep out of the way on, a camera layer watches the picture under it and glides to the corner or edge middle where it covers the least that matters: motion, text, and HUD. BUILD shows what it saw, where it could go, and where it was. The design is the autoplace frame of mock/later.html. The decision is made locally and nothing is uploaded, as the Plan settled.

The quest file is rewritten to what is left, the app-in-front trigger and per-app memory. Both need moq to report the app in front (pin-capture-foreground).

Approach

Looking, on the GPU (rs/moq-scene/src/place.rs, place.wgsl)

  • For each item with keep_clear, the compositor draws the scene's items under it into a small picture, with a long side of 512 px. It does this at most 4 times a second, and only while the scene is drawn somewhere (program, preview, an output, or nested in one of them).
  • A compute pass compares that picture with the previous read, one 16x16 workgroup per tile. For each tile it reports the share of pixels that moved, the share on an edge, the share on an edge that held still, and the mean brightness. These are 4 bytes per tile, at most 4 KB per read.
  • The result comes back through map_async and is read on a later frame, so the frame never waits for it.

Deciding, on the CPU (Sight, Placer)

  • Each tile carries three signals:
    • Motion: pixels that moved.
    • Text: edges that hold still.
    • HUD: edges that hold still while the rest of the picture moves.
  • A signal rises at once and lingers for 2 s, so one busy moment never triggers a move.
  • The candidate places are 8 slots: the corners first, then the edge middles, all inside a margin of a twentieth of the shorter side. A slot costs the weighted signal under the area it would cover (motion 0.5, text 0.8, HUD 1.0). An edge middle costs 0.05 extra, because captions and a game's own bars go there.
  • When it moves:
    • After its scene goes on, the item looks for 1 s, then chooses.
    • It chooses again when the content changes kind. That is when the tile layout averaged over the last 500 ms stays apart from the one averaged over 6 s for 1 s; a flash or a fast pan passes sooner.
    • Otherwise it moves only once its place has been the worst for 10 s.
    • Each move must improve the cost by at least 0.1, so it never wavers between two slots that are alike.
  • A move by hand wins. Dragging the layer, or typing its x or y, holds it where it was put until the content changes kind. Undoing that move puts back where it was before, such as its corner (decision 11).
  • It glides to its new place with glide::Glide, 0.4 s by default with in-out easing, or jumps.
  • The scene file keeps the rect where it was put. The program draws the placed rect from a per-render copy of the collection, as the auto-layout PR's reflow does.

BUILD (rs/moq-studio/src/ui/keep.rs)

  • With the layer selected, the stage shows:
    • a glow over what matters, tagged MOVING, TEXT, or HUD;
    • the other places it may go, as dashed outlines in the layer's shape;
    • where it was, with an arrow, the glide length, and "Was here, over the HUD".
  • A legend under the stage ends with "Decided on this machine".
  • The inspector has a Keep out of the way toggle. When it is on, a box below it shows:
    • Stays clear of chips for Motion, Text, and HUD;
    • a 3x3 map of the slot it is in, with its name and why ("Corners first", "Where you put it / Until the content changes");
    • the glide (Jump, 0.2, 0.4, 0.8, or 1.5 s).
  • The Place fields, dragging, nudging, and snapping all use the placed rect, so the layer is moved from where it is drawn.

Sharing with #191 and #192

  • I read feat: a capture that went black says so #192's dark.rs first. This PR uses the same pattern: a compute reduction of 16x16 tiles, a 4 Hz cap, and a map_async readback polled on a later frame. It does not share the code, because the inputs differ. dark.rs reads the capture texture alone. This PR draws the items under one layer, then compares that drawing with the previous read. The shader outputs differ too. Decision 6 settles one readback helper once both land, as a follow-up.
  • feat: scenes that lay themselves out, by slots and roles #191 added glide::Glide, which this PR uses (decision 5): the placer keys it by how many times the layer moved, so each choice glides from where the layer is drawn, while a move by hand or a new glide length starts a fresh one, which shows its first place at once.
  • With feat: scenes that lay themselves out, by slots and roles #191's slots: placing runs before the slots reflow, on the collection as saved, so each keeper's index is its item's, and the arrangement then places the slots around the placed layers. A layer in a slot cannot keep clear: validation refuses it, the inspector hides the switch, and putting a layer in a slot turns it off. BUILD's layers, drags, and browser input apply both placements.

Impact

  • Scene file: a scene's item gains keep_clear, an inline table: keep_clear = {}, or keep_clear = { hud = false, glide = "1s" }. Its fields are motion, text, and hud (bool, default true) and glide (a duration, default 400 ms). The key is omitted when off. Documented in the README.
  • Validation refuses:
    • a glide over 5 s;
    • keep_clear on a top-layer item, which has no picture under it;
    • keep_clear on an item showing a published source, since viewers place that layer where the scene file does.
  • Public API, moq-scene:
    • Item.keep_clear, KeepClear
    • place::{Slot, Signal, Heat, Placement, Left, INTERVAL, SETTLE, WORST, margin}
    • Compositor::placement(scene, index, source) -> Option<&Placement>
  • No change to the wire, the catalog, or moq/.
  • Cost: see Validation. With 3000 frames at 1080p in release on an AMD Radeon 610M (radv), a read frame costs 0.10 ms more GPU time, other frames under 0.01 ms more. There are no added texture copies, only a 2.3 KB buffer copy per read.

Screenshots

View Built Mock-up
BUILD, the facecam glided off the minimap HUD, selected (build_keep_clear)
BUILD in the smallest window (711 px): the legend's labels shortened, the note left out, and the notes over the tags (build_keep_clear_compact)
LIVE, the real release binary under headless sway on radv, 45 s in

The show is rs/moq-studio/shots/keep.toml. A game overlay has a bobbing sun, a rolling road, an HP bar, a minimap, a score, and a lower third. A facecam is saved over the minimap at the top right. In both rows the camera has moved to the bottom right, away from the minimap the analysis marked as a HUD. The view follows the mock-up's layout, with two differences:

  • The tags name only what the analysis found. A tag sits where its signal is strongest, so the score and the lower third glow but carry no TEXT or LAYER tag here.
  • The legend sits right under the stage, not at the foot of the page.

Alternatives

  • Reading the picture on the CPU. This would need a full readback each time, and the compositor keeps frames on the GPU.
  • Reading the final program and masking out the item. This would see the item itself and every layer above it. Drawing only the items under it reads what it would cover.
  • Choosing a slot on every read. It wavered between alike slots and chased each busy moment. The lingering signals, the 0.1 improvement, and moving only on a content change or after 10 s worst fix that.
  • Any rect, not slots. The mock-up and streamers' habits put a camera in corners and edge middles. Free placement would land it at odd spots and make a move by hand harder to respect.

Follow-ups

Validation

  • just ci passes on the merge of main (feat: drop and paste anything on the picture #193): fmt, clippy with and without the system features, and 397 tests.
  • just render-sw passes on lavapipe (58 + 16 + 14 + 1). It includes the new compositor::tests::a_camera_keeping_clear_glides_off_what_moves: a 320x180 scene with a checkerboard, lines, a moving block, and a camera saved over the block. The test checks that:
    • the camera ends at the top left;
    • the place it left was over Motion;
    • the heat marks the block;
    • the glide passes between the two places;
    • the program's texels show the camera there;
    • program and preview agree.
  • place::tests (CPU, 9):
    • slots_keep_inside_the_margin
    • a_rect_covers_the_tiles_under_it_by_area
    • still_edges_are_a_hud_only_while_the_rest_moves
    • it_glides_off_a_hud_once_it_has_looked
    • a_busy_moment_does_not_move_it
    • a_move_by_hand_holds_until_the_content_changes
    • new_content_is_a_new_choice
    • a_signal_switched_off_does_not_matter
    • the_scene_going_on_again_looks_again
  • collection::tests::reads_and_writes_keep_clear and refuses_keeping_clear_where_it_cannot pass.
  • just shots build_keep_clear live build_drop passes. The new view waits until the camera has left and reached its slot.
  • Compositor cost: an uncommitted bench, 3000 frames each, alternating on and off three times, on a 1080p program in release on the AMD Radeon 610M (radv). The median render plus GPU done time was about 0.645 ms with keep-clear off. With it on, a read frame took 0.75 to 0.76 ms and other frames about 0.654 ms. The CPU render median went up 0.01 ms. A collection clone, which builds the placed copy, costs 0.15 to 0.83 µs.
  • The real binary under headless sway (WLR_BACKENDS=headless, radv, 1440x900, the show above): the facecam moved off the minimap on the program (screenshot). Thread CPU was measured over 30 s, twice each:
    • The program thread used 9.3 and 6.3 ms/s with keep-clear on, and 6.3 and 5.0 ms/s with it off: about 0.05 ms more per frame at 30 fps.
    • The UI thread used 101 and 83 ms/s on, and 85 and 78 ms/s off. That is within run-to-run noise: the overlays thread, which this PR does not touch, varied as much (40 to 25 ms/s).
  • Encoder load was not measured. This host has no NVIDIA GPU, so the program had no hardware encoder; keep-clear adds no work to encoding.
  • Review rounds 1 and 2 (0867795, 91cb8e2): a move by hand is explicit, through Compositor::hold from BUILD's drag and nudge and the inspector's X/Y, which also drops a choice about to be made; BUILD's resizes and crops of a placed layer keep the saved x/y. Keepers follow their item by source and saved rect, then index, across a layer added or removed under it and a scene rename. Browser input maps the pointer to where the layer is drawn. New tests: place::tests::a_move_by_hand_while_it_looks_drops_the_choice, a_resize_keeps_its_place, compositor::tests::keepers_follow_their_items, and the GPU test extended with an insert under the camera and a hold. just ci passes on the merge of main (feat: a multiview window with every scene and the sources picked #195, chore: bump quest to v0.17.1 #196): 405 tests.
  • Option 1 of the checkpoint (61cce15): the placer holds only through hold; a top-left resize leaves it in its slot (place::tests::a_rect_saved_elsewhere_is_no_move_by_hand); undo, redo, and reload hold what they move (engine::tests::an_undo_or_a_reload_moves_by_hand, and on lavapipe an_undo_holds_a_layer_keeping_clear). just ci passes: 407 tests.
  • Merge of feat: scenes that lay themselves out, by slots and roles #191 and the glide switch (f20002d, c5157a7): just ci passes (426 tests); just render-sw passes on lavapipe (64 + 18 + 14 + 1); just shots build_keep_clear passes. New tests: collection::tests::refuses_keeping_clear_where_it_cannot covers a layer in a slot, and place::tests::it_glides_as_long_as_it_is_set_to covers a jump and a 1.5 s glide through glide::Glide.
  • Review after the design change (7a9ab71, 5c8730e): a layer keeping clear chooses only while its picture is read, its scene drawn and it shown, and looks again once it is, unless a move by hand holds it (place::tests::it_chooses_only_while_watched); a resize part way along a glide lands at once (a_resize_while_it_glides_lands_at_once); Keep always on top turns keeping clear off and keeps the layer where it was drawn; the legend fits a 711 px window (build_keep_clear_compact). just ci passes on the merge of main (chore: bump quest to v0.18.1 #198): 428 tests; just render-sw passes on lavapipe (64 + 18 + 14 + 1).
  • Review of the head (795e450, 60b4dd5, 442404b): the OpenAI review of 6ab2c58 found a drag could race a choice; a layer keeping clear now stays put while a drag has it in hand, a lease BUILD renews each frame and at least every 125 ms (place::tests::it_chooses_only_once_let_go). Two local rounds followed: a window hidden mid-drag lets go, and a 1 fps canvas keeps the lease. just ci passes on the merge of main (chore: bump quest to v0.20.0 #199): 429 tests; just render-sw passes on lavapipe (64 + 18 + 14 + 1).
  • Pullfrog's review of 6ab2c58: switching keep-clear off now leaves the layer where it is drawn (cb39347). Its other thread became decision 11. The local review of cb39347 found nothing.
  • The OpenAI review of 5316d38: a glide under way pauses where the layer is drawn while a gesture has it in hand, and goes on once let go (3c0b485, place::tests::a_glide_pauses_while_in_hand). just ci passes on the merge of main (chore: bump quest to v0.21.0 #201): 430 tests; just render-sw passes on lavapipe (64 + 18 + 14 + 1). The local review of 3c0b485 found nothing.
  • Decision 11 (6a538ce, 67d6446): a move by hand remembers where the layer was saved and where it was meant to be; an undo back to that rect puts that back (place::tests::an_undo_of_a_move_by_hand_puts_back_its_place, and on lavapipe engine::tests::an_undo_puts_back_a_layer_keeping_clear, which also checks a redo holds and an undo of an edit that held nothing holds). A layer switched off parks its placer, and an undo that switches it back on returns it to its slot (compositor::tests::a_camera_keeping_clear_glides_off_what_moves). just ci passes: 431 tests; the compositor's GPU tests and the engine's undo test pass on lavapipe.
  • Review after decision 11 (30c0f42, 94ceaeb): the round-9 checkpoint was settled by continuing (Design change). An undo now restores the slot even when it comes before the next render or between an undo and a redo, and an undo of switching a held layer off keeps it held. The inspector's switch-off holds the layer before the edit (place::tests::a_move_by_hand_is_held_at_once, extended an_undo_of_a_move_by_hand_puts_back_its_place and compositor::tests::a_camera_keeping_clear_glides_off_what_moves). Two further fixes: nudges made faster than renders add up, and a drag that selects a layer keeping clear keeps the picture's size. The new session's round 2 and round 3 found nothing.
  • Merge of feat: a capture that went black says so #192 and fix: a scene pad marks a lost mask, and the flash names its readback #203 (f5bedbc): just ci passes (446 tests); just render-sw passes on lavapipe (68 + 19 + 14 + 1); just shots build_keep_clear build_keep_clear_compact build_dark passes. quest/m1/readback.md no longer waits on capture-health, which landed.
  • The OpenAI review of f5bedbc: a glide taken in hand now pauses where the last render drew it, even when the next render comes after the glide would have landed (ad6568c, place::tests::a_glide_pauses_where_last_drawn). Pullfrog approved f5bedbc. just ci passes (447 tests); the GPU tests pass on lavapipe; the local review of ad6568c found nothing.

Decisions

The Plan settled that the decision is made locally and nothing is uploaded. The maintainer settled the rest in this comment:

  1. The numbers: ✅ keep them as built and tune them in auto-place-check (rejected: tuning them in this PR without a real game).
    • It looks for 1 s after its scene goes on or the content changes, then chooses.
    • It moves without a content change only after 10 s in the worst place.
    • A move must improve the cost by 0.1.
    • Weights: motion 0.5, text 0.8, HUD 1.0. An edge middle costs 0.05 extra.
    • The content counts as changed when the 500 ms and 6 s layouts stay 0.12 apart for 1 s.
    • Signals linger for 2 s.
  2. Margin: ✅ fixed at a twentieth of the shorter side (rejected: a canvas setting).
  3. Where it is refused: ✅ the top layer and published sources; nested scenes are allowed and look at their own scene (rejected: allowing it everywhere).
  4. Names: ✅ keep_clear in the scene file; "Keep out of the way" and "Stays clear of" in the inspector, with the chips Motion, Text, and HUD.
  5. Glide: ✅ use feat: scenes that lay themselves out, by slots and roles #191's glide::Glide in whichever PR merges second (rejected: keeping a private Glide in place.rs). feat: scenes that lay themselves out, by slots and roles #191 landed first, and c5157a7 switches to it.
  6. Readback: ✅ one shared helper with feat: a capture that went black says so #192's dark.rs once both land, as the follow-up quest/m1/readback.md (rejected: sharing it in this PR before feat: a capture that went black says so #192 lands).
  7. Glide choices: ✅ Jump, 0.2, 0.4, 0.8, and 1.5 s, default 0.4 s; the scene file takes up to 5 s.
  8. When it reads: ✅ only while its scene is drawn somewhere, at most 4 times a second, from a 512 px copy. A scene that is not drawn keeps its last placement and looks again once it goes on.
  9. Scope: ✅ the app-in-front trigger and per-app memory stay in the rewritten quest/m1/auto-place.md, blocked on pin-capture-foreground; auto-place-check stays (rejected: doing them here).
  10. The round-3 review checkpoint: ✅ explicit intent: the placer no longer reads a changed saved x/y as a move by hand; BUILD's moves call Compositor::hold, and undo, redo, and reload hold each layer keeping clear whose saved x/y they change. Identity stays source, then saved rect, then index (rejected: a runtime item identity in this PR, which is the follow-up item-identity; patching each finding; merging as is). Decision, applied.
  11. Undo after a hold: ✅ an undo back to where a layer was saved before a move by hand held it puts back the placement it had before that hold, such as the corner it had glided to; every other undo, and every redo or reload, still holds. An undo puts back what the streamer saw (rejected: letting it choose again after an undo; keeping decision 10 as built). Decision. Design change: an undo of switching keep-clear off is the same, so a layer switched off keeps its placer, parked, while it is there, and the undo puts it back in its slot.

Models

Size: L
Implementer: Claude, claude-opus-5-5, unknown effort
Implementer: Claude Code, claude-opus-5-5, unknown effort (iterate, L)

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

Review

Rewritten by quest review after each round.

  • Head: ad6568c

  • Base: origin/main at 7e75078

  • Risk: medium, the base has no RISK.md to name what is always high or customer-facing, so every change is medium

  • Classified: medium by default

  • Reviewer: Codex, gpt-6.1-sol, xhigh effort

  • Session: 01a1e991-37bf-7fe2-92b9-f92f837a4db4, round 5

  • Score: 10/10, The glide pause now follows the last drawn position, with no new issues found by code inspection; builds and tests were not run under the review constraints.

  • Rounds: 17; first round 3 (medium 3); findings 11 (medium 11): fixed 11, refuted 0, open 0

  • Findings: none

  • Reviewer: Codex, gpt-6.1-sol, high effort, shadow

  • Session: 01a1e991-37bf-7460-85bc-5ed9e65242bb

  • Rounds: 8; first round 4 (medium 4); findings 9 (medium 9): fixed 9, refuted 0, open 0

kixelated and others added 5 commits November 16, 2026 19:20
Quest-Claim: 9aca337475494867
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e rest as quests

A published source and a top-layer item cannot keep out of the way, so
validation refuses them. The README covers the feature and its scene-file
key, and the quest is rewritten to the per-app memory left, with a
follow-up to live with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title A camera that keeps out of the way feat: a layer that keeps out of the way Nov 17, 2026
@kixelated

Copy link
Copy Markdown
Contributor Author

Maintainer decision (taken on the orchestrator's recommendation, per the maintainer's standing instruction to take recommended decisions; the merge pick is still the maintainer's):

  1. ✅ Keep the numbers as built; tune them in auto-place-check.
  2. ✅ Margin fixed at a twentieth of the shorter side, not a setting.
  3. ✅ Refused on the top layer and published sources; nested scenes allowed.
  4. ✅ keep_clear, "Keep out of the way", "Stays clear of", Motion, Text, HUD.
  5. ✅ Switch to feat: scenes that lay themselves out, by slots and roles #191's glide::Glide in whichever PR merges second. feat: scenes that lay themselves out, by slots and roles #191 is ready and expected to land first, so this PR does it when it merges main.
  6. ✅ Pull one shared readback helper out of this PR and dark.rs once both land (a follow-up, not this PR).
  7. ✅ Glide choices Jump, 0.2, 0.4, 0.8, 1.5 s, default 0.4 s, at most 5 s.
  8. ✅ Read only while its scene is shown, at most 4 times a second, from a 512 px copy.
  9. ✅ App-in-front trigger and per-app memory stay in the rewritten quest; keep auto-place-check.

(Written by Claude Opus 5.5)

kixelated and others added 5 commits November 16, 2026 23:40
# Conflicts:
#	rs/moq-studio/src/shots.rs
#	rs/moq-studio/src/ui/build.rs
The maintainer chose to share keep-clear's and the dark probe's readback
once both land, so it is a follow-up quest on capture-health.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The placer no longer guesses a move by hand from the saved rect: BUILD's
drag and nudge and the inspector's X and Y call Compositor::hold, which
also drops a choice it was about to make. A resize or a crop of a placed
layer keeps the saved position, so it stays free to move. A keeper follows
its item when a layer added or removed under it shifts it, or its scene is
renamed, and browser input maps the pointer to where the layer is drawn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A keeper matches its item by source and saved rect before index, so two
layers keeping clear of the same camera keep their own histories when a
layer added under both shifts them. A saved position that changed, as a
reload or an undo gives, holds the layer too, beside Compositor::hold;
BUILD's resizes keep the saved position, so they never do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

Maintainer decision on the round-3 checkpoint (taken on the orchestrator's recommendation, per the maintainer's standing instruction):

✅ Option 1: drop the "changed saved x/y means a move by hand" check; reload and undo/redo call Compositor::hold for each layer whose position they change. Accept source/position/index matching for identity in this PR.

✅ Option 2 becomes a follow-up quest in this PR: a runtime item ID that edits and undo keep and the scene file never saves, used to match placement state (and usable by auto-layout's slots), covering Duplicate, paste, and drop.

Rejected: patching each finding (option 3), merging as is (option 4).

(Written by Claude Opus 5.5)

…clear

Per the round-3 checkpoint, the placer no longer reads a changed saved
x/y as a move by hand, so a resize from the top or left never holds it.
Undo, redo, and reload hold each layer keeping clear whose saved x/y they
change. A runtime identity for items, which ends matching placers by
source and rect, is a follow-up quest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

Design change: applying the round-3 checkpoint decision, option 1, in 61cce15.

  • Intent is explicit. Placer no longer reads a changed saved x/y as a move by hand, so a resize or crop from the top or left handle never holds a layer or drops its pending choice (place::tests::a_rect_saved_elsewhere_is_no_move_by_hand). A move by hand reaches it only through Compositor::hold: BUILD's Move drag and arrow-key nudge, and the inspector's X and Y.
  • Undo, redo, and reload hold. State::seek and State::replace list each layer keeping clear that they move, meaning the same scene's item at the same index with the same source at another x or y (engine::moved). They call Compositor::hold for each once the collection is adopted (engine::tests::an_undo_or_a_reload_moves_by_hand, and the GPU test an_undo_holds_a_layer_keeping_clear). An ordinary edit made through commit holds nothing by itself.
  • Identity stays source, then saved rect, then index (compositor::follow). Option 2 is the new follow-up quest/m1/item-identity.md [M]: a runtime item identity that edits and undo keep and the scene file never saves.

The next review starts a new round count.

(Written by Claude Opus 5.5)

kixelated and others added 7 commits November 17, 2026 02:40
Placing the layers keeping clear runs before the slots reflow, on the
collection as saved, so a keeper's index stays the item's; the reflow then
arranges the slots around the placed layers. A layer in a slot cannot keep
clear (validation refuses it; putting one in a slot switches it off), and
BUILD's layers, drags, and browser input apply both placements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per decision 5, place.rs drops its own Glide for auto-layout's, keyed by
how many times the layer moved: a move by hand or a change of glide length
starts a new one, so the first place shows at once and the next glides
from where it is. A Jump glides in no time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d more

A layer keeping clear chooses only while its picture is read, its scene
drawn and it shown, and looks again once it is; a resize part way along a
glide lands at once; Keep always on top switches keeping clear off and keeps
where it was drawn; BUILD's legend shortens its labels, then drops its note,
in a narrow window, and the notes draw over the tags they name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 17, 2026 11:26
@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 45 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e3ca66a3-3980-47ee-8f3d-6ad374abed64
Commits

Reviewing files that changed from the base of the PR and between 7e75078 and ad6568c.

Files selected for processing: 18

  • README.md
  • quest/m1/auto-place-check.md
  • quest/m1/auto-place.md
  • quest/m1/item-identity.md
  • quest/m1/readback.md
  • rs/moq-scene/src/collection.rs
  • rs/moq-scene/src/compositor.rs
  • rs/moq-scene/src/lib.rs
  • rs/moq-scene/src/place.rs
  • rs/moq-scene/src/place.wgsl
  • rs/moq-studio/shots/keep.toml
  • rs/moq-studio/src/engine.rs
  • rs/moq-studio/src/shots.rs
  • rs/moq-studio/src/ui.rs
  • rs/moq-studio/src/ui/build.rs
  • rs/moq-studio/src/ui/inspector.rs
  • rs/moq-studio/src/ui/keep.rs
  • rs/moq-studio/src/ui/slots.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed 6ab2c58f3c9ed08ad04e4f6fc19ebf31aca66729. The bounded asynchronous readback is a sensible direction; I’d fix one placement bug before shipping:

  • BUILD drag, lines 411–435: the gesture only holds the placer when a move changes x/y. A resize or crop can still auto-place while the pointer is down, and a stationary move can re-arm when the scene changes. Suspend auto-placement for the selected item until the gesture ends, then preserve the existing manual hold semantics.

Static review only; I didn’t run the GPU fixtures or gesture tests.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Two paths put a layer that keeps clear back on the rect it moved away from, on program, with nothing to move it off again: undoing a drag, and switching Keep out of the way off. Details inline.

Reviewed changes

I reviewed the full diff at 6ab2c58: the GPU look and the CPU placer in moq-scene, how the compositor stages placements, the engine's undo/reload hold, and the BUILD/inspector UI. I left out the drag-gesture finding the OpenAI review already raised on build.rs:411–435.

  • place.rs / place.wgsl: a compute shader turns the picture under a layer into 16×16 tiles of motion, edges, still edges, and brightness. The result is read back with map_async at up to 4 Hz. A Placer turns the tiles into lingering Motion/Text/HUD heat and picks one of 8 corner or edge-middle slots, using settle, worst-place, and content-change rules, then glides there with glide::Glide.
  • compositor.rs: keepers are keyed by (scene, index) and re-matched each render by follow (source, then saved rect, then index). place() stages the placed rects before reflow. Reads happen only while a scene is drawn on program, preview, an output, or nested in one of those. New public API: placement and hold.
  • collection.rs: adds Item.keep_clear / KeepClear. Validation refuses it on the top layer, in a slot, on a published source, and with a glide over 5 s.
  • engine.rs: seek (undo/redo) and reload call Compositor::hold for each layer that keeps clear and whose saved x/y they change.
  • BUILD UI (keep.rs, build.rs, inspector.rs, slots.rs): the overlay shows heat, slots, and where the layer was, with a legend. The inspector gets the toggle, chips, slot map, and glide. Drags and nudges start from the placed rect and call hold. Putting a layer in a slot or always on top turns keeping clear off.
  • Quests: rewrites auto-place.md to the remaining app-in-front work and adds auto-place-check, item-identity, and readback.

quest context was not run: this review shell has no quest binary.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

Comment thread rs/moq-studio/src/engine.rs Resolved
Comment thread rs/moq-studio/src/ui/inspector.rs Resolved
BUILD tells the compositor which layer a drag holds, each frame of it, and
a layer keeping clear neither chooses nor counts its place the worst while
in hand: a resize or a crop no longer races a choice, and a move that has
not yet moved is not re-armed. Let go, it goes on as the gesture left it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

On the review of 6ab2c58: fixed in 795e450, 60b4dd5, and 442404b, head 5316d38.

  • BUILD tells the compositor which layer a drag has in hand, each frame of the gesture (keep::grab, Compositor::grab). A layer keeping clear neither chooses a place nor counts its place the worst while in hand (Placer::grab), so a resize or crop no longer races a choice. A move that has not yet moved holds nothing, and nothing re-arms it.
  • Let go, it goes on as the gesture left it: a move holds it, as before, and a resize or crop leaves it free.
  • The hand is a lease of place::GRAB (500 ms), renewed every frame of the drag. While a drag lasts, BUILD repaints at least every 125 ms whatever the canvas rate, so a still pointer keeps it, and a window hidden mid-drag lets go.
  • Test: place::tests::it_chooses_only_once_let_go. The local review of 442404b found nothing more.

(Written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed 5316d38104708196f9cfb8d2faf7e0661e475808. The lease and UI repaint now prevent new choices during a stationary drag, resize, or crop. One related case still moves the layer under the pointer: place.rs:593–608 skips choose while grabbed, but still advances an already-running glide. Start cropping during a glide: the crop gesture has captured the displayed rect, but the layer continues toward the new slot because a crop needn’t change the placement target. Pause or cancel the glide at the displayed rect when grabbed, and extend the regression to start a gesture partway through a glide.

Static inspection only; I didn’t run the reported GPU or UI tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as draft November 17, 2026 11:59

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues in the commits since 6ab2c58. The undo thread on engine.rs:231 stays open for the maintainer's call. The OpenAI review's point that a glide keeps running while grabbed (place.rs:593–608) is not addressed yet, and I have not repeated it here.

Reviewed changes

This review covers the delta from 6ab2c58 to cb39347: the drag lease (795e450, 60b4dd5, 442404b), the merge of main (#199), and the switch-off fix (cb39347).

  • Added a drag lease on a layer keeping clear. Compositor::grab records (scene, index, source) with an expiry of now + GRAB (500 ms). place() matches it per keeper and calls Placer::grab, which skips choose and clears worst while the layer is in hand. BUILD renews the lease every frame through keep::grab and repaints at GRAB / 4, so a 1 fps canvas keeps it. A window hidden mid-drag lets go once the lease lapses. On release the layer carries on as the gesture left it, held after a move. This is covered by place::tests::it_chooses_only_once_let_go.
  • Kept a layer where it is drawn when keeping clear is switched off. The inspector now saves placement.rect when keep_clear goes to None, matching Keep always on top. Turning it on, or changing the chips or the glide, still saves item.rect. The README says so. place() runs for every keeper on every render, so placement.rect is never the placer's initial zero rect by the time the inspector reads it.

Pullfrog  | Fix it ➔ | View workflow run | Using claude-opus-5-5 | 𝕏

kixelated and others added 2 commits November 17, 2026 04:16
Taken in hand part way along a glide, as a crop started then is, a layer
keeping clear stops where it is drawn, so it stays under the pointer, and
glides on from there once let go. A move by hand still holds it at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

On the review of 5316d38: fixed in 3c0b485, head 07704f8.

  • When a gesture takes a layer keeping clear in hand part way along a glide, as a crop started then does, the placer pauses the glide at the rect it was drawn at, sized as it is meant to be. It stays under the pointer for as long as it is in hand.
  • Once let go, a new glide runs from there to its slot, as long as its settings say.
  • A move by hand still holds it at once, and cancels the pause.
  • Test: place::tests::a_glide_pauses_while_in_hand starts the gesture halfway along a glide, checks the layer stays put for four glide lengths, then checks it glides on and lands once let go. The local review of 3c0b485 found nothing more.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Contributor Author

Maintainer decision on the undo question from #194 (comment) (taken on the orchestrator's recommendation, per the maintainer's standing instruction):

✅ Option 1: an undo that returns a layer to where it was saved before a hold restores the placement it had before that hold (such as the corner it had glided to). Every other undo, redo, or reload still holds. An undo puts back what the streamer saw.

Rejected: letting it choose again after an undo (option 2), keeping decision 10 as built (option 3).

(Written by Claude Opus 5.5)

kixelated and others added 2 commits November 17, 2026 04:36
… was

A move by hand remembers where the scene file put the layer and where it
was meant to be, such as the corner it had glided to. An undo back to that
saved rect puts that place back at once instead of holding there, so the
undo shows what the streamer saw before the move. Every other undo, and
every redo or reload, still holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t was

A layer that stops keeping clear parks its placer, held, for as long as
it is there. An undo that turns keeping clear back on, saving the rect it
had before, revives it, so it goes back to the slot it had glided to
rather than starting over from its saved rect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

Design change: an undo of switching Keep out of the way off now puts the layer back where it was, as decision 11 does for a move by hand (67d6446).

Switching it off saves the drawn rect, and the layer's placer used to be dropped with it. Undoing the switch-off started a fresh placer at the old saved rect, which can be over the minimap, and a second later it glided away again. Now a layer that stops keeping clear parks its placer, held, for as long as the layer is there. An undo that turns keeping clear back on and saves the old rect revives the placer, so the layer returns to its corner at once. Switching it back on by hand still starts fresh and looks again.

This follows from "an undo puts back what the streamer saw". It is recorded in the PR body under decision 11, and tested in compositor::tests::a_camera_keeping_clear_glides_off_what_moves on lavapipe.

(Written by Claude Opus 5.5)

- Switching keeping clear off in the inspector holds the layer as it was,
  so an undo before the next render still finds where it was.
- An undo that switches keeping clear back on tells the compositor even
  when the rect is unchanged, so a parked placer that was held stays held.
- Undo and redo set the placer's saved rect at once, so an undo, redo, and
  undo between renders, as on a slow canvas, still puts the slot back.
- item-identity: an undo of Keep always on top, whose layer leaves its
  scene, waits on the runtime identity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

Design change: the review checkpoint at 67d6446 (round 9, 3 findings, score 7) is settled by continuing, with three fixes in 30c0f42.

Stepping back. The problem is that an undo should put back what the streamer saw before the edit it undoes.

From first principles, the most exact design snapshots each keep-clear layer's placement with every history entry. An undo would then restore that snapshot. It would cover every edit, including ones that move a layer between layers. But it couples the history to the compositor's runtime state. It would also have to merge snapshots across a drag's many steps, and it repeats the identity problem that item-identity (decision 10) solves.

The current design keeps a "held from" memory in each placer, and the engine tells the compositor which layers an undo moved. That stays inside the placer. All three findings were bookkeeping gaps in it, not a wrong approach:

  1. A held layer switched off and undone lost its hold. An undo that turns keeping clear back on now notifies the compositor even when the rect is unchanged, so the parked placer stays held.
  2. An undo that came before the switch-off rendered held the old rect. The inspector's switch-off now holds the layer before the edit, the same way a move by hand does.
  3. Undo, redo, and undo between renders failed. Undo and redo now set the placer's saved rect at once.

The recommendation is to continue. That fits decision 11 ("an undo puts back what the streamer saw").

Still open, recorded in quest/m1/item-identity.md: an undo of Keep always on top moves the layer back into its scene from the top layer, and starts a fresh placer. That waits on the runtime identity.

(Written by Claude Opus 5.5)

kixelated and others added 4 commits November 17, 2026 05:07
- A move by hand marks the placement held at once, and a held layer is
  drawn where it is saved, so each nudge before the next render starts
  from the last one rather than the last rendered place.
- The keep-clear legend's room under the picture holds for a drag, so a
  drag that selects a layer keeping clear does not resize the picture
  under the pointer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 17, 2026 13:21

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: f5bedbc633e01b865190eea9fb073e39eaf1b739

Low, rs/moq-scene/src/place.rs:690–697: pause the last displayed glide position even if the nominal deadline passed since it was drawn. At a slow or skipped render, the user can begin cropping a visibly mid-glide layer, but the next place() can run after gliding expires. That first grabbed frame never sets paused and snaps the item to the slot under the pointer. The new regression starts the grab and renders before the deadline, so it misses this. Freeze from the last rendered rect when the gesture takes hold, and test a grab whose first render is after the deadline.

This closes the earlier case in the normal render cadence. The parked placer and explicit undo/redo tests are useful additions. Static review; I did not run GPU or gesture tests.

pullfrog[bot]
pullfrog Bot previously approved these changes Nov 17, 2026

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues in the commits since cb39347. Both earlier Pullfrog threads are resolved: the undo thread through decision 11, and the switch-off thread in cb39347. I have not repeated the OpenAI review's low finding on f5bedbc, which is about a grab whose first render lands after the glide deadline (place.rs:690–697).

Reviewed changes

This pass covers the branch commits from cb39347 to f5bedbc (3c0b485, 6a538ce, 67d6446, 30c0f42, 94ceaeb, 827f503) and the merges of main (#192, #203). After the merges, place() still runs before reflow in Compositor::render.

  • Pausing a glide while the layer is in hand. If a gesture grabs the layer part way through a glide, it stops at the last placement.rect, taking its target size. Once let go, a fresh glide starts from that point toward the target.
  • Undoing a move by hand. The first Placer::hold records Before { saved, at, looking }. Placer::undone(saved) puts back the old slot when the undo restores the saved rect from before the hold. Any other undo holds, and keeps before. A redo or a reload calls Placer::redone, which holds and records saved straight away, so several undos and redos between two renders still match.
  • Parked placers. A keeper whose item stops keeping clear is held and parked. It follows items that don't keep clear by source, then rect, then index. Compositor::undone revives it when an undo turns keeping clear back on. engine::moved now also lists items whose keep_clear went from off to on, so that undo reaches the compositor. I traced a redo of a switch-on with the placer still parked: it does nothing, and the next render starts a fresh keeper, which is what the original switch-on did.
  • The inspector's switch-off. It holds the layer with its old keep_clear before the edit, so an undo that comes before the next render still finds the slot it should go back to.
  • Placement::drawn and the held flag. hold() now marks the placement held at once, and drawn(saved) returns the saved rect while it is held. Nudges made faster than renders therefore add up. Panels.legend keeps the room for the legend fixed during a drag, so the picture under the pointer does not resize.
  • Quests. item-identity now includes undoing Keep always on top. readback no longer requires capture-health, and its tile grids (64×36 for the probe, 32×18 for the look) match dark.rs TILES and place.rs SIDE/TILE.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

Whether a glide was under way when a gesture takes the layer now comes
from the last render, which drew it short of where it aimed, rather than
from when the glide would land, so a grab whose first render comes after
that, as on a slow canvas, still stops it under the pointer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Contributor Author

Fixed the low finding from the OpenAI review of f5bedbc in ad6568c.

The pause no longer depends on the glide's deadline. A gesture now pauses the glide when the last render drew the layer short of where it was aiming. In that case the layer stops at its last drawn rect, however late the next render comes.

A hold or an undo clears that aim, so a held layer still follows the pointer. The new test place::tests::a_glide_pauses_where_last_drawn grabs the layer mid-glide and first renders after the deadline.

just ci passes (447 tests), and the GPU tests pass on lavapipe. The local review of ad6568c found nothing.

(Written by Claude Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
Reviewed commit: ad6568c998fb7f30b72f1bb744029fd45761b60b

The slow-render grab case is fixed. rs/moq-scene/src/place.rs:688–693 now checks whether the last drawn rectangle reached its destination, rather than whether a timer expired. That freezes the visible position even if the first grabbed frame is late. The new regression exercises that timing, and manual holds and undo clear the old destination. No new actionable issue found by inspection; I did not run the tests.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found. ad6568c fixes the OpenAI review's low finding on f5bedbc: the first grabbed render coming after the glide deadline.

Reviewed changes

This pass covers f5bedbc..ad6568c, a single commit in rs/moq-scene/src/place.rs.

  • Replaced the glide timer with a drawn-rect check. The Placer.gliding deadline is gone. A glide now counts as under way when the last target in aimed differs from the last drawn placement.rect. A grab therefore pauses where the layer was last drawn, however late the next render comes. Glide::show returns to exactly once linear >= 1, so a landed glide compares equal and never pauses by mistake. hold and undone clear aimed, so neither leaves a stale target behind.
  • Added place::tests::a_glide_pauses_where_last_drawn. It renders part way along a glide, grabs, and then renders well past where the glide would have landed. The layer stays at the rect drawn last.

Pullfrog  | View workflow run | Using claude-opus-5-5 | 𝕏

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant