Skip to content

fix(rendering): drive keyboard overlay from recording_time so it follows cuts and encoder start offset - #2118

Open
aacarcrash wants to merge 1 commit into
CapSoftware:mainfrom
aacarcrash:fix/keyboard-overlay-recording-time
Open

fix(rendering): drive keyboard overlay from recording_time so it follows cuts and encoder start offset#2118
aacarcrash wants to merge 1 commit into
CapSoftware:mainfrom
aacarcrash:fix/keyboard-overlay-recording-time

Conversation

@aacarcrash

@aacarcrash aacarcrash commented Aug 12, 2026

Copy link
Copy Markdown

The keyboard overlay uses frame_number / frame_rate (raw output time) to pick active
keyboard segments, while keyboard events are stamped on the recording clock and the
video layer maps output time through the timeline. Two visible desyncs result:

  1. Constant lag by the encoder start offset. The first video frame lands at
    display.start_time on the recording clock (measured 0.4998s on a machine where
    the encoder pre-flight fell back to software x264). Keystroke overlays render that
    much later than the typing visible in the video, on every recording.
  2. Overlays don't follow edits. Cutting or trimming the timeline remaps video
    frames but not the keyboard clock, so overlays drift by the length of whatever was
    removed. The cursor layer doesn't have either problem because it uses
    segment_frames.recording_time, which travels with the decoded frame through
    get_frame_mapping.

Fix: use the same clock as the cursor layer — segment_frames.recording_time — in
KeyboardLayer::prepare (crates/rendering/src/layers/keyboard.rs). The parameter was
already passed in and ignored (_segment_frames).

Note for reviewers: keyboard segments baked into existing project-config.json files
are already on the recording clock (they're generated directly from event timestamps at
finalization), so this change corrects existing recordings too — no migration needed.

Verified on a Windows 11 machine: before the change, overlays lagged ~0.5s on an
uncut recording and stayed at absolute times after cuts (confirmed by exporting a
trimmed project and inspecting frames); after, the overlay clock matches the decoded
frame's recording time, the same contract the cursor layer already uses.

Known gap this PR does NOT address: the timeline UI's keyboard track still draws
segment blocks at raw times, so blocks don't visually slide on ripple edits even
though the rendered overlay is correct. That needs the same output↔source mapping in
the frontend track components — happy to file it separately.

🤖 Generated with Claude Code


Greptile Summary

The PR changes keyboard-overlay segment selection to use the decoded frame’s recording timestamp, aligning keyboard rendering with timeline cuts, trims, and encoder start offsets.

  • Uses DecodedSegmentFrames.recording_time instead of output-frame time.
  • Matches the clock already used by the cursor layer.
  • No schema or persisted-project migration is introduced.

Confidence Score: 5/5

The PR appears safe to merge with no actionable defects identified.

The keyboard overlay now consumes the decoded frame’s recording timestamp, consistent with timeline remapping and the cursor layer, and no blocking failure remains.

Important Files Changed

Filename Overview
crates/rendering/src/layers/keyboard.rs Replaces output-time keyboard lookup with the timeline-mapped recording timestamp; no concrete regression was established.

Reviews (1): Last reviewed commit: "fix(rendering): drive keyboard overlay f..." | Re-trigger Greptile

Context used:

…ows cuts and start offset

The keyboard overlay picked active segments with raw output time
(frame_number / frame_rate) while keyboard events are stamped on the
recording clock and video frames are mapped through the timeline. Result:
overlays lag every recording by the encoder start offset
(display.start_time, ~0.5s when the encoder is slow to start), and detach
entirely after cuts or trims.

Use segment_frames.recording_time instead - the same clock the cursor
layer already uses - which travels with the decoded frame through
get_frame_mapping and therefore follows cuts, trims and timescale, and
includes the recording->first-frame offset.

The parameter was already passed to prepare() and ignored.

Related to CapSoftware#2115 (findings 3 in the report).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant