fix(rendering): drive keyboard overlay from recording_time so it follows cuts and encoder start offset - #2118
Open
aacarcrash wants to merge 1 commit into
Conversation
…ows cuts and start offset The keyboard overlay picked active segments with raw output time (frame_number / frame_rate) while keyboard events are stamped on the recording clock and video frames are mapped through the timeline. Result: overlays lag every recording by the encoder start offset (display.start_time, ~0.5s when the encoder is slow to start), and detach entirely after cuts or trims. Use segment_frames.recording_time instead - the same clock the cursor layer already uses - which travels with the decoded frame through get_frame_mapping and therefore follows cuts, trims and timescale, and includes the recording->first-frame offset. The parameter was already passed to prepare() and ignored. Related to CapSoftware#2115 (findings 3 in the report). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The keyboard overlay uses
frame_number / frame_rate(raw output time) to pick activekeyboard segments, while keyboard events are stamped on the recording clock and the
video layer maps output time through the timeline. Two visible desyncs result:
display.start_timeon the recording clock (measured 0.4998s on a machine wherethe encoder pre-flight fell back to software x264). Keystroke overlays render that
much later than the typing visible in the video, on every recording.
frames but not the keyboard clock, so overlays drift by the length of whatever was
removed. The cursor layer doesn't have either problem because it uses
segment_frames.recording_time, which travels with the decoded frame throughget_frame_mapping.Fix: use the same clock as the cursor layer —
segment_frames.recording_time— inKeyboardLayer::prepare(crates/rendering/src/layers/keyboard.rs). The parameter wasalready passed in and ignored (
_segment_frames).Note for reviewers: keyboard segments baked into existing
project-config.jsonfilesare already on the recording clock (they're generated directly from event timestamps at
finalization), so this change corrects existing recordings too — no migration needed.
Verified on a Windows 11 machine: before the change, overlays lagged ~0.5s on an
uncut recording and stayed at absolute times after cuts (confirmed by exporting a
trimmed project and inspecting frames); after, the overlay clock matches the decoded
frame's recording time, the same contract the cursor layer already uses.
Known gap this PR does NOT address: the timeline UI's keyboard track still draws
segment blocks at raw times, so blocks don't visually slide on ripple edits even
though the rendered overlay is correct. That needs the same output↔source mapping in
the frontend track components — happy to file it separately.
🤖 Generated with Claude Code
Greptile Summary
The PR changes keyboard-overlay segment selection to use the decoded frame’s recording timestamp, aligning keyboard rendering with timeline cuts, trims, and encoder start offsets.
DecodedSegmentFrames.recording_timeinstead of output-frame time.Confidence Score: 5/5
The PR appears safe to merge with no actionable defects identified.
The keyboard overlay now consumes the decoded frame’s recording timestamp, consistent with timeline remapping and the cursor layer, and no blocking failure remains.
Important Files Changed
Reviews (1): Last reviewed commit: "fix(rendering): drive keyboard overlay f..." | Re-trigger Greptile
Context used: