feat(encode): extract and mix embedded video audio#55
Merged
Conversation
Video components' soundtracks now reach the output mix — frames-only extraction silently dropped them and the volume field was never read. - pure scene_start_offsets replicates build_frame_tasks placement (transition overlaps, inter-view transitions) — unit-tested - ffmpeg -vn extraction honoring trim_start/trim_end and playback_rate (chained atempo beyond the 0.5..2.0 per-instance limit), cached in temp_dir by content hash; no ffmpeg => warning + skip - synthesized AudioTracks (start/end/volume) appended before the existing mixer on both encode paths (openh264 mux and ffmpeg) - documented v1 limits: world views skipped, loop_video audio plays once
This was referenced Jul 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #44.
Goal: a
videocomponent's soundtrack reaches the output mix — a talking-head embed no longer renders silent (itsvolumefield existed and was never read).scene_start_offsetsreplicatesbuild_frame_tasksplacement exactly — scene-transition overlaps and inter-view transitions unit-tested ([0.0, 1.5]with an incoming 0.5s fade; view 1 at 2.5s across a view transition)-vnextraction honoringtrim_start/trim_end/playback_rate(chainedatempofor rates beyond 0.5–2.0), temp-dir cache keyed by content hash, graceful skip with warning when ffmpeg is absentAudioTracks appended before the unchanged existing mixer, wired on both encode paths (openh264 mux + ffmpeg);volume: 0skips extraction; nested videos found through containersClonederived onAudioTrackloop_videoaudio plays onceTests: 13 new (timeline offsets ×3, atempo chains ×5, collection rules ×3, ffmpeg-gated integration with a generated sine fixture asserting
start=1.2). 235 total, all green.Verify:
cargo test --workspace→ 235 passed ✓ ·cargo fmt --check✓ · clippy 0 warnings ✓