Skip to content

fix: preserve render audio intent#580

Closed
miguel-heygen wants to merge 1 commit intomainfrom
fix/audio-video-render-duration
Closed

fix: preserve render audio intent#580
miguel-heygen wants to merge 1 commit intomainfrom
fix/audio-video-render-duration

Conversation

@miguel-heygen
Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen commented May 1, 2026

Problem

Studio previews can be correct while the rendered MP4 has bad audio timing or the wrong mix for A-roll projects that use a muted visual <video> plus a separate <audio> track.

The Studio Export button in local Vite preview could also fail before rendering because the dev adapter accidentally called a cached producer import promise like a function.

What this fixes

  • Stop synthesizing data-has-audio="true" on every <video> during timing compilation.
  • Only mix embedded video audio when the video explicitly declares data-has-audio="true".
  • Clamp final audio muxing to the exact encoded frame duration, avoiding -shortest dropping the last captured frame because of AAC padding.
  • Fix Studio dev export by caching the retrying producer module loader function instead of invoking the cached promise.

Root cause

The timing compiler promoted muted visual videos into audible render inputs. In projects with separate audio, the renderer could then mix the embedded video audio as an extra source even though browser preview keeps the muted video silent. Separately, final FFmpeg muxing used -shortest, which could cut one captured frame when the AAC sidecar carried encoder padding beyond the video stream.

For the Studio Export button, packages/studio/vite.config.ts stored _producerModulePromise = createRetryingModuleLoader(... )() but later did return _producerModulePromise(). That throws _producerModulePromise is not a function when the render job starts.

Verification

Local

  • bun run --filter @hyperframes/core test -- src/compiler/timingCompiler.test.ts
  • bun run --filter @hyperframes/engine test -- src/services/chunkEncoder.test.ts
  • bun test packages/producer/src/services/htmlCompiler.test.ts
  • bunx --bun vitest run packages/producer/src/services/renderOrchestrator.test.ts
  • bun run --filter @hyperframes/studio test -- vite.producer.test.ts
  • bun run --filter @hyperframes/core typecheck
  • bun run --filter @hyperframes/engine typecheck
  • bun run --filter @hyperframes/producer typecheck
  • bun run --filter @hyperframes/studio typecheck
  • bunx oxfmt --check packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/timingCompiler.test.ts packages/engine/src/index.ts packages/engine/src/services/chunkEncoder.ts packages/engine/src/services/chunkEncoder.test.ts packages/producer/src/services/chunkEncoder.ts packages/producer/src/services/compilationTester.ts packages/producer/src/services/htmlCompiler.test.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/timingCompiler.ts packages/studio/vite.config.ts
  • bunx oxlint packages/core/src/compiler/timingCompiler.ts packages/core/src/compiler/timingCompiler.test.ts packages/engine/src/index.ts packages/engine/src/services/chunkEncoder.ts packages/engine/src/services/chunkEncoder.test.ts packages/producer/src/services/chunkEncoder.ts packages/producer/src/services/compilationTester.ts packages/producer/src/services/htmlCompiler.test.ts packages/producer/src/services/renderOrchestrator.ts packages/producer/src/services/timingCompiler.ts packages/studio/vite.config.ts
  • git diff --check
  • bun packages/cli/src/cli.ts validate /Users/miguel07code/Downloads/hyperframes-roll-overlay
  • bun packages/cli/src/cli.ts validate /tmp/hyperframes-different-repro/hyperframes-different

Render repros:

  • /Users/miguel07code/Downloads/hyperframes-roll-overlay rendered to /tmp/hf-roll-overlay-fixed2.mp4: final MP4 has 500 video frames, 16.666667s video duration, and 16.666667s container duration.
  • /tmp/hyperframes-different-repro/hyperframes-different rendered to /tmp/hf-different-fixed.mp4: compiled metadata reports audioCount: 1, final MP4 has 537 video frames, 17.900000s video duration, 17.900000s audio duration, and 17.900000s container duration.
  • Studio Export button render in local Vite preview completed to packages/studio/data/renders/hyperframes-different_2026-05-01_20-13-36.mp4 with 537 video frames and 17.900000s audio/video/container duration.

Browser

  • Ran Studio preview for /tmp/hyperframes-different-repro/hyperframes-different.
  • Exercised the preview with agent-browser: opened http://localhost:5194, captured a screenshot, clicked Play, captured a playing screenshot, and stopped a recording.
  • Exercised the Studio Export button with agent-browser: opened http://localhost:5194/#project/hyperframes-different, opened the Renders panel, selected Draft/MP4, clicked Export, captured the rendering state, and verified the render completed.
  • Local proof artifacts:
    • /tmp/hf-audio-render-proof/browser/different-preview.png
    • /tmp/hf-audio-render-proof/browser/different-preview-playing.png
    • /tmp/hf-audio-render-proof/browser/different-preview.webm
    • /tmp/hf-audio-render-proof/browser/studio-export-rendering.png
    • /tmp/hf-audio-render-proof/browser/studio-export-complete.png
    • /tmp/hf-audio-render-proof/browser/studio-export-fixed.webm

Notes

The second supplied project still has authored composition warnings unrelated to this fix: composition_file_too_large for compositions/mograph-overlay.html and contrast warnings from validation. This PR leaves those project-content issues unchanged.

@miguel-heygen miguel-heygen force-pushed the fix/audio-video-render-duration branch from b719597 to ece625c Compare May 1, 2026 00:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant