perf(rendering): cut Metal command buffers 70% and block on frame submissions

[harshav167]

Summary

RendererLayers::render draws every editor preview, playback, and export frame. Before this branch, macOS frames paid for empty Metal command buffers, one render pass per layer, repeated static-background blurs, and frame waits that slept in whole timer ticks.

This PR removes the empty buffers, lets consecutive same-target layers share a render pass, enables the existing blurred-background cache outside the render farm, and waits on the frame's actual submission instead of sleep-polling. Output stays identical in the checks below.

Headline results on an M4 Max:

• Metal command buffers: 19.4 to 5.8 per frame (-70%).
• 1080p MP4 export CPU: median -10.9% and -11.1% on two recordings, lower in all 10 paired rounds. Wall time stays encoder-bound.
• Repository performance suite render p50: 3.843 ms on main to 1.650 ms here (-57.1%), with 7.1% less suite CPU.
• GPUI playback: same frames painted in the final comparison, no drops, 13.7% less CPU in debug builds.

Review route

Start with the four production changes in this order:

1. vendor/wgpu-hal/src/metal/{command,mod,device}.rs: lazy MTLCommandBuffer creation and the fence signal move. Then vendor/wgpu-hal/src/dx12/device.rs: one event per fence wait.
2. crates/rendering/src/lib.rs: LayerPasses, the layer order in RendererLayers::render, and the blur-cache key. The three layer files only relax render-pass lifetimes.
3. crates/rendering/src/frame_pipeline.rs: session texture COPY_DST, wait_for_submission, PendingReadback, and PendingSurface.
4. apps/desktop-gpui/src/camera_blur.rs and crates/rendering/examples/preview-surface-benchmark.rs: callers now pass the submission that produced the surface.

Tests are concentrated in crates/rendering/src/overlay_layers/native_tests.rs. The two findings files carry the raw benchmark tables and session history, so this description only repeats the numbers needed to judge the change.

What changed

Stop committing empty Metal command buffers

wgpu-core opens one HAL command buffer for each pass and another for barriers ahead of it. Barriers do nothing in the Metal backend, so many of those buffers were created and committed empty. The backend now creates the MTLCommandBuffer when the first encoder starts, skips empty buffers at submit, and signals the fence on the last encoded buffer. begin_debug_marker still creates the buffer so a debug group pushed before a pass is popped from the same buffer.

This follows the direction suggested in gfx-rs/wgpu#5738. It is also the first local edit to the vendored wgpu-hal; a comment at the patch site says to reapply it when that copy is replaced.

Share one render pass across same-target layers

LayerPasses keeps a pass open while consecutive layers load from and draw into the same texture. It closes the pass for a target switch, a clear, a mask, a blur-cache copy, an encoder-level operation, or a texture swap. Draw order does not change.

Each removed boundary saves one wgpu-core command buffer and encoder. It also avoids a full-attachment store and reload on tile-based GPUs, although that part was not measurable in GPU busy time on this machine. The measured gain is CPU and driver submission work.

Sharing a pass adds one rule for layer authors: a layer that changes scissor, viewport, blend constant, or stencil state must restore it before returning. Captions and keyboard are the only current scissor users, both already reset it, and a regression test now covers the contract.

Reuse a static blurred background

The render farm already had a BlurResultKey cache for image, wallpaper, colour, and gradient backgrounds. The editor and desktop exports now use it too, and the process-wide opt-in flag is deleted. The key covers background generation, blur amount, output size, and session texture size. Animated backgrounds never use the cache. Session textures therefore always carry COPY_DST.

The cache holds one texture the size of the renderer's output while a static background is blurred (about 33 MB for a 4K export renderer; preview-sized in the editor) and releases it on the first frame without blur. That memory buys skipping two full-resolution gaussian passes on every frame with a static blurred backdrop.

Turning the cache on in the editor also exposed a pre-existing bug, fixed here: the blur layer only refreshed its strength while blur was above zero, so setting blur to 0 on a live renderer left the previous strength in effect. Strength now refreshes every frame.

Block on the frame's submission

PendingReadback::wait and the macOS preview's PendingSurface::wait used to poll, yield nine times, sleep 100 µs, then sleep 1 ms. tokio's timer rounds the 100 µs sleep to about 1 ms, so completion could wait for the next tick even when the GPU had finished.

Both waits now use wait_for_submission, which runs device.poll(WaitForSubmissionIndex) in spawn_blocking under the existing GPU wait timeout. PendingReadback stores the index returned by its readback submit. PendingSurface::wait now takes the index for the encoder that wrote its surface; its in-tree callers in the editor, camera-blur preview, tests, and surface benchmark pass the index from their own submit. This follows the shape #2541 uses for readbacks.

The MP4 NV12 surface waits are untouched. #2464 owns those.

Results

Unless a section says otherwise, measurements are release builds on an M4 Max running macOS 27.2. The two Studio recordings are 2542x1226 with background blur, shadow, zoom, and motion blur, and 1260x860 with captions and keyboard overlays.

Submission work

Metal System Trace, 10-second windows, cap process only, blur recording at 1080p:

Build	Command buffers/frame	Encoders/frame	GPU busy/frame
main	19.4	16.6	744-858 µs
+ HAL change	10.6	16.6	739 µs
+ pass merging	7.6	10.7	736 µs
+ blur cache	5.8	7.0	527-731 µs

Buffer and encoder counts repeat exactly. GPU busy varies by roughly a quarter between identical runs at about 20% load, so the table gives ranges. At 4K, pass merging cuts buffers from 10.2 to 7.2 and encoders from 16.9 to 11.0, with GPU busy unchanged within noise. Render passes fall from 6 to 1 per frame with blur and from 4 to 1 without it.

Export CPU and profile

Five interleaved rounds per recording, each round compared with itself because absolute CPU drifted between rounds:

Recording	Median CPU change	Range
Blur	-10.9%	-23.1% to -6.8%
No blur	-11.1%	-18.0% to -7.3%

The branch used less CPU in all 10 rounds. MP4 wall time did not move on this machine. A standalone h264_videotoolbox encode of the same frames takes as long as the 1080p and 4K exports, so VideoToolbox is the limit.

A 10-second Time Profiler sample of the 1080p blur export puts numbers to the mechanism. Total sampled CPU fell from 3,589 ms to 2,937 ms (-18%). Command-buffer submission fell from 905 ms to 630 ms, other Metal/AGX/IOGPU driver work from 658 ms to 468 ms, and wgpu from 231 ms to 198 ms. VideoToolbox/CoreMedia (712 to 705 ms) and FFmpeg (247 to 228 ms) stayed flat. The changed waits are not on the MP4 NV12 path.

GIF uses the RGBA readback wait, so it was measured again with the blocking wait: 5 of 5 paired rounds used less CPU than main (median -1.5%), instructions fell 0.5%, and all ten outputs were byte-identical. GIF remains encoder-bound. Fifteen earlier rounds without the wait fix gave no defensible wall-time claim, so none is made here.

Repository performance suite

cap-test suite performance --duration 8 on cap-performance-fixtures, seven interleaved triplets (main, branch without the wait change, branch with the blocking wait), all 21 runs passing:

Metric, branch + blocking wait vs main	Result
Render p50	1.650 ms vs 3.843 ms (-57.1%, 7/7 lower)
Render mean / p95 / minimum	-40.1% / -26.2% / -46.6%
Playback throughput	+44.3%
Suite CPU	-7.1%
Instructions retired	-2.1%

This suite is also how the wait problem showed up. Before the wait change, the branch's render p50 landed in a 3.4-3.7 ms band in 5 of 7 pairs (median +24.8%) while mean, p95, and throughput were flat. A diagnostic build using the existing NV12 spin rule in both builds removed the bands, which identified the timer tick as the cause. That spin is not shipped: it used 10.7% more suite CPU than main. The blocking wait keeps the latency result and uses 3.7% less CPU than the no-fix branch in the same session.

The blocking build's p95 still splits around 4.2 and 5.3 ms. That looks like the vendored Metal HAL's own 1 ms sleep inside Device::wait; it has not been isolated. An event-driven HAL wait is possible follow-up work. It is not in this PR.

GPUI app

apps/desktop-gpui has its own workspace and resolves the same wgpu-core 25.0.2 and vendored wgpu-hal. GPUI's UI path uses wgpu 29 and is separate from these changes.

Debug cap-gpui builds of main and this branch ran the app's CAP_GPUI_AUTO_* harness on a copy of the blur recording:

• Playback painted 238, 239, and 240 frames across the three final branch runs, against 239, 236, and 237 on main, with no drops in any final branch run and 13.7% less CPU.
• Seeking results were mixed by counter: submitted paints were 4.7% lower and painted frames 5.5% lower on average, while paint rate was 0.4% lower and the branch dropped no frames against main's 13. Mean seeking CPU was 5.8% higher, carried by the first cold launch; the median was 16% lower. Instructions retired was not available for the final run because gtimeout sat between the app and time -l.
• An earlier branch set painted 12.5% fewer frames while seeking and had one 57-frame playback drop. Neither result recurred in the final set. The cause was not isolated.
• Before the wait change, an in-editor MP4 export produced 2,808 frames whose decoded output matched CLI exports from main and the branch byte for byte. Dragging Background Blur from 12% to 59% also repainted the preview live and played back at 60.0 fps after reopening.

No warnings, validation errors, or panics were logged in those runs. Release GPUI builds were not measured.

Correctness

• 1080p CLI exports from main and this branch decode to identical frames (ffmpeg -f framemd5): 5,616/5,616 on the blur recording, 2,712/2,712 on the captions/keyboard recording, and 5,616/5,616 for each blurred image, wallpaper, gradient, colour, and mid-export style-segment variant. GIF outputs after the wait change were byte-identical in all five pairs.
• layers_sharing_a_pass_render_like_one_pass_per_layer requires byte-identical output from merged and split rendering across overlays, blurred backgrounds, masks, camera3d gaussian and bokeh focus blur, colour grade, device frame and notch, camera overlay, camera-only transition, and all of them together. Its one-pass-per-layer reference uses a #[cfg(test)] split switch that is compiled out of release builds. That test-only state is the main structural wart in the diff; it earns its keep by making the reference exact, but reviewers should look at it directly.
• scissored_overlays_leave_the_full_target_to_the_next_layer_in_their_pass places an image after keyboard or captions in one pass and checks that it is not clipped by a stale scissor. On the WARP family only, the separate "frame changed" sanity check is skipped when that adapter leaves the frame unchanged, matching the existing notch test. The merged-versus-split equality check always runs.
• cached_background_blur_matches_a_fresh_blur_and_follows_edits covers blur changes, background changes, an output resize and back, and the cache releasing when blur turns off.
• Each new or extended test was shown to fail under a targeted mutation: removing either scissor reset, forcing a continued pass to inherit a 1x1 scissor, dropping sizes from the blur-cache key, or removing the cache release.
• cargo test --lib: cap-rendering 308, cap-editor 191, cap-export 42, cap-camera-effects 27. The ignored GPU tests (text-layer ordering, both animated-gradient tests) pass on the final tree, as does export_sequence_render_matches_editor_reference_for_real_recording at 1080p and 4K on copies of both real recordings.
• Local gates: cargo fmt --all --check; cargo clippy --workspace --exclude cap-desktop --all-features --locked -- -D warnings on Rust 1.88.0; cap-rendering clippy with --all-targets. In apps/desktop-gpui, cap-gpui tests pass (1,155 passed, 10 ignored) and cap-gpui-kernels passes (30), on Rust 1.96.1 where CI uses 1.95.0.

API changes and platform risk

• cap_rendering::enable_blur_result_cache is removed. The render farm's call is removed with it.
• Text, captions, and keyboard render methods now take &self and any RenderPass lifetime, instead of tying the layer and pass lifetimes together.
• macOS-only PendingSurface::wait takes a SubmissionIndex instead of &wgpu::Queue. Callers must pass the submission that wrote the surface. All in-tree callers were updated and searched again after the GPUI build exposed two outside the first search.
• The blocking wait can stall another submit on the same device until the waited submission finishes, because wgpu-core 25 holds the fence read lock across the HAL wait while submit takes the write lock. The queue is in-order, so the other renderer's GPU work could not have started sooner; the loss is CPU-side submit overlap. On a hung GPU, the caller regains control at its existing timeout, but the blocking thread can retain the lock until wgpu-core's 60-second cleanup wait expires.
• Windows: PendingReadback::wait is cross-platform and reaches DX12 Device::wait. The vendored DX12 wait originally shared one auto-reset event, which loses wakeups with concurrent waiters; this branch backports the per-wait event fix from Create a seperate event for each fence wait. gfx-rs/wgpu#8273, byte-identical to perf(windows): fence-waited readbacks, overlapped MF export decode, cheaper screen capture #2541's vendored file. It does not carry upstream's present-queue idle change or the shared event's removal, which sit on the surface present/resize path rather than these offscreen waits. The Windows compile is CI-only evidence: no Windows target is installed on the test machine. Linux Vulkan uses a timeline semaphore and is safe with concurrent waiters; the GLES wait takes the GL context lock, and which Linux backend ships was not verified.
• The camera-blur preview is a third PendingSurface path. It has its own device and thread, so it has no shared-device stall exposure, but its timing change was not measured.
• perf: wait for NV12 export surfaces without millisecond sleeps #2464 touches the same file in the NV12 waits and merge-trees cleanly against this branch. perf(windows): fence-waited readbacks, overlapped MF export decode, cheaper screen capture #2541 conflicts only in PendingReadback::wait because it makes the same design change for Windows readbacks; the DX12 files are identical, so whichever lands second has a mechanical rebase.

Reproduce

node scripts/setup.js
LIBCLANG_PATH=/Library/Developer/CommandLineTools/usr/lib cargo build --release -p cap
# build main the same way and keep the binary as the baseline
mkdir -p target/bench

# export CPU
/usr/bin/time -p ./target/release/cap export <project>.cap -o target/bench/out.mp4 --fps 60 --resolution 1920x1080 --quality maximum

# command buffers, encoders and GPU busy per frame
xcrun xctrace record --template 'Metal System Trace' --time-limit 10s --output target/bench/t.trace --launch -- \
  ./target/release/cap export <project>.cap -o target/bench/out.mp4 --fps 60 --resolution 1920x1080 --quality maximum
xcrun xctrace export --input target/bench/t.trace --output target/bench/cb.xml \
  --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-application-command-buffer-submissions"]'
xcrun xctrace export --input target/bench/t.trace --output target/bench/gpu.xml \
  --xpath '/trace-toc/run[@number="1"]/data/table[@schema="metal-gpu-intervals"]'
# frames = "RGBA to NV12 Conversion" submissions; filter gpu.xml rows to the cap process
# xctrace leaves a 1-5 GB kernel trace per recording in $TMPDIR and ignores TMPDIR:
rm -f "$TMPDIR"/instruments*.ktrace

# pixel equivalence
ffmpeg -v error -i before.mp4 -map 0:v -f framemd5 target/bench/before.md5
ffmpeg -v error -i after.mp4  -map 0:v -f framemd5 target/bench/after.md5

# repository suite (clone CapSoftware/cap-performance-fixtures first)
cargo run --release -p cap-test -- suite performance --recording-path <fixture>.cap --duration 8

LIBCLANG_PATH matters only when Homebrew LLVM is first on PATH; bindgen 0.70 mis-parses FFmpeg's headers with libclang 23.

Not in this PR

• About six command buffers per frame remain: one per remaining pass, PendingWrites, and copy segments. Folding those into one needs pass coalescing in wgpu-core (gfx-rs/wgpu#10506), which is not vendored here.
• An event-driven Metal HAL wait (MTLSharedEvent listener plus a condvar) would remove the residual p95 split caused by the HAL's own 1 ms sleep. That is a second vendored-HAL patch and is left as follow-up work.
• VideoToolbox bounds MP4 export wall time on the test machine. No encoder settings change here.

Confidence Score: 5/5

The PR appears safe to merge; no actionable new defect was established.

Summary

This PR reduces frame submission work and changes how rendered frames wait for the GPU.

• Consecutive layers that draw to the same texture share a render pass.
• Static blurred backgrounds are reused across rendering workflows.
• Frame readbacks and macOS preview surfaces wait for their own submission.

No actionable new findings were established. Builds, GPU tests, and visual checks were not run during this review.

harshav167 explicitly acknowledges the loss of CPU-side submit overlap on shared devices and the longer lock hold after a hung GPU. Those trade-offs are not reported again.

_{Reviews (4) · Last reviewed commit: "test(rendering): blur a checkerboard in ..." · Reviewed by Greptile}

[vercel]

@harshav167 is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

[richiemcilroy]

Thanks @harshav167, these numbers are great and we would like to get this in. It now conflicts with main in crates/rendering/src/frame_pipeline.rs, where the readback waits changed since you branched:

• fix(export): keep offline export frames in order when decode or readback is slow #2636 added export deadlines: gpu_buffer_wait_timeout(export_deadlines) and the readback_delay test seam that readback_waited uses;
• perf(rendering): wake macOS surface exports on Metal completion instead of polling #2648 made macOS surface exports wait on queue.on_submitted_work_done instead of polling.

Your wait_for_submission replaces both waits, so the merge is a design choice rather than a mechanical one. Could you rebase onto main and keep the export deadline behaviour (the tests in frame_pipeline.rs cover it)? Once it is rebased and Greptile is still 5/5, we will run the full Engine verify suite and the perf budget on it before merging.

[harshav167]

Thanks @richiemcilroy — rebased onto main at cc3f99511 and pushed; the head is now a1ef731a9. It still merges cleanly with the latest main (6ba5e2e15) by merge-tree; the gates below were run at the cc3f99511 base.

Export deadlines are preserved. wait_for_submission receives the wait's start time, frame number, and deadline. It calculates the remaining time with readback_waited(started, frame_number) on the calling thread and checks it again after the GPU signals. PendingReadback passes gpu_buffer_wait_timeout(self.export_deadlines), so offline exports use EXPORT_STALL_LIMIT; PendingSurface passes gpu_buffer_wait_timeout(false), as on main. wgpu's internal 60-second PollError::Timeout is retried until the caller's deadline expires. The blocking poll runs on a blocking thread, but readback_waited stays on the calling thread, preserving the thread-local readback_delay seam.

export_order_tests pass unchanged, including slow_readback_keeps_every_frame_in_order (61 s) and stalled_readback_stops_the_export_instead_of_dropping_the_frame (1 h). Replacing readback_waited with started.elapsed() makes the stalled-readback test fail, so the seam is covered.

#2648 is untouched. It changed PendingNv12Surface, the NV12 surface MP4 export hands to VideoToolbox. This PR changes PendingSurface, the separate BGRA editor-preview surface, and the cross-platform RGBA readback. Main's NV12 marker/oneshot wait and surface_completion_wakes_without_device_polling remain as-is. If you'd prefer PendingSurface to use #2648's marker pattern for consistency, I'm happy to align it; the readback would still need the blocking poll for its map callbacks.

On the rebased head, fmt and the exact CI clippy command are clean on aarch64-apple-darwin; lib tests pass (cap-rendering 335, cap-editor 191, cap-export 42, cap-camera-effects 27), as do cap-gpui (1,183) and cap-gpui-kernels (30). The DX12 backport also type-checks locally for x86_64-pc-windows-msvc; it has not been linked or run on Windows.

Two overlap corrections to the original description: #2464 currently conflicts with main itself after #2636/#2648, independent of this branch. #2541 merges cleanly with main and conflicts with this PR only in PendingReadback::wait, where both use the same design.

The five workflow runs for a1ef731a9 are still waiting for approval: CI 39090291139, Engine verify 39090291151, Perf budget 39090291135, A/V Sync Tests 39090291152, and Verify recording 39090291157. Could you approve them when you get a chance?

@greptileai please re-review

[richiemcilroy]

Pushed f152c12 to fix the sync-tests build: crates/rendering/tests/background_resize.rs (added by #2611 after this branch was cut) still called enable_blur_result_cache, which this PR removes. The cache is always on now, so the call is simply dropped. Export A/B on main vs this branch: every frame identical on three real recordings (including a 60% blurred wallpaper) at 1080p and 4K, instructions 3.4-5.4% lower, wall time and peak memory unchanged.

[richiemcilroy]

@greptileai review

[richiemcilroy]

Verified: frame-identical output vs main, CI and engine/real-capture suites green, Greptile 5/5. Thanks!