fix(rendering): prevent edge apron artifact when window border or shadow is disabled

[melcheikh]

Summary

Fixes #2026

Resolves the thin rectangular outline artifact visible just outside recorded windows (most noticeable on Windows D3D11/D3D12 when window borders or shadows are disabled, or when motion/zoom blur is active):

1. Zero inactive border_width in CompositeVideoFrameUniforms and shader:

   • CompositeVideoFrameUniforms::default() previously initialized border_width to 5.0 even with border_enabled: 0.0, and ProjectUniforms passed the configured border.width into display/frame uniforms when border_on was false or delegated to frame_chrome. This inflated edge_padding to 7.0 px (5.0 + 2.0) around unbordered windows, matching the 7px apron perimeter seen in the Thin line visible just outside recorded window #2026 screenshots.
   • CompositeVideoFrameUniforms::default() now defaults border_width to 0.0, ProjectUniforms zeroes border_width whenever the border is disabled on that pass, and composite-video-frame.wgsl clamps border_w to 0.0 unless uniforms.border_enabled > 0.0.

2. Eliminate remaining non-uniform implicit derivatives (fwidth / textureSample) after early returns:

   • apply_rounded_corners is called inside the directional and zoom blur loops after the non-uniform apron and border early returns (and on per-pixel dithered UVs in zoom blur). It now calls rounded_rect_coverage_with_aa(..., 1.0) instead of rounded_rect_coverage so fwidth is never evaluated across exited helper lanes or dithered UVs.
   • sample_texture now uses textureSampleLevel(frame_texture, frame_sampler, ..., 0.0) consistently across all branches instead of textureSample(...) inside non-uniform control flow (frame_texture is single-mip).

3. Guard zero-span shadow evaluation and custom smoothstep:

   • Avoids 0.0 / 0.0 (NaN) in the custom smoothstep function and skips shadow evaluation when shadow == 0.0 or shadow_opacity <= 0.0001.

Verification

• Added unit and WGPU render regression tests in crates/rendering/src/composite_frame.rs and crates/rendering/src/lib.rs (default_uniforms_have_zero_border_width_and_disabled_border, uniforms_zero_border_width_when_border_disabled_or_delegated_to_frame).
• Verified with cargo clippy -p cap-rendering --all-targets -- -D warnings and cargo test -p cap-rendering.

---

cc @richiemcilroy: if you find this fix helpful, a maintainer tip via @algora-pbc would be greatly appreciated!

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking request to check rendered edge pixels.

Findings

1.  Outline can escape the test ▶

Fix with agent prompt

### Issue 1
crates/rendering/src/lib.rs:6133
The new render check only asserts that `preview.data` is nonempty. A frame with the unwanted outline, wrong alpha, or broken shadow still passes. Keep the uniform assertions, but add pixel checks around a window with borders and shadows disabled, including a moving or zooming case. That would cover the shader changes rather than only successful rendering.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Prevents the window-edge outline by zeroing inactive border widths and avoiding shader operations that depend on neighboring pixels after early returns.

• Recorded windows no longer reserve extra edge padding for disabled borders.
• Blur and shadow passes use stable edge calculations.

Review was based on source inspection; no tests or visual captures were run.

_{Reviews (1) · Last reviewed commit: "fix(rendering): prevent edge apron artif..." · Reviewed by Greptile}

[vercel]

@melcheikh is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Outline can escape the test

The new render check only asserts that preview.data is nonempty. A frame with the unwanted outline, wrong alpha, or broken shadow still passes. Keep the uniform assertions, but add pixel checks around a window with borders and shadows disabled, including a moving or zooming case. That would cover the shader changes rather than only successful rendering.

Prompt To Fix With AI

This is a comment left during a code review.
Path: crates/rendering/src/lib.rs
Line: 6133

Comment:
**Outline can escape the test**

The new render check only asserts that `preview.data` is nonempty. A frame with the unwanted outline, wrong alpha, or broken shadow still passes. Keep the uniform assertions, but add pixel checks around a window with borders and shadows disabled, including a moving or zooming case. That would cover the shader changes rather than only successful rendering.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[richiemcilroy]

Thanks for this! Before we can review it further it needs a Greptile confidence score of 5/5. Could you address Greptile's remaining comments and push an update? Once it's at 5/5 we'll run the full CI and regression suites on it.