IsHelperLane() emulation pre SM 6.6 is broken when using sample-rate shading.
The existing workaround checks for input coverage being non-zero to determine if the lane is a helper or not, but this breaks if there are sibling samples which are in-fact not helpers. A helper invocation will be treated as not being a helper by this check, which could lead to catastrophic hang behavior in some cases.
This test uncovers the broken behavior: https://github.com/HansKristian-Work/vkd3d-proton/blob/master/tests/d3d12_pso.c#L5154
D3D11.3 spec is a little unclear if sample-rate shading should receive coverage for samples which are not part of the current pixel, but all real world implementations seem to do that at least, including WARP. Vulkan defines the sample mask to only contain the currently shaded samples when per-sample shading is used, so that's an interesting behavioral difference (which would have avoided this bug), but not super relevant here.
IsHelperLane() emulation pre SM 6.6 is broken when using sample-rate shading.
The existing workaround checks for input coverage being non-zero to determine if the lane is a helper or not, but this breaks if there are sibling samples which are in-fact not helpers. A helper invocation will be treated as not being a helper by this check, which could lead to catastrophic hang behavior in some cases.
This test uncovers the broken behavior: https://github.com/HansKristian-Work/vkd3d-proton/blob/master/tests/d3d12_pso.c#L5154
D3D11.3 spec is a little unclear if sample-rate shading should receive coverage for samples which are not part of the current pixel, but all real world implementations seem to do that at least, including WARP. Vulkan defines the sample mask to only contain the currently shaded samples when per-sample shading is used, so that's an interesting behavioral difference (which would have avoided this bug), but not super relevant here.