test(transforms): cover all four shadow-mask directions deterministically - #2144
linhongyu510 wants to merge 1 commit into
Conversation
…ally create_shadow_mask picks one of four intensity gradients from an unseeded np.random.rand(1), so test_random_shadow exercises only one of them per run and the set of covered lines in transforms/functional/base.py changes between runs. Measured over 12 runs, the miss count for that file swings between 15 and 29, which shows up as spurious codecov/project deltas on unrelated PRs. Pin the draw with monkeypatch so every gradient is exercised on each run: the miss set becomes identical across all 12 runs, and line 180 (the quad_idx == 1 branch) is now covered.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2144 +/- ##
==========================================
+ Coverage 97.23% 97.25% +0.01%
==========================================
Files 169 169
Lines 10039 10039
==========================================
+ Hits 9761 9763 +2
+ Misses 278 276 -2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
The Short version: Checks that this is unrelated to the diff:
The other 8 workflows on this PR are green, including |
What's wrong
create_shadow_maskpicks one of four intensity gradients from an unseeded draw, and branches on it:test_random_shadowcalls it four times (its parametrization) with no seed, so each CI run reaches only some of those lines. Over 400 seeds the four quadrants come up 104 / 99 / 95 / 102 times, so which lines get counted is close to a coin flip.The visible effect is that coverage of
doctr/transforms/functional/base.pymoves on its own. Measured withcoverage.py, running the existing test 12 times and changing nothing at all:Four different miss patterns, and the file's miss count swinging between 15 and 29.
Why it matters beyond this file
That swing lands on unrelated pull requests as a spurious
codecov/projectfailure. Two examples from this repository's own history:main60cb8791a6f4c1c7f7218A docs-only commit moved project coverage by 2 misses / 0.02%.
The same thing hit #2138, where Codecov's per-file data attributes the whole project delta to this file even though the PR never touches it (
has_diff: false,patch: null, line count unchanged 68 → 68, misses 4 → 6) while all three files that PR does touch keep their miss counts exactly.Fix
Pin the two draws with
monkeypatchso all four gradients are exercised on every run. After the change, 12 repeated runs:One pattern, one count, every time — and line 180 (
quad_idx == 1) is now covered where the existing test left it to chance.I chose
monkeypatchovernp.random.seed(...)deliberately: a fixed seed only reaches whichever quadrant that seed happens to produce, and the mapping from seed to quadrant is an implementation detail of numpy's RNG stream that can change between versions. Patching the draw states the intent directly — "exercise the top-to-bottom gradient" — and keeps working if numpy's generator changes.The existing
test_random_shadowis left exactly as it is; this is an addition, not a replacement, so the end-to-end random path stays covered.Verification
Diff is
+37/-0in one test file; no source change.Each of the four cases asserts the mask shape, that values stay in
[0, 1], that the mask is a real gradient (min < max, not a constant plane), and that exactly two draws were consumed — so the test fails loudly ifcreate_shadow_maskever changes how it consumes randomness, rather than silently passing on a stale assumption.