Fix fill_alpha and outline_alpha being ignored in datashader render path#647
Merged
Fix fill_alpha and outline_alpha being ignored in datashader render path#647
Conversation
ds.tf.shade(min_alpha=...) is a floor on alpha for non-empty pixels, not a scaling factor. Passing fill_alpha as min_alpha left shapes at full opacity regardless of the user's choice. Post-multiply the alpha channel in the shade result, matching the existing fix-up pattern in _apply_cmap_alpha_to_datashader_result and the matplotlib path.
timtreis
changed the title
Narrow Any to ds.tf.Image | np.ndarray to match the codebase convention from _datashader_map_aggregate_to_color, and prefer truthiness over an explicit length check on the AxesImage list.
Same root cause as the fill_alpha fix: outline_alpha was passed only as ds.tf.shade(min_alpha=...), which is a floor on alpha for non-empty pixels rather than a scaling factor, so outlines rendered at full opacity regardless of outline_alpha. Apply the same _apply_user_alpha post-multiply after the outline shade, with a parametrized regression test. Note: outline_alpha=0.0 with an explicit outline_color is silently remapped to 1.0 by _set_outline (because the truthiness check `outline_alpha and outline_alpha == 0.0` is always False for 0.0). That is a separate latent bug and is left out of scope for this PR.
timtreis
changed the title
The arm `outline_alpha and outline_alpha == 0.0` short-circuited to False when outline_alpha was the literal 0.0 (falsy), so the 'user doesn't want outlines' branch was unreachable whenever outline_color was set. The user-facing symptom: passing outline_alpha=0.0 alongside an explicit outline_color silently rendered an outline at full opacity. Drop the truthiness gate; the equality check alone is correct (and naturally False for None or non-zero numerics). Adds a focused unit test on _set_outline and extends the datashader outline_alpha regression test to cover the 0.0 case.
Avoids 3 redundant get_array() calls per item via walrus.
timtreis
changed the title
Drop multi-paragraph regression-narrative blocks in favor of one-line docstrings naming the invariant under test. Strip restating-the-bug inline comments inside parametrize tables.
Both tests use alpha < 1.0 (0.6 and 0.8). Pre-fix, alpha was ignored and points rendered fully opaque; the references baked that in. The fix correctly scales alpha, so the layered yellow-on-blue plot now shows the expected color blend, and the continuous-color plot shows the expected partial transparency. Reference images taken from CI (visual_test_results_hatch-test.py3.14-stable, run 25553060603).
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #647 +/- ##
==========================================
+ Coverage 76.35% 76.37% +0.02%
==========================================
Files 11 11
Lines 3239 3251 +12
Branches 762 764 +2
==========================================
+ Hits 2473 2483 +10
- Misses 467 468 +1
- Partials 299 300 +1
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three related bugs all surface as
fill_alpha/outline_alphabeing silently ignored or wrong:fill_alphaignored._ds_shade_continuousand_ds_shade_categoricalpassed the user alpha intods.tf.shade(min_alpha=...).min_alphais a floor on the alpha of non-empty pixels, not a scaling factor — so for shapes (every covered pixel maps to full coverage) the result is always alpha=255 regardless offill_alpha.outline_alphaignored._render_ds_outlineshad the identical bug._set_outlinesilently remappedoutline_alpha=0.0to1.0. The armoutline_alpha and outline_alpha == 0.0short-circuited toFalsewheneveroutline_alphawas the literal0.0(falsy in Python). When a user pairedoutline_alpha=0.0with an explicitoutline_color, the 'invisible outline' branch was unreachable and the value got reassigned to(1.0, 1.0)further down.Because the matplotlib → datashader switch is silent at >10k shapes, large datasets diverged from small ones with no warning.
Fix
_apply_user_alpha(result, alpha)in_datashader.pythat post-multiplies the alpha channel of the shade result by the user-supplied alpha. Same pattern already used by_apply_cmap_alpha_to_datashader_result(issue Issue with set_zero_in_cmap_to_transparent Function Not Making Zero Values Transparent in Spatial Data Plotting #376). Applied in_ds_shade_continuous(bothshadedandnan_shaded),_ds_shade_categorical, and_render_ds_outlines. Carries over to the points datashader path automatically._set_outline's zero-alpha branch:outline_alpha == 0.0alone is correct (naturallyFalseforNoneor non-zero numerics).