Repository navigation
Conversation
rwb27
force-pushed
the
diff-coverage-action
branch
2 times, most recently
from
November 9, 2026 16:24
cd19545 to
66fa048
Compare
This comment was marked as outdated.
This comment was marked as outdated.
rwb27
force-pushed
the
diff-coverage-action
branch
from
November 9, 2026 22:37
4a8a2bc to
45330e0
Compare
bprobert97
reviewed
Nov 18, 2026
rwb27
force-pushed
the
diff-coverage-action
branch
2 times, most recently
from
November 18, 2026 13:51
ea3cae2 to
e2d02ca
Compare
This uses the popular `5monkeys/coberatura-action` to show coverage in PRs. It doesn't rely on external services. I hope it will be better than our current solution, as I think it is able to check coverage of the diff rather than just global coverage. I added an xml coverage report to make this work. Rather than have an ever-growing list of reports in `pyproject.toml` I have removed those options from the pytest section. Instead, coverage reports should now be specified when they're needed. We only really need the xml report here, but I've left the others in just in case they're useful in the future. I've switched to using the VSCode tests panel as it puts coverage bars in my editor. Other developers are welcome to switch on whatever coverage reports they prefer. Turning off coverage when I run it at the command line takes tests from ~30s to ~20s on my system.
I hope this does a line-by-line diff.
I need to check out the repo in order to generate the git diff, even though it's not needed to run the tests.
The coverage badge is reinstated. I removed the code quality upload, as it's reserved for premium tiers.
rwb27
force-pushed
the
diff-coverage-action
branch
from
November 18, 2026 14:37
e2d02ca to
d59d47d
Compare
I've restored the base coverage job, and added steps to the coverage job that compare total coverage of this branch to `main`. The job will now fail if coverage has dropped by more than 0.2%. I no longer feel the need to set the threshold at 0. I've also set the test job to run on the `main` branch, so that we can get the badge updating again.
rwb27
force-pushed
the
diff-coverage-action
branch
2 times, most recently
from
November 18, 2026 14:57
645a61f to
cbff0ff
Compare
bprobert97
reviewed
Nov 18, 2026
| echo "delta=$change" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Check coverage tolerance | ||
| if: ${{ steps.coverage-delta.outputs.delta < -0.2 }} |
Contributor
There was a problem hiding this comment.
.totals.percent_covered_display is rounded to coverage.py’s precision setting. That defaults to 0 and .coveragerc doesn’t change it, so both numbers are whole percentages. A real drop of 0.2 points across a rounding boundary (94.6 to 94.4) appears as −1 and fails. A real drop of 0.9 (95.4 to 94.5) can appear as 0 and pass. We can either use .totals.percent_covered (the unrounded value) in both jobs, or update .coveragerc to change the precision
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This replaces
barecheckwithdiff-cover-actionto show coverage in PRs. It doesn't rely on external services.I hope it will be better than our current solution, as I think it is able to check coverage of the diff rather than just global coverage. It wraps the Python
diff-coverutility, adding some nice reporting to the PR thread.Rather than have an ever-growing list of reports in
pyproject.tomlI have removed those options from the pytest section. Instead, coverage reports should now be specified when they're needed. The action works with many formats of report - we now generate XML, JSON and LCOV of which we're currently using XML (fordiff-cover) and JSON (to extract overall coverage). I don't think LCOV does any harm so I've left it.I tried Github's native coverage action as per the github docs but this is restricted to the AI-enhanced security/quality service that we don't have. I'm not aware of an open source/educational tier that would cover it
For monitoring coverage as I develop, I've switched to using the VSCode tests panel as it puts coverage bars in my editor. Other developers are welcome to switch on whatever coverage reports they prefer, but I think it's not massively helpful to have this all turned on by default. Turning off coverage when I run it at the command line takes tests from ~30s to ~20s on my system, and I think most folk will value the ten seconds more than the pre-emptive coverage report.
I've set the test workflow to run on pushes to
mainand I hope this will fix the coverage badge.I've also added some manual logic to check that overall coverage hasn't decreased too much (current threshold is -0.2%). While
diff-coverenforces that the whole diff is covered, it doesn't stop someone e.g. deleting lots of test code. I think a fraction of a percent is a reasonable threshold (e.g. to allow for fewer import statements as a result of tidying-up, which can drop coverage slightly because the imports are always covered).This PR contains no AI-generated code.
This PR should have commits squashed before merging!