fix: stop scraper from finishing ZIM when bad exception arises#55
Conversation
benoit74
left a comment
There was a problem hiding this comment.
This is not the correct way to fix the issue.
Creator object already has required primitive to indicate ZIM must not be created (finish).
Please have a look at logic around can_finish at https://github.com/openzim/mindtouch/blob/d1909c9b225afc92b44140a5bf9ff5087f6558ec/scraper/src/mindtouch2zim/processor.py#L287-L305 for instance.
|
I didn't know the creator object had that, thank you. Let me know if this works |
d34ac1e to
35f6118
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #55 +/- ##
==========================================
+ Coverage 92.28% 92.36% +0.08%
==========================================
Files 4 4
Lines 376 380 +4
Branches 37 37
==========================================
+ Hits 347 351 +4
Misses 25 25
Partials 4 4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
benoit74
left a comment
There was a problem hiding this comment.
LGTM, please add an entry to the CHANGELOG.md and squash all your commits into a single one on top of main branch.
- Set `Creator` object `can_finish` to `False`. - Add test coverage for failure cleanup.
35f6118 to
0eb08cd
Compare
|
Done |
fixes #54
Changes
Wrapped the ZIM creation context manager in a
try...except Exception:block insidegenerator.py.If an exception occurs, delete the incomplete ZIM file and
bubble the exception to
entrypointand exit with code 1Added
test_generate_zim_cleans_up_on_failuretotests/test_generator.pyto ensure the file is deleted and the exception is propagated whenget_dbfails.