Skip to content

fix: submission_status_cleanup recovers all non-terminal states#2414

Open
hanane-ca wants to merge 1 commit into
codalab:developfrom
hanane-ca:fix/cleanup-all-non-terminal-states
Open

fix: submission_status_cleanup recovers all non-terminal states#2414
hanane-ca wants to merge 1 commit into
codalab:developfrom
hanane-ca:fix/cleanup-all-non-terminal-states

Conversation

@hanane-ca

Copy link
Copy Markdown

Fixes #2413

Problem

submission_status_cleanup() only filters status=Running, so submissions stuck in Submitted, Preparing, or Scoring are never recovered by the periodic cleanup. They hang indefinitely and consume quota permanently.

The existing started_when check also fails for submissions that never reached Running (null started_when).

Changes

  • src/apps/competitions/tasks.py: extend filter from status=Running to status__in=[Submitted, Preparing, Running, Scoring]
  • Use created_when as fallback reference time when started_when is null
  • src/apps/competitions/tests/test_submissions.py: add 4 unit tests for the new cleanup coverage + 1 negative test ensuring recent submissions are not prematurely cleaned

Testing

All 6 tests pass:

$ docker compose exec django python manage.py test competitions.tests.test_submissions.TestSubmissionTasks --verbosity=2

test_cancelling_parent_submission_cancels_all_children ... ok
test_cleanup_does_not_touch_recent_non_terminal_submissions ... ok
test_cleanup_recovers_stuck_preparing_submissions ... ok
test_cleanup_recovers_stuck_scoring_submissions ... ok
test_cleanup_recovers_stuck_submitted_submissions ... ok
test_submissions_are_cancelled_if_running_24_hours_past_execution_time_limit ... ok

Ran 6 tests in 2.231s
OK

…just Running

Extend cleanup to cover Submitted, Preparing, Running, and Scoring.
Use created_when as fallback when started_when is null (submissions
that never reached Running).

Add 4 unit tests covering the new states and a negative test for
recent submissions.

Fixes codalab#2413
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

submission_status_cleanup only recovers Running submissions — Submitted, Preparing, and Scoring hang forever

2 participants