Skip to content

fix(sessions)!: Never reuse a retired session - #2348

Merged
vdusek merged 8 commits into
apify:v2from
gauravch-code:fix/2312-terminal-session-retirement
Oct 29, 2026
Merged

vdusek merged 8 commits into
apify:v2from
gauravch-code:fix/2312-terminal-session-retirement

Conversation

@gauravch-code

@gauravch-code gauravch-code commented Oct 21, 2026 •

Copy link
Copy Markdown

Description

A session retired after a blocked request could become usable again, because mark_good() lowered its error score below the limit. Retirement is now tracked separately from the error score and is permanent, including after the session pool is saved and restored. The behavior matches Crawlee JS v4.

  • Session keeps a retired flag, exposed as the Session.retired property and saved in SessionModel. Saved state without the field loads with retired=False.
  • is_usable checks the flag. is_blocked still depends only on the error score.
  • mark_good() does nothing on a retired session, so its error score and usage count stay as they are.
  • retire() adds max_error_score to the error score once. Calling it on a session that is already retired changes nothing.
  • When mark_good() or mark_bad() finds a session expired, at its usage limit, or at its error score limit, it only sets the flag and adds no extra max_error_score.
  • The v2 upgrade guide has a new "Session retirement is permanent" section, and the standalone session management example calls retire() last.

Issues

Closes: #2312

Testing

  • New unit tests cover permanent retirement, mark_good() on a retired session, repeated retire() calls, automatic retirement at each limit, JSON round trips, loading older saved state, pool lookups, and saving and reopening the pool.
  • tests/unit/sessions and tests/unit/crawlers/_basic: 211 passed, 1 existing skip.
  • ruff check, ruff format --check and ty check pass.

Persist retirement independently of error scores and cover session and pool restoration.

BREAKING CHANGE: Retired sessions cannot become usable again after mark_good().
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Oct 21, 2026

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor things

Comment thread docs/upgrading/upgrading_to_v2.md Outdated
Comment thread tests/unit/sessions/test_session.py Outdated
@vdusek
vdusek requested a review from Pijukatel October 26, 2026 14:49

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A three more comments, regarding comparision with Crawlee JS

Comment thread src/crawlee/sessions/_session.py Outdated
Comment thread docs/upgrading/upgrading_to_v2.md Outdated
Comment thread src/crawlee/sessions/_session.py
@gauravch-code

gauravch-code commented Oct 26, 2026 •

Copy link
Copy Markdown
Author

Thanks for the careful review, @vdusek! I addressed the five comments in 261dab6:

  • Retirement is checked in is_usable; is_blocked depends only on the error score.
  • Automatic retirement sets only the retired flag, without adding a blocking penalty.
  • mark_good returns immediately for retired sessions, preserving both score and usage count.
  • Added the public retired property and regression coverage for expiration, usage limits, error limits, and restored retired state.
  • Moved the migration section to the end and changed the legacy-state test to delete the retired field explicitly.

All 49 session tests, full repository lint, and the migration Markdown lint pass. The full type check passes targeting Linux; native Windows reports three unrelated multiprocessing fork-type errors in tests/unit/_utils/test_system.py. Both commit hooks passed using an external Linux-targeted type configuration with the same repository rules.

Happy to adjust anything further. Thanks again!

CI note: the new factory-review approval lookup failed because GitHub returned HTTP 502, while the AI review job passed. GitHub requires repository admin rights to rerun the failed job, so I could not retry it from my fork. Could you rerun that lookup when you have a chance?

Comment thread src/crawlee/sessions/_session.py
@vdusek vdusek changed the title fix(sessions)!: make session retirement permanent fix(sessions)!: Never reuse a retired session Oct 28, 2026

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vdusek vdusek linked an issue Oct 29, 2026 that may be closed by this pull request
@vdusek
vdusek merged commit 57740f4 into apify:v2 Oct 29, 2026
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make Session.retire() terminal

4 participants