Skip to content

gmail: retry messages after body fetch failures - #244

Open
fallintoplace wants to merge 2 commits into
ClickHouse:mainfrom
fallintoplace:fix/gmail-body-fetch-cursor
Open

gmail: retry messages after body fetch failures#244
fallintoplace wants to merge 2 commits into
ClickHouse:mainfrom
fallintoplace:fix/gmail-body-fetch-cursor

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Problem

Gmail search and body retrieval happen separately. When gog gmail get failed or timed out, the source still created a record from the search metadata with an empty body and advanced the shared timestamp cursor. The recovered message was then behind the cursor, leaving only the header-only record permanently stored.

Fix

  • represent an unsuccessful body retrieval separately from a valid empty body
  • skip records whose body could not be retrieved
  • keep the previous cursor when any message in the batch fails
  • continue returning successfully retrieved messages and rely on inbox ID deduplication for unchanged retries

Verification

Regression coverage exercises a partial failure followed by recovery, an exception during initial sync, and a nonzero body-fetch command.

  • .venv/bin/python -m pytest -q tests/test_gmail_source.py tests/test_source_filters.py tests/test_db.py: 134 passed
  • broader suite: 2941 passed

@fallintoplace
fallintoplace force-pushed the fix/gmail-body-fetch-cursor branch from a816203 to 0eb9a7d Compare August 1, 2026 15:56
@alex-clickhouse

Copy link
Copy Markdown
Collaborator

One issue with this approach: suppose there is a message that for some reason fails gog gmail get deterministically. Then the cursor is going to be stuck at that point, until it falls out of the batch size window (and, since any failure means no cursor progress, it would take a bunch of other messages with it).

One option would be to retry in place for 2-3 times with exponential backoff. That still doesn't save us from longer-lived failures though.

More complex would be to do something similar to the Telegram one, which stores json, and then we could keep track of individual ids and their status. But that's probably a bit too much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants