Skip to content

Wait out a concurrent token refresh instead of failing the caller - #313

Closed
gynsus wants to merge 2 commits into
trypostit:mainfrom
gynsus:pr/token-refresh-race
Closed

gynsus wants to merge 2 commits into
trypostit:mainfrom
gynsus:pr/token-refresh-race

Conversation

@gynsus

@gynsus gynsus commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

The race

When a token sits right at its expiry, the verify cron and a scheduled publish fire on the same tick and both call refreshToken(). The loser of the lock currently re-reads the row immediately — but the winner hasn't persisted the new token yet, so the loser re-reads the old token and fails its caller with "a refresh is already in progress", even though a perfectly fresh token lands milliseconds later.

We hit this in production as a YouTube account flapping to "session expired" once a day with nothing actually wrong: both jobs collided on the same cron tick, the loser verified with the stale token, got a 401, and marked the account disconnected. (On current main the failure mode is softer — a transient PlatformUnavailableException — but it still fails a publish attempt that had no reason to fail.)

The fix

On a lock miss, wait for the concurrent refresh to finish before re-reading:

try {
    $lock->block(15);
    $lock->release();
} catch (LockTimeoutException) {
    // Fall through — the row is re-read either way.
}

$account->refresh();

15 seconds comfortably covers a provider round-trip; on timeout the behavior degrades to exactly what it is today. The existing is_token_expired guard after the re-read stays as the backstop.

Testing

ConnectionVerifierTest passes unchanged — the fix only changes when the row is re-read, not any observable contract. The race itself was verified live: after deploying this, the daily false "session expired" on the colliding account stopped.

@gynsus
gynsus force-pushed the pr/token-refresh-race branch from 0bcbe73 to d71e3b1 Compare September 16, 2026 08:25
…e row

When RefreshSocialToken and VerifyUpcomingPostConnections fire on the same
scheduler tick, the verifier's refreshToken() lost the refresh lock race,
immediately re-read the account — before the winning process had saved the
new token — and retried verification with the old token, which had just
crossed its expiry. The account was then wrongly marked token_expired and
disconnected mid-schedule. Block on the lock (up to 15s) so the winner's
freshly saved token is what gets re-read.
@gynsus
gynsus force-pushed the pr/token-refresh-race branch from d71e3b1 to 489b4c2 Compare September 22, 2026 08:24
@paulocastellano

Copy link
Copy Markdown
Contributor

Thanks for digging into this, @gynsus! The YouTube flapping you saw was real on the code this branch started from. Since then main handles both ends of it: refreshThenVerify() reloads the account and verifies with the token another process persisted (ba8068f), and a lock miss on a token that has already expired now throws PlatformUnavailableException, so the publish is rescheduled instead of using a dead token or disconnecting the account (96ec995). Tokens are also refreshed 30 minutes before they expire, so the remaining case (an already-expired token plus two refreshes in the same second) is rare and costs one retry. Waiting on the lock would add up to 15 seconds to callers, including some web requests, so I'm closing this one. Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants