Skip to content

fix(core): forward abortSignal to retryWithBackoff in BaseLlmClient - #29089

Open
chelsealong wants to merge 1 commit into
google-gemini:mainfrom
chelsealong:fix/session-summary-abort-signal-retry
Open

fix(core): forward abortSignal to retryWithBackoff in BaseLlmClient#29089
chelsealong wants to merge 1 commit into
google-gemini:mainfrom
chelsealong:fix/session-summary-abort-signal-retry

Conversation

@chelsealong

Copy link
Copy Markdown

Relates to #29065

What

BaseLlmClient.generateContent / generateJson (the client used by
SessionSummaryService, chat compression, the classifier, prompt
completion, etc.) accept an abortSignal and thread it into each
individual API call, but never pass it as the signal option of
retryWithBackoff. As a result, a caller-side timeout only aborts the
in-flight HTTP request — the retry loop itself is not bounded by it and
keeps waiting out its full exponential-backoff schedule and issuing
further attempts regardless of the timeout having already fired.

geminiChat.ts and web-fetch.ts already pass signal to
retryWithBackoff for this exact reason; BaseLlmClient was the
outlier.

Relationship to #29065

#29065 reports that session summary generation hardcodes
gemini-3.1-flash-lite (via the summarizer-default model config in
defaultModelConfigs.ts) instead of using the user's configured/active
model, which causes every summary attempt to fail outright on
custom/proxy endpoints that don't serve that model. This PR does not
fix that.
The hardcoded-model issue is the actual root cause reported
in #29065 and needs a separate fix (making summarizer-default, and
likely the other UTILITY_* configs with the same pattern, derive from
the user's configured model).

What this PR does fix is a related but secondary symptom visible in the
same issue's logs: because BaseLlmClient never forwarded abortSignal
into retryWithBackoff, SessionSummaryService's own 5s timeout
couldn't stop the retry loop, which otherwise runs ~65s
(5s/10s/20s/30s backoff across 5 attempts) printing "Attempt N failed...
Retrying with backoff" stack traces the whole time. With this fix, a
timed-out or unreachable summary model now fails fast and quietly
(SessionSummaryService already handles AbortError by logging at
debug level and returning null) instead of spamming the terminal for
over a minute past its own timeout budget.

In short: after this change, users on custom endpoints will still see
every session summary attempt fail (since it still targets a hardcoded
model their endpoint may not serve), but they'll see it fail in ~5s
with no retry spam instead of ~65s with a wall of stack traces. This
PR should not auto-close #29065 — leaving it open until the
model-hardcoding is addressed.

Change

One line: pass signal: abortSignal into the retryWithBackoff call
inside BaseLlmClient._generateWithRetry.

Test plan

Added a regression test in packages/core/src/core/baseLlmClient.test.ts
asserting that generateContent forwards its abortSignal to
retryWithBackoff as the signal option.

Confirmed the test fails without the fix (reverted the one-line change
locally and re-ran):

$ npx vitest run packages/core/src/core/baseLlmClient.test.ts -t "forward the caller-provided abortSignal"
 FAIL  packages/core/src/core/baseLlmClient.test.ts > BaseLlmClient > generateContent > should forward the caller-provided abortSignal to retryWithBackoff so the overall retry loop honors it
AssertionError: expected "spy" to be called with arguments: [ Any<Function>, ObjectContaining{…} ]
  1st spy call:
  [
-   Any<Function>,
-   ObjectContaining { "signal": AbortSignal {...} },
+   [Function apiCall],
+   { "authType": "gemini-api-key", "maxAttempts": 5, ... }  // no `signal` key
  ]

With the fix applied:

$ npx vitest run packages/core/src/core/baseLlmClient.test.ts packages/core/src/services/sessionSummaryService.test.ts
 ✓ packages/core/src/services/sessionSummaryService.test.ts (33 tests) 40ms
 ✓ packages/core/src/core/baseLlmClient.test.ts (30 tests) 43ms

 Test Files  2 passed (2)
      Tests  63 passed (63)

Also ran:

  • npx eslint packages/core/src/core/baseLlmClient.ts packages/core/src/core/baseLlmClient.test.ts — clean.
  • npx tsc -p packages/core/tsconfig.json --noEmit — clean.

AI assistance disclosure

This change (investigation, fix, and test) was authored with the
assistance of an AI coding agent (Claude Code) and reviewed by me
before submission.

BaseLlmClient.generateContent/generateJson pass abortSignal into each
individual API call, but never as the `signal` option of
retryWithBackoff. This means a caller-side timeout (e.g. Config's
5s session-summary timeout) only cancels the in-flight request; the
retry loop keeps waiting out its full backoff schedule and issuing
further attempts regardless, which is why users on custom/proxy
endpoints see several retries with backoff and stack-trace spam
before the call finally gives up (google-gemini#29065). geminiChat.ts and
web-fetch.ts already pass `signal` to retryWithBackoff; this aligns
BaseLlmClient with that pattern.
@chelsealong
chelsealong requested a review from a team as a code owner August 25, 2026 22:22
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request improves the reliability and responsiveness of LLM client operations by ensuring that cancellation signals are properly propagated through the retry logic. Previously, while individual HTTP requests could be aborted, the retry loop itself would continue to execute for its full duration. By passing the abortSignal to the retry handler, the system now fails fast when a timeout or cancellation occurs, preventing unnecessary background activity and reducing log noise.

Highlights

  • AbortSignal Propagation: Updated BaseLlmClient to pass the caller-provided abortSignal to the retryWithBackoff function, ensuring the entire retry loop respects cancellation requests.
  • Regression Testing: Added a new test case in baseLlmClient.test.ts to verify that the abortSignal is correctly forwarded to the retry mechanism.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions github-actions Bot added the size/s A small PR label Aug 25, 2026
@github-actions

Copy link
Copy Markdown

📊 PR Size: size/S

  • Lines changed: 23
  • Additions: +23
  • Deletions: -0
  • Files changed: 2

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates BaseLlmClient to forward the caller-provided abortSignal to the retryWithBackoff utility, ensuring that the retry loop honors the abort signal. It also adds a corresponding unit test in baseLlmClient.test.ts to verify this behavior. There are no review comments, and I have no additional feedback to provide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/agent Issues related to Core Agent, Tools, Memory, Sub-Agents, Hooks, Agent Quality priority/p2 Important but can be addressed in a future release. size/s A small PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Session summary generation ignores configured model and hardcodes gemini-3.1-flash-lite, causing failures on custom endpoints

1 participant