Repository navigation
feat(models): send the ADK session id to a LiteLLM proxy - #8187
Open
thomaslprr wants to merge 1 commit into
Open
thomaslprr wants to merge 1 commit into
thomaslprr wants to merge 1 commit into
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
LiteLlm did not tell a LiteLLM proxy which conversation a call belongs to, so the proxy logged every call of an ADK session under a different session_id and calls could not be grouped per conversation. The LLM flow now sets the session id on LlmRequest.session_id, a field excluded from serialization, and LiteLlm sends it as the x-litellm-session-id header when the model is served through a LiteLLM proxy. A session id header set by the caller is kept, and no header is sent to other providers or when the request has no session id.
thomaslprr
force-pushed
the
feat/litellm-propagate-session-id
branch
from
November 16, 2026 16:45
f59536a to
1e4ec4d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem:
LiteLlmdoes not tell a LiteLLM Proxy which conversation a call belongs to, so the proxy logs every call of an ADK session under a differentsession_idand the calls cannot be grouped per conversation.Solution:
LlmRequest.session_id: a new optional field,exclude=Trueso thatmodel_dump()output (recorded or traced requests) does not change.invocation_context.session.id, next to theadk_agent_namelabel, unless it is already set.LiteLlmsends it as thex-litellm-session-idheader when the model is served through a LiteLLM Proxy (the existing_is_proxied_modelcheck).Behavior that does not change:
extra_headersor inhttp_options.headers, is kept.Out of scope: the native Gemini model (the Gemini API has no session to map to) and live mode.
Testing Plan
Unit Tests:
tests/unittests/models/test_litellm.py: header sent for a proxied model (prefix andcustom_llm_provider), not sent to a direct provider, not sent without a session id, caller's header kept (constructor andhttp_options), other extra headers preserved,session_idabsent frommodel_dump().tests/unittests/flows/llm_flows/test_other_configs.py: the request carries the session id, and a sub-agent reached by transfer gets the same one.tests/unittests/models/test_fallback_model.py: the new field is added to the list of fieldsFallbackModelleaves as is (models only read it).Manual End-to-End (E2E) Tests:
Re-run with this branch against a real LiteLLM Proxy (v1.105.1, Postgres spend logs), no plugin or callback:
an agent with a tool, a sub-agent reached by transfer with its own tool, and an AgentTool child; three user turns.
session_idlitellm_proxy/<model>openai/<model>withapi_base= the proxyx-litellm-session-idinextra_headersWithout this branch the same
litellm_proxy/conversation gives onesession_idper call (#8186).Also checked: the new tests fail on
main's sources (7 failures) and pass on this branch;pytest tests/unittests/{models,flows,agents,runners,plugins,tools} -n auto: 10064 passed, 2 skipped, 4 xfailed.Known limits, for reviewers:
AgentToolchild runs in its own in-memory session, so its calls carry that session's id, not the parent's.openai/<model>+api_baseis not detected; use thelitellm_proxy/prefix.model_consult, classifiers, eval judges) carry no session id.Checklist
Additional context
Two questions for reviewers are in #8186: whether a field on
LlmRequestis the channel you want, and whether this should be on by default.