🔴 Required Information
Describe the Bug:
Running an evaluation with VertexAiSessionService fails because ADK generates evaluation session IDs using the following format:
___eval___session___<UUID>
The underscores in this ID violate the Vertex AI Agent Engine Sessions restrictions for custom session IDs. Agent Engine only accepts lowercase letters, digits, and hyphens ([a-z0-9-]), with a letter or digit as the first and last character.
The eval session ID is generated in google/adk/evaluation/local_eval_service.py:
|
EVAL_SESSION_ID_PREFIX = "___eval___session___" |
|
|
|
|
|
def _get_session_id() -> str: |
|
return f"{EVAL_SESSION_ID_PREFIX}{str(uuid.uuid4())}" |
It is then passed to the configured session service by _get_or_create_eval_session() in google/adk/evaluation/evaluation_generator.py:
|
return await session_service.create_session( |
|
app_name=app_name, |
|
user_id=user_id, |
|
state=initial_session.state if initial_session else {}, |
|
session_id=pinned_session_id or fallback_session_id or str(uuid.uuid4()), |
|
) |
When the configured session service is VertexAiSessionService, Agent Engine rejects the generated ID and the evaluation cannot start.
There is also a validation mismatch in VertexAiSessionService. Its current local pattern accepts underscores and uppercase letters:
ref: https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sessions/manage-with-api?hl=en#create_a_session
_SESSION_ID_PATTERN = re.compile(r'^[A-Za-z0-9_-]+$')
Therefore, the invalid eval session ID passes ADK-side validation and is rejected only by the remote Agent Engine API.
Steps to Reproduce:
-
Install ADK with the required extras:
pip install "google-adk[gcp,eval]==2.6.3"
-
Configure an Agent Engine-backed session service:
adk web \
--session_service_uri="agentengine://projects/<PROJECT>/locations/<LOCATION>/reasoningEngines/<ENGINE_ID>" \
.
-
Open the ADK Web UI and create or select an eval set.
-
Run an evaluation.
-
Observe that creation of the generated ___eval___session___<UUID> session fails.
The problem can also be isolated through the session creation endpoint:
| Request |
Result |
| No custom session ID |
200 OK |
eval-<UUID> |
200 OK |
___eval___session___<UUID> |
500 Internal Server Error |
Expected Behavior:
Evaluations should be able to create their temporary sessions when VertexAiSessionService is configured.
Observed Behavior:
Agent Engine rejects the generated session ID:
400 INVALID_ARGUMENT
session_id can only contain lowercase letters, digits and hyphens.
The first and last characters must be a letter or number.
The exception is subsequently exposed by the ADK Web/API server as an HTTP 500, and the evaluation fails before inference begins.
Environment Details:
- ADK Library Version (pip show google-adk):
google-adk==1.38.0
google-adk==2.6.3
- Desktop OS: macOS 26.4.1
- Python Version: Python 3.13.12
Model Information:
- Are you using LiteLLM: No
- Which model is being used: N/A — the failure occurs before model inference
🟡 Optional Information
Regression:
This is related to #2731, which reported that VertexAiSessionService did not support user-provided session IDs.
Current ADK releases now pass custom IDs to Agent Engine. However, evals still fail because the ADK-generated ___eval___session___ prefix does not satisfy the Agent Engine ID format.
Issue #2731 was closed as stale rather than by a merged fix, so this appears to be the remaining/current form of the compatibility problem.
Logs:
400 INVALID_ARGUMENT. {
'error': {
'code': 400,
'message': (
'List of found errors: '
'1.Field: session_id; Message: session_id can only contain '
'lowercase letters, digits and hyphens. The first and last '
'characters must be a letter or number.'
),
'status': 'INVALID_ARGUMENT'
}
}
Screenshots / Video:
N/A
Additional Context:
I developed adk eval <api_server_url> version, and I found this issue.
https://github.com/ftnext/pytest-adk/releases/tag/0.0.7
Minimal Reproduction Code:
import asyncio
from google.adk.sessions import VertexAiSessionService
async def main() -> None:
service = VertexAiSessionService(
project="<PROJECT>",
location="<LOCATION>",
agent_engine_id="<ENGINE_ID>",
)
await service.create_session(
app_name="<APP_NAME>",
user_id="test-user",
session_id="___eval___session___550e8400-e29b-41d4-a716-446655440000",
)
asyncio.run(main())
How often has this issue occurred?:
🔴 Required Information
Describe the Bug:
Running an evaluation with
VertexAiSessionServicefails because ADK generates evaluation session IDs using the following format:The underscores in this ID violate the Vertex AI Agent Engine Sessions restrictions for custom session IDs. Agent Engine only accepts lowercase letters, digits, and hyphens (
[a-z0-9-]), with a letter or digit as the first and last character.The eval session ID is generated in
google/adk/evaluation/local_eval_service.py:adk-python/src/google/adk/evaluation/local_eval_service.py
Lines 64 to 68 in 0b55dcf
It is then passed to the configured session service by
_get_or_create_eval_session()ingoogle/adk/evaluation/evaluation_generator.py:adk-python/src/google/adk/evaluation/evaluation_generator.py
Lines 118 to 123 in 0b55dcf
When the configured session service is
VertexAiSessionService, Agent Engine rejects the generated ID and the evaluation cannot start.There is also a validation mismatch in
VertexAiSessionService. Its current local pattern accepts underscores and uppercase letters:ref: https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sessions/manage-with-api?hl=en#create_a_session
Therefore, the invalid eval session ID passes ADK-side validation and is rejected only by the remote Agent Engine API.
Steps to Reproduce:
Install ADK with the required extras:
pip install "google-adk[gcp,eval]==2.6.3"Configure an Agent Engine-backed session service:
Open the ADK Web UI and create or select an eval set.
Run an evaluation.
Observe that creation of the generated
___eval___session___<UUID>session fails.The problem can also be isolated through the session creation endpoint:
200 OKeval-<UUID>200 OK___eval___session___<UUID>500 Internal Server ErrorExpected Behavior:
Evaluations should be able to create their temporary sessions when
VertexAiSessionServiceis configured.Observed Behavior:
Agent Engine rejects the generated session ID:
The exception is subsequently exposed by the ADK Web/API server as an HTTP 500, and the evaluation fails before inference begins.
Environment Details:
google-adk==1.38.0google-adk==2.6.3Model Information:
🟡 Optional Information
Regression:
This is related to #2731, which reported that
VertexAiSessionServicedid not support user-provided session IDs.Current ADK releases now pass custom IDs to Agent Engine. However, evals still fail because the ADK-generated
___eval___session___prefix does not satisfy the Agent Engine ID format.Issue #2731 was closed as stale rather than by a merged fix, so this appears to be the remaining/current form of the compatibility problem.
Logs:
Screenshots / Video:
N/A
Additional Context:
I developed
adk eval <api_server_url>version, and I found this issue.https://github.com/ftnext/pytest-adk/releases/tag/0.0.7
Minimal Reproduction Code:
How often has this issue occurred?: