Repository navigation
feat(tracing): stamp sgp evals run and row ids on SGP spans - #545
mohammadatallah-scale wants to merge 13 commits into
Conversation
Spans of a task whose task_metadata carries sgp_evals now get flat sgp_evals_generation_run_id, sgp_evals_row_id and sgp_evals_attempt_idx keys on the SGP copy. Sync and async agents read the task in the ACP server. Temporal agents carry it in the workflow memo and activity headers to the worker. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…val attrs Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
P3 Refuted: continue-as-new keeps the memo (real dev-server run stamps the post-CAN run), |
Local activities go through start_local_activity, which the outbound interceptor did not override, so their spans carried no run or row id. Also pin the worker's interceptor wiring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ry none Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…id cannot strip a queued span Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…load succeeds Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…o a span flood cannot evict them Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
andrewvo-scale
left a comment
There was a problem hiding this comment.
LGTM. The SDK is the right layer, since it is the only place that sees both the task metadata and the span. The keys it reads (generation_run_id, row_id, attempt_idx under the sgp_evals marker) match what sgp-evaluations writes in _unit_task_metadata. With no eval ids, _sgp_metadata returns exactly what base returns, so existing users see no change. All seven threads are resolved with fixes at head. On the first P1 (unregister on later events), keeping the unregister is right, because the server sends the full task, task_metadata included, on every call. Two non-blocking notes inline.
…ns without an SGP processor Child workflows inherit no memo and their workflow id is not the task id, so the parent hands both down in the child's memo and activities carry the task id in a header. Span pinning now starts only once an SGP processor exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both closed. Child workflows now inherit the ids and the task id (a789175), and |
|
@mohammadatallah-scale I re-checked a789175/8a170c1. The child-workflow memo + task-id header fix and skipping pins with no SGP processor both look right, and neither runs without eval ids. Two things before merge:
|
…it started A span that started before an SGP processor existed has no pin, so the export fell back to the live registry and could pick up a run registered later under the same task id. The registry now records when each task's ids took effect, and the fallback skips ids registered after the span started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@andrewvo-scale The P1 is fixed rather than left open (b45d3b3): an unpinned span now only gets ids its task had at its start. On CI, the same three tutorials fail or cancel the same way on two unrelated branches ( |
…orker start-span and end-span are separate activities, so a span can end on a worker that first registered its task after the span started, and the start-time gate dropped its ids. Ids read from a workflow memo are fixed for the workflow's life, so they now cover every span of the workflow. Only ids the ACP server registers keep the start-time gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erver openai-websearch-mcp pins mcp 1.13.1, which imports a pydantic internal that pydantic 2.14 removed, so the server died at import and the 010, 020 and 040 tutorials timed out waiting for tool calls on every branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@andrewvo-scale Correction to my earlier reply: it is not a flake, and not this change either. |
OVE-1253. Pairs with an sgp-evaluations PR that tags each generation-unit task.
The problem. Spans of an evals generation-unit task carry no run or row id, so a spans search cannot find them.
The fix.
trace_idandtask_idare both not the task id.Test plan
Changes since review


Confidence Score: 5/5
The numbered finding is addressed, and no separate new blocking finding was accepted.Summary
Adds run, row, and attempt IDs to the SGP copy of eval-task spans.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Task metadata] --> B[ACP task registration] A --> C[Temporal workflow memo] C --> D[Child workflow memo] C --> E[Activity headers] D --> E E --> F[Worker task registration] B --> G[Span-start capture] F --> G G --> H[SGP metadata copy] F --> H H --> I[Upload] I --> J[Release saved span IDs]Reviews (8) · Last reviewed commit: "ci(tutorials): pin pydantic below 2.14 f..." · Reviewed by Greptile