Environment
- Langfuse server: self-hosted v4 (
docker.io/langfuse/langfuse:4), default write mode events_only
- Python SDK: 4.14.4 (also verified against 4.15.1 source)
What happened
Running batch evaluation against existing traces fails. The runner fetches items via endpoints that are unavailable on v4 events_only deployments:
scope="traces" → client.api.trace.list → GET /api/public/tracesscope="observations" → legacy observations_v1 → GET /api/public/observations
Both return:
"This endpoint is not available on deployments running in Langfuse v4 events_only mode."
Root cause
BatchEvaluationRunner._fetch_items (langfuse/batch_evaluation.py) still calls the v3 read APIs. Per the v3→v4 migration guide, reads should go through GET /api/public/v2/observations (client-side grouping by traceId, root observation for trace-level io).
Expected
batch_evaluation works on events_only deployments, e.g. by fetching via the v2 observations API.
Workaround
We currently run our own thin loop: fetch v2 observations grouped by traceId → evaluate → create_score (score-create ingestion works fine on v4).
Happy to contribute a PR if the v2-based approach sounds right — glad to align on the design first.
Environment
docker.io/langfuse/langfuse:4), default write modeevents_onlyWhat happened
Running batch evaluation against existing traces fails. The runner fetches items via endpoints that are unavailable on v4 events_only deployments:
scope="traces"→client.api.trace.list→GET /api/public/tracesscope="observations"→ legacyobservations_v1→GET /api/public/observationsBoth return:
Root cause
BatchEvaluationRunner._fetch_items(langfuse/batch_evaluation.py) still calls the v3 read APIs. Per the v3→v4 migration guide, reads should go throughGET /api/public/v2/observations(client-side grouping bytraceId, root observation for trace-level io).Expected
batch_evaluationworks onevents_onlydeployments, e.g. by fetching via the v2 observations API.Workaround
We currently run our own thin loop: fetch v2 observations grouped by
traceId→ evaluate →create_score(score-create ingestion works fine on v4).Happy to contribute a PR if the v2-based approach sounds right — glad to align on the design first.