feat(observability): add GenAI semantic attributes

[nachiketb-nvidia]

What

Add OpenTelemetry GenAI semantic attributes to the existing Rust server and libsy spans while preserving the current telemetry surface.

• Name model-call spans chat <model> and mark them as client spans.
• Record the request model, streaming mode, sampling controls, output type, and reasoning level from the neutral IR.
• Record response IDs, served models, finish reasons, input/output/cache/reasoning token usage, and typed failures for buffered and streamed responses.
• Retain Switchyard route, algorithm, routing tier, session, and correlation attributes.
• Continue incoming W3C traceparent and tracestate at the Rust server boundary.

gen_ai.provider.name remains unset because the current configuration does not identify the provider authoritatively. Wire format, hostname, and model name are not reliable provider identities.

Why

OTLP consumers need standard model-call data to reconstruct per-trace and per-session usage without an Intake-specific Rust sink. The existing spans already represent the correct request and model-call boundaries; this adds the portable GenAI projection and inbound trace continuity.

How

• Extend libsy.run and libsy.client_call instead of creating duplicate GenAI spans.
• Populate request, response, and usage attributes from the neutral Switchyard IR.
• Keep the client span alive until a response stream completes, errors, or is abandoned.
• Write finish reasons directly to OpenTelemetry so they remain a typed string array.
• Let switchyard-llm-client add the resolved upstream server address and port.
• Validate exported spans with an in-memory OpenTelemetry exporter.

No prompt, completion, raw body, API key, or header content is recorded. Existing metrics, logs, and HTTP endpoints remain unchanged.

What to review

• GenAI attribute names and numeric wire types.
• Cache-aware total input usage and finish-reason normalization.
• Streaming span completion, error, and cancellation behavior.
• W3C parent extraction at the server boundary.
• The intentional absence of inferred provider identity.

Validation

cargo fmt --all -- --check
cargo clippy -p switchyard-libsy -p switchyard-llm-client --all-targets -- -D warnings
cargo test -p switchyard-libsy

Related: SWITCH-1096

Follow-up: SWITCH-1142

[coderabbitai]

Walkthrough

The PR adds W3C trace propagation for server requests and expands tracing across routing, LLM requests, responses, token usage, errors, and streaming telemetry. Tests cover standardized span attributes and deferred stream observation.

Changes

Observability instrumentation

Layer / File(s)	Summary
Server trace-context propagation crates/switchyard-server/src/observability.rs, crates/switchyard-server/src/lib.rs	Incoming traceparent and tracestate headers now parent endpoint request spans.
Run and client-call telemetry crates/libsy/src/observability.rs, crates/libsy/src/core/algorithm.rs, crates/libsy/tests/observability.rs	Run and client-call spans now record request, routing, response, usage, streaming, error, and session fields. Streaming tests verify terminal usage handling.
Provider and GenAI request telemetry crates/libsy-llm-client/src/backend.rs, crates/libsy-llm-client/src/client.rs	LLM client spans now record provider, model, stream mode, and upstream server attributes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

I’m a rabbit with traces to share,
From request headers through streaming air.
Tokens hop, spans glow bright,
Errors leave fields in plain sight.
Provider names now guide the way.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding GenAI semantic attributes to observability spans.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (3)

crates/libsy/src/observability.rs (2)

198-218: 🚀 Performance & Scalability | 🔵 Trivial

Streaming responses keep the libsy.client_call span open until the stream finishes.

observe_client_stream retains the libsy.client_call span for the lifetime of the returned stream. Until the stream is fully consumed or dropped, the span does not close or export. For a slow consumer, the recorded span duration includes the full stream-consumption time, and the trace does not appear in the backend until then. Confirm this is the intended tradeoff for any alerting or dashboarding that keys on libsy.client_call duration.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/libsy/src/observability.rs` around lines 198 - 218, Update
observe_client_stream and the surrounding libsy.client_call instrumentation to
ensure the client-call span closes when the provider response is received,
rather than retaining it until the returned stream is consumed or dropped.
Preserve the existing chunk-level attribute and error recording while separating
stream observation from the span’s lifetime so duration and export reflect the
client call itself.

---

220-250: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Document the cache-token summation rationale.

record_gen_ai_usage computes gen_ai.usage.input_tokens as input_tokens + cache_read + cache_creation. This matches the OTel GenAI semantic convention for providers whose native input_tokens excludes cache tokens, but the rationale is not obvious from the code. Add a short comment stating that this follows the GenAI semconv computation, to prevent a future change from "simplifying" this to just usage.input_tokens (a real regression of this exact kind has been reported against other OTel GenAI instrumentations).

As per coding guidelines, non-obvious private helper logic should carry an explanatory comment: "document module/file intent, public structs and enums, public methods, non-obvious private helpers... use... block comments before complex validation... logic."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/libsy/src/observability.rs` around lines 220 - 250, The non-obvious
input-token aggregation in record_gen_ai_usage needs documentation. Add a short
explanatory comment immediately before the gen_ai.usage.input_tokens calculation
stating that summing native input, cache-read, and cache-creation tokens follows
the OTel GenAI semantic convention; leave the existing computation unchanged.

Source: Coding guidelines

crates/libsy-llm-client/src/client.rs (1)

181-181: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Document the implicit span contract and add test coverage for record_gen_ai_request.

record_gen_ai_request records fields onto tracing::Span::current(). This assumes the caller is still inside the libsy.client_call span created in libsy/src/core/algorithm.rs. No function between that span's creation and this call carries its own #[instrument], so it works today, but the dependency is implicit across two crates. If a future change adds an intermediate span, this function will silently record onto the wrong span with no compile or runtime error, since tracing drops unrecognized fields.

Add a short comment stating this assumption, and add a unit test in this file's test module that enters a test span, calls record_gen_ai_request directly, and asserts gen_ai.provider.name, gen_ai.request.model, gen_ai.request.stream, server.address, and server.port are recorded as expected — the existing backend.rs tests and the switchyard-server request_span test both show the pattern for asserting recorded span fields without a real HTTP call.

As per coding guidelines: "document... non-obvious private helpers... use /// doc comments for public items and block comments before complex... async... logic."

Also applies to: 605-618

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/libsy-llm-client/src/client.rs` at line 181, Document in
record_gen_ai_request that it records fields onto the caller’s existing
libsy.client_call span and must be invoked within that span context. Add a unit
test in this file’s test module that enters a test span, calls
record_gen_ai_request directly, and verifies gen_ai.provider.name,
gen_ai.request.model, gen_ai.request.stream, server.address, and server.port
using the existing span-field assertion pattern, without making an HTTP request.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/libsy-llm-client/src/client.rs`:
- Line 181: Document in record_gen_ai_request that it records fields onto the
caller’s existing libsy.client_call span and must be invoked within that span
context. Add a unit test in this file’s test module that enters a test span,
calls record_gen_ai_request directly, and verifies gen_ai.provider.name,
gen_ai.request.model, gen_ai.request.stream, server.address, and server.port
using the existing span-field assertion pattern, without making an HTTP request.

In `@crates/libsy/src/observability.rs`:
- Around line 198-218: Update observe_client_stream and the surrounding
libsy.client_call instrumentation to ensure the client-call span closes when the
provider response is received, rather than retaining it until the returned
stream is consumed or dropped. Preserve the existing chunk-level attribute and
error recording while separating stream observation from the span’s lifetime so
duration and export reflect the client call itself.
- Around line 220-250: The non-obvious input-token aggregation in
record_gen_ai_usage needs documentation. Add a short explanatory comment
immediately before the gen_ai.usage.input_tokens calculation stating that
summing native input, cache-read, and cache-creation tokens follows the OTel
GenAI semantic convention; leave the existing computation unchanged.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 52ad2506-20ea-41b9-94c9-53b46a3b2c46

📥 Commits

Reviewing files that changed from the base of the PR and between 6ff491a and 1a7672c.

📒 Files selected for processing (7)

• crates/libsy-llm-client/src/backend.rs
• crates/libsy-llm-client/src/client.rs
• crates/libsy/src/core/algorithm.rs
• crates/libsy/src/observability.rs
• crates/libsy/tests/observability.rs
• crates/switchyard-server/src/lib.rs
• crates/switchyard-server/src/observability.rs