feat(server): forward x-litellm-response-cost upstream header

[deepujain]

What

Adds x-litellm-response-cost to the forwarded upstream response headers, so LiteLLM's per-request cost header reaches Switchyard clients the same way it does on direct LiteLLM routes. One regression test covers the allowlist.

Why

Fixes #805. The header was dropped by the allowlist gate, so cost tracking broke on switchyard/... routes while direct openai/openai/... routes carried it.

Notes for reviewers

Start at FORWARDED_UPSTREAM_HEADERS in crates/switchyard-server/src/lib.rs. The header is a plain numeric cost, not a credential, so it is safe to expose downstream. Validated with rustc 1.96.1: cargo fmt --all -- --check, cargo clippy -p switchyard-server --all-targets (plus --features prefill-router), cargo test -p switchyard-server (56 passed), cargo clippy --workspace --all-targets, cargo test --workspace (all suites green), and cargo test -p switchyard-runner --features prefill-router (green). The new test fails with the allowlist entry removed.

Summary by CodeRabbit

• New Features

  • Clients can now receive upstream response cost information through the x-litellm-response-cost header.

• Bug Fixes

  • Confirmed that approved response headers are forwarded correctly while sensitive authorization headers remain blocked.

[coderabbitai]

Understand this PR’s impact

Explore downstream dependencies and potential security impact with Blast Radius.

View blast radius →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: NVIDIA-NeMo/Switchyard/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ae9e7abd-c3c0-4ddd-8113-7937da6f50ab

📥 Commits

Reviewing files that changed from the base of the PR and between bfcd023 and 6e1eefb.

📒 Files selected for processing (1)

• crates/switchyard-server/src/lib.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The upstream response-header allowlist now forwards x-litellm-response-cost. Tests verify forwarding for approved headers and rejection of authorization.

Changes

Response header forwarding

Layer / File(s)	Summary
Allowlist update and validation crates/switchyard-server/src/lib.rs	The response-header allowlist includes x-litellm-response-cost. Tests verify approved headers pass through and authorization remains blocked.

Priority: ➖ Normal

Estimated code review effort: 1 (Trivial) | ~5 minutes

Severity of issue fixed: Medium

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely identifies the main change: forwarding the x-litellm-response-cost upstream header.
Linked Issues check	✅ Passed	Issue #805 requires forwarding x-litellm-response-cost through Switchyard routes when cost data is available. The PR adds this header to FORWARDED_UPSTREAM_HEADERS in `crates/switchyard-server/src…
Out of Scope Changes check	✅ Passed	The PR changes only the upstream response-header allowlist and adds a focused regression test. These changes directly support issue #805. No unrelated changes are present in the reviewed diff.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 1 files.

---

_{A rabbit checks the headers in a row
Cost data now has room to flow
Approved names pass the gate
Authorization must wait
Tests keep the boundary straight}

_{Comment @coderabbitai help to get the list of available commands.}

[afourniernv]

I checked this against LiteLLM’s docs. x-litellm-response-cost is their documented response contract, so exact allowlisting here looks like the right boundary and is safer than forwarding every x-litellm-* header. One caveat: LiteLLM says the value can be unreliable for streaming, so this forwards the header when present rather than guaranteeing streaming cost accuracy.

Cost tracking docs · Architecture · Streaming limitation

[deepujain]

Agreed, thanks for checking against their docs. The PR forwards the header verbatim only when LiteLLM sends it and makes no claim about streaming cost accuracy; no behavior change is needed on that front. I can add a note to the PR description capturing the streaming caveat if that would be useful.

[afourniernv]

LGTM. Approved. We’ll merge this once the 0.3 release code freeze ends. Thank you for the contribution!