enhancement: spike TypeSafe System One support

[afourniernv]

Overview

This draft preserves an exploratory spike for first-class TypeSafe System One / Jev support in NeMo Relay. The current product signal is limited: the Switchyard team is experimenting with Jev for routing, but we do not currently see broad product demand. The likely outcome is to close this draft without merging after capturing feedback, while retaining the branch and design as a reference if demand emerges.

This is a technical exploration, not a production-support commitment.

• I confirm this contribution is my own work, or I have the right to submit it under this project license.
• I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

• Add a lossless TypeSafeSystemOneCodec and provider-neutral evaluation request, response, question, answer, and usage types.
• Route /systemone, /v1/systemone, and TypeSafe-namespaced System One and model-catalog endpoints through both direct and daemon gateway paths.
• Add bearer-secret configuration, secret redaction, model aliasing, pricing metadata, and bounded SDK-compatible retries for transport failures, 408, 429, and 5xx responses including 529.
• Emit structured evaluation, decision, usage, and billing observability without placing raw questions, state, or answers in OTEL attributes.
• Expose codec types through Rust, Python, Node, and experimental C surfaces, with explicit non-streaming behavior.
• Define explicit evaluation behavior for adaptive controls, response caching, PII projection, and the deprecated NeMo Guardrails integration.
• Add round-trip, conformance, gateway, daemon, observability, binding, and documentation coverage.
• Breaking changes: none intended.

Validation completed:

• just test-rust
• just test-python
• just test-node
• uv run pre-commit run
• Focused TypeSafe gateway and evaluator lifecycle regressions
• Documentation link validation
• Live TypeSafe smoke test returned HTTP 200 and resolved jev-latest to jev-1.13.0; the API credential is not stored in this branch.

Where should the reviewer start?

Start with crates/core/src/codec/typesafe_system_one.rs for the wire contract and lossless mapping, then crates/core/src/api/evaluation.rs for the provider-neutral operation. Review crates/cli/src/gateway/mod.rs and crates/cli/src/typesafe_retry.rs for routing, authentication, and retry semantics. The end-to-end rationale and usage are in docs/integrate-into-frameworks/typesafe-system-one.mdx.

The main review question is whether any part of this spike should be retained upstream now, or whether the draft should simply remain a reference for a future demand-driven implementation.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• Relates to: none

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

_{Comment @coderabbitai help to get the list of available commands.}

[afourniernv]

Follow-up after looking at Switchyard #739 and #762: Relay does not need first-class System One codec support for the Switchyard integration.

The boundaries are:

OpenAI/Anthropic caller
  -> Relay
  -> Switchyard plugin
  -> Switchyard Runner
  -> internal TypeSafe client -> /v1/systemone
  -> selected target through the existing model codecs

Relay owns the outer agent-facing request and response. Switchyard owns the private Jev decision call: libsy defines the I/O-free provider interface, the dedicated TypeSafe client owns the System One JSON, and Runner injects it into the router. The Relay plugin already loads that Runner, so it does not need another codec.

There is still an observability gap. The internal Jev call is not represented as a Relay LLM call, and the current plugin projection drops the selected label, probability map, and decision latency. If we pursue this in Switchyard, that should be handled through a bounded runner observation or plugin mark/metric, not a Relay codec.

A Relay codec would only make sense if Relay itself chose to expose or intercept System One as a first-class public operation. That is not required for the Switchyard plugin, so closing this broader spike was the right boundary.