Skip to content

Validate gateway destinations before credentialed dispatch - #98

Merged
CompleteDotTech merged 3 commits into
mainfrom
fix/gateway-destination-validation
Nov 13, 2026
Merged

CompleteDotTech merged 3 commits into
mainfrom
fix/gateway-destination-validation

Conversation

@CompleteDotTech

@CompleteDotTech CompleteDotTech commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Gateway route configuration accepted arbitrary destinations for credentialed study requests, so a misconfigured or hostile .env.local could send route credentials and study state to any host.

Change

  • Pins each route to its exact approved HTTPS URL and validates before client construction and again before each dispatch (before study state is serialized). Configuration is copied, the policy mapping is immutable (MappingProxyType), and redirects are disabled at client construction and per request.
  • Errors never echo untrusted URLs.
  • The endpoint-policy owner approved this exact-URL policy on 2026-11-12. README, the policy comment and the error message now describe it as approved and enforced.
  • CHANGELOG.md entry: a worker whose .env.local points routes 3-5 at any other host, port or path now fails at startup, before any credential is sent.
  • Rebased onto current main (6532cc8).

No operator .env.local was inspected, no live worker configuration was changed and no provider request was made. This PR does not establish live route qualification.

Closes #88

Validation

Local, Python 3.14.8, dspy/gepa/typesafe-sdk absent:

  • Full pytest -q: 737 passed, 79 skipped (main baseline is 606 passed / 79 skipped; the added tests are the new gateway cases).
  • pytest -q tests/test_gateway_routes.py: 151 passed, 1 skipped (TypeSafe SDK absent).
  • ruff check src tests examples: clean.
  • No file listed in MANIFEST.sha256 is touched.
  • Uses fake credentials and local httpx.MockTransport only.

An independent code review passed the security mechanics. Its mutable-policy finding was addressed with MappingProxyType and regression coverage, and its follow-up NITs (stale wording, CHANGELOG, rebase) are addressed here.

Not executed: live or paid provider calls, live worker or route qualification, private destination-compatibility review against operator configuration.

🤖 Generated with Claude Code

@CompleteDotTech CompleteDotTech left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent offline review of b9e57f8; freshly rechecked unchanged head and full patch. No actionable code defect found in exact destination validation, config copying/revalidation before state serialization, immutable policy, or redirect blocking.

Validation: full pytest 681 passed / 46 optional-integration skips; gateway tests 152 passed with typesafe-sdk 0.7.0 installed; Ruff passed. An additional 60 synthetic checks covered all five routes, control-character/Unicode/string-subclass/wrong-route destinations, and relative/scheme-relative/HTTP/HTTPS/self redirects, using MockTransport only. Exact-head hosted tests also passed: https://github.com/Jev-Engineering/TypeWright/actions/runs/37119106070.

This is a COMMENT, not merge approval. Endpoint-policy owner approval and private live-configuration compatibility review for #88 remain unresolved gates. Passing offline tests does not qualify live providers or authorize adoption. No real credentials, operator .env.local, study inputs, or provider calls were used.

@CompleteDotTech

Copy link
Copy Markdown
Contributor Author

Fresh-context review (round 1) of head b9e57f8

Scope: I read the full diff (examples/ai-gateway/paced_routes.py, examples/ai-gateway/README.md, tests/test_gateway_routes.py), the touched code paths (configurations, make_backend, Native, ManagedAlias, HTTPFailure/failure), the other URL consumers in examples/ai-gateway/ (check_native.py hardcodes the Vercel URL, and index.ts only lists models), and issue #88. I made no provider or network calls. The tests use fake keys and httpx.MockTransport only.

Validation (run locally, Python 3.14.8, dspy/gepa/typesafe-sdk absent)

  • PR head as-is (base e259dac, behind main): pytest 678 passed, 49 skipped. ruff check src tests examples: clean.
  • PR merged locally into current main 6532cc8 (merges cleanly): pytest 737 passed, 79 skipped. ruff: clean. The main baseline is 606 passed, 79 skipped. The +131 matches the new parametrized tests exactly (23+3+1+23+5+5+1+69+1), so nothing regressed and no skips changed.
  • tests/test_gateway_routes.py alone: 151 passed, 1 skipped. This matches the PR body.
  • Hosted CI checks on the PR are green (core 3.11/3.12/3.13, optional-contracts).
  • MANIFEST: the PR touches no file listed in MANIFEST.sha256, and it changes no schemas or pinned fixtures. (sha256sum -c MANIFEST.sha256 fails on pristine origin/main too, with the same 32 mismatches and 29 missing src/s1compiler/* files. That is not caused by this PR.)

Correctness and invariants

  • The destination is validated by exact string equality against a per-route constant in three places:
    1. In configurations(), before any key is read from settings (paced_routes.py:421).
    2. In Native.__init__, before httpx.Client is built with the bearer header (:170).
    3. In Native.evaluate, before program/state are touched or serialized (:180).
  • The validated constant, not the config value, is what gets posted (:191).
  • Config is copied (:173), so mutating the caller's config has no effect.
  • Redirects are disabled on the client and per request. A 3xx response becomes a non-retryable HTTPFailure, and the error text holds only the status code.
  • Error text never includes the URL.
  • Route 6 (Pollinations) is still refused outright.
  • Everything stays inside the example. It touches no import boundary, DSPy/GEPA placement, split, budget or identity check, and adds no eval/exec/pickle or real credentials.
  • The tests check behavior, not strings:
    • The client-construction spy proves no client was built.
    • The evaluate(None, object()) test proves the URL is rejected before the program or state is touched.
    • The redirect test forces client.follow_redirects = True and still sees exactly one request, so it would fail if the per-dispatch follow_redirects=False were removed.
    • The dispatch test checks the real request URL, the auth header and the state body.
    • The .env.example consistency test pins the constants to the public example, so the test does not compute its expected values with the code it is testing.
  • Issue Gateway routes send keys and study state to any URL set in .env.local #88 acceptance: the stricter option the issue offered (exact URLs pinned in code) is implemented. It covers https-only and rejects userinfo, query, fragment, alternate hosts and ports. The owner approval has now been given, as the coordinator noted.

Findings

  1. NIT: examples/ai-gateway/README.md:120 and :131-139, paced_routes.py:24 and :39. The docs, the code comment and the user-facing error still say "Proposed ... policy", "a proposal requiring endpoint-policy owner approval before live use" and "the proposed exact HTTPS policy". With exact-URL pinning now approved, this will be stale on merge. An operator who hits the error will read it as a draft policy that might be relaxed. The PR body's line "Endpoint-policy owner approval ... remain pending" is likewise out of date. Suggest rewording to "approved exact endpoint policy (Gateway routes send keys and study state to any URL set in .env.local #88)" and keeping the "no .env.local inspected / no live worker changed" caveat, which is still true.
  2. NIT: there is no CHANGELOG.md entry. The change alters live-tooling behavior: a worker whose .env.local points routes 3 to 5 at any other host, port or path now fails closed at configurations(). The repo's convention adds a CHANGELOG line when documented behavior changes.
  3. NIT: the branch is behind main (base e259dac vs 6532cc8). It merges cleanly and passes after merge (counts above), so this matters only for review hygiene.

REVIEW CLEAN

The exact HTTPS URL pinning for gateway routes is now owner-approved
(#88). Replace "proposed / pending approval" wording in the README,
the policy comment and the user-facing error with the approved,
enforced policy, keeping the caveat that no operator .env.local was
inspected and no live route qualification was done. Tighten the two
tests that match the error text to the new wording, and add a
CHANGELOG entry for the fail-at-startup behavior change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CompleteDotTech
CompleteDotTech force-pushed the fix/gateway-destination-validation branch from b9e57f8 to edbc82d Compare November 13, 2026 02:49
@CompleteDotTech

Copy link
Copy Markdown
Contributor Author

Fresh-context review (round 2) of head edbc82d

I reviewed the diff, the code it touches (examples/ai-gateway/paced_routes.py, src/typewright/backends.py ManagedBackend, PacedBackend retry and failure() classification), .env.example, check_native.py, and the new tests. I did not rely on the PR description.

Executed locally

Python 3.14.8, PR head checked out in an isolated worktree, PYTHONPATH set to the worktree's src, dspy/gepa/typesafe-sdk absent:

  • pytest -q: 737 passed, 79 skipped. The main baseline (6532cc8) is 606 passed / 79 skipped. The +131 difference matches the new cases exactly: 23 unsafe URLs x 5 parametrized uses = 115, plus 3 + 1 + 5 + 5 + 1 + 1.
  • ruff check src tests examples: clean.
  • None of the four touched files appears in MANIFEST.sha256, and the PR does not touch schemas/.

What I verified

  • Every credentialed dispatch path is covered. SYSTEMONE_URL is read only in configurations(). make_backend calls configurations() (which now validates, paced_routes.py:421) before it reads any key (paced_routes.py:447). Native.__init__ validates before httpx.Client is built (:170). Native.evaluate validates its own copied config before the model check and before building the payload (:180). check_native.py and make_calibration_backend use hardcoded URLs or the direct route only.
  • Drift fails closed and is not retried. A dispatch-time ConfigurationError has no status, so failure() returns non-transient and PacedBackend.evaluate re-raises it. It is not turned into a zero-quality example.
  • Redirects are refused. A redirect becomes HTTPFailure (non-transient), and its message does not include the Location header. Because the per-request follow_redirects=False overrides the client setting, the test that sets client.follow_redirects = True would see a second request if the per-request flag were removed. The test therefore checks real behavior.
  • The tests are not tautological. The drift test calls evaluate(None, object()), so moving the check after program.model would raise AttributeError and fail the test. The tests compare against literal constants and .env.example. They do not recompute results with validate_destination. Only fake keys and MockTransport are used.
  • Error messages do not echo the untrusted URL, and the test asserts this.
  • AGENTS.md invariants hold. There is no DSPy/GEPA import, no eval/exec/pickle and no real credentials. No result is presented as measured.
  • Issue Gateway routes send keys and study state to any URL set in .env.local #88 acceptance is met by the stricter option the issue offers (exact URLs pinned in code), which was owner-approved on 2026-11-12. HTTPS, userinfo, query, fragment and host are all enforced through exact equality. The issue's "check .env.local first" step is openly disclosed as not done, and the README tells operators to confirm their destinations before restarting.

Findings

  • NIT examples/ai-gateway/README.md:144: "reject later destination drift before serializing study state" holds for the adapter's wire payload. However, ManagedBackend.evaluate has already canonicalized the state in-process (src/typewright/backends.py:350) and reserved one budget slot (backends.py:372) before Native.evaluate rejects the drift. Nothing is sent over the network, so this is wording only. One rejected call also uses one budget unit, which is harmless because the run aborts. "Before the request payload is built or sent" would be more precise.
  • NIT examples/ai-gateway/README.md:139: this line is 125 characters, while the rest of the paragraph wraps at about 80.

No BLOCKING findings.

REVIEW CLEAN

Not executed: live or paid provider calls, live worker or route qualification, inspection of any operator .env.local.

@CompleteDotTech

Copy link
Copy Markdown
Contributor Author

Coordinator: approved for merge on head edbc82d after review round 2 posted REVIEW CLEAN (round 1 also clean; its NITs were fixed in edbc82d). All 10 hosted checks pass on this exact head.

Left as noted NITs, not fixed (README wording only; changing it would move the head and need another review round):

  • examples/ai-gateway/README.md:144: 'before serializing study state' is accurate for the network, but state is serialized locally (backends.py:350) and one budget slot is used before the URL is rejected. Nothing is sent. More precise wording: 'before the request payload is built or sent'.
  • examples/ai-gateway/README.md:139: line length.

.env.local host check: no .env.local exists in this checkout or anywhere under the operator home directory on this machine, so there were no route 3-5 hosts to compare with the pinned URLs. Exact pinning is kept as approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gateway routes send keys and study state to any URL set in .env.local

1 participant