Repository navigation
egress: an "inject" rule intercepts only the claims that inject on the host; query and body credentials - #309
Merged
Merged
Conversation
…e host, and a credential can fill a query or body placeholder (#308) A pool rule with intercept "inject" terminates TLS only for a claim holding a credential for the host and stays a plain rule for every other claim, so a control plane can bind per-user keys to any host the operator allows without breaking HTTP/2, WebSocket or pinned clients for the claims that hold none. Such a rule cannot carry a secret. inject.query swaps a placeholder query value byte-exactly; body: true also covers a form or JSON body of known length up to 64 KiB. A value a response header echoes back reaches the guest as the placeholder. The egress usage event names the injected credentials.
…ub their echoes, and a JSON body fills only the named member Review of the first commit: an earlier inject rule shadowed a later always rule, dropping its secret and method gate; a JSON body filled any string equal to the placeholder, which a guest could store and read back; Graph-style paging links echoed a query key in the response body; an empty credential still intercepted; and the SOCKS door refused a host the claim injects on. The credentials leg on bare metal now covers the inject pool.
…nd \/ escapings are scrubbed, and a JSON body fills only top-level members A guest could fetch a reflected value in pieces with Range, each piece too short to match; the proxy now drops Range and If-Range with Accept-Encoding on a request that carried a query credential. encodeURIComponent-style and PHP/Graph-style JSON echoes escaped past the scrub. Filling a nested JSON member let the guest choose where the value lands.
…m or JSON body A guest repeating the placeholder made the proxy expand every copy into the full value before sending: 3600 pairs in a 64 KiB form with an 8 KiB credential allocated about 29 MB on the host per request. Filling the first match bounds the growth by the value length.
…t modes, and a flat inject view The scrub built a strings.Replacer and its escaped forms per response, copied every body byte three times through an 8 KiB chunk, and searched for a held-back tail in quadratic time; it now precomputes the forms and a first-byte table once, reads straight into a pooled 16 KiB half buffer, and checks a tail only where an old can start. A/B, six interleaved runs: a query-credential request -49%, a form or JSON fill -42%/-38%, a 64 KiB scrubbed page -52% and 24.8 KiB to 160 B, an event stream -65%. The claim inject view drops a map lookup per entry (64 entries, no match: 1004 to 580 ns per CONNECT). Intercept modes are ordered by rank, so EvalHost picks the highest and EvalInner reuses it; query and JSON fills skip a document that never names the parameter.
…encoded parameter name is filled Echo forms were tried in credential-name order, so a filled app id or app secret matched inside an app token (id|secret) and the rest of the token reached the guest; the forms now try longest first. The name pre-check compared the decoded name with raw query bytes and skipped a name a client percent-encodes, such as user[token].
… an escaped JSON name is filled Longest-first only ordered whole matches inside one read: when a read ended right after an app id that also starts the app token, the id was replaced at once and the token's tail streamed out in the next read. A match is now held while a longer echo may still complete from it. The JSON pre-check compared the decoded name with raw bytes and skipped a key written with escapes; it now skips only a body with no backslash.
…s the JSON-escaped form and the short-body restore The always rule outranks every other, so EvalHost returns on it again instead of scanning on (-3% per CONNECT). A JSON-escaped echo differs from the raw value only for a credential holding a quote, backslash or tab, which no query key does. A body shorter than its Content-Length fails the upstream request with or without the restored prefix. The SOCKS docs say an inject rule splices on 1080.
…horter comments Proxy.inject returns a scrubber and the test send returns a reply, so each moves below the block of the type it produces (/code layout). echo becomes echoForm, no longer shadowed by test locals named echo. Six comments drop restated facts.
…dsmoke shares one context The tunnel gate is one condition instead of a switch with an empty case, the intercept decoder has no fallthrough, and the header scrub calls ReplaceAll alone (it returns the value untouched on no match). The pool buffer size is written as the two scrub chunks newScrubReader splits it into. credsmoke creates one context in main; tests table their named cases with t.Run and share a chunked reader; docs name the CONNECT cost of an inject rule and that only all-true intercept policies refuse SOCKS5. A/B against 8b1ad8a, six interleaved runs: geomean +0.3%, no path moved.
… whole one The hold only covered a longer echo starting where the shorter match did, so with B = X+A+Y split right after A the scan wrote X raw, replaced A and let Y through. A match within the last echo length of a read now waits whenever an unfinished echo starts at or before it, which is what partial already finds; the same rule covers the prefix case and drops the held flag. The egress docs name the JSON escapes of a quote, backslash or tab as outside the scrub. A/B against 940889d, six interleaved runs: -0.1%.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #308 (core plus bounded body substitution; WebSocket, HTTP/2 inside interception, and the rollout notes are out of scope).
What changes
Conditional interception. A pool rule's
intercepttakestrue,falseor"inject". An"inject"rule terminates TLS for a CONNECT only when the claim holds a non-empty credential whose hosts match. For every other claim it is a plain rule: its tunnels splice (origin certificate, HTTP/2, WebSocket and pinning intact) and plain HTTP forwards. The operator decides which hosts may be intercepted; the claim'sinjectentries decide which of them are.{"host": "*", "intercept": "inject"}Query and body credentials.
injecttakesquery(a parameter name) as the alternative toheader, and then requires a placeholder.<query>=<placeholder>pair of the URL gets the value, URL-escaped. The placeholder may arrive percent-encoded, and every other byte of the query stays as sent.body: true(opt-in per credential) and no placeholder in the URL, the first such pair of anapplication/x-www-form-urlencodedbody, or the first top-level string member of that name holding the placeholder in anapplication/jsonbody, is filled instead. The body must have a knownContent-Lengthof at most 64 KiB; any other body streams unchanged.Echo scrub. On a request that carried a
querycredential, the value reaches the guest as the placeholder in every response header and in the response body. This covers the value as sent, query- or path-escaped, and with/written\/as JSON may (a redirect'sLocation, a Graphpaging.nextlink, an echoed URL).Accept-Encoding,RangeandIf-Rangeon such a request, so the transport decodes compression itself and no range splits a value.headercredentials are not scrubbed: origins do not reflect them, and scrubbing them would put decompression and a scan on every streamed model response.Telemetry. The
egressusage event names the injected credentials insecret(gh,claim:<entry>), as the audit line does.Design decisions
"inject"rule cannot carrysecret. It acts as a plain rule for claims without a credential, andEvaldeliberately skips always-intercept rules so a pool secret never rides a spliced or plaintext request. Refusingsecreton"inject"letsEvalmatch the rule as a plain allow. A pool secret such as a gateway key stays on its own rule, unchanged.false < "inject" < true.EvalHostpicks the highest mode among the host's rules (the first rule on a tie), andEvalInneradmits only rules of that mode. So[{"*","inject"}, {"api.github.com", true, secret, GET}]keeps that host's secret and method gate for every claim, whatever the order.URL.Query()/Encode. Those re-sort and re-escape the whole query. Names and values are compared unescaped (@as%40,user[token]asuser%5Btoken%5D), and only the filled value's bytes change. A form body uses the same routine.Locationand, on the Graph API, in everypaginglink of the response body. A header-only scrub would hand the guest the key on the first paged call.query. It never fills any string equal to the placeholder, and never a nested member, so a guest cannot choose a field where the key lands and is read back.id|secret) within one read. A match near the end of a read waits whenever an unfinished echo starts at or before it (what the held-back tail check already computes), so prefix, middle and suffix containment are all safe at any read boundary; the hold stays under one echo length.Secretsmethod. The tunnel decision calls the existingCredentials(host), which returns nil with 0 allocs when nothing matches. It allocates only on a match, which then takes the intercept path.Known limits (documented, or for the client side):
\/(including JSON escapes of a quote, backslash or tab, and HTML-safe&, none of which a query key contains);appsecret_proof) cannot do so from a placeholder.SSL_CERT_FILE,REQUESTS_CA_BUNDLE,NODE_EXTRA_CA_CERTS)."inject"on for an already-served pool key is refused liketrue, since goldens must carry the root.Hot path
Spliced tunnels and forward requests do no new work. An intercepted request without a claim credential allocates nothing new. The body read runs only for a
bodycredential on a matching request, and the scrub only for a request that carried aquerycredential.Per CONNECT (
BenchmarkTunnelDecision, 0 allocs in every arm):"inject""inject"trueThe production
claimSecrets.Credentialscosts 2 ns with no inject entries, and 586 ns for a claim with 64 entries and no match. The inject view is a flat slice, with no map lookup per entry.Credential and scrub paths: the last two commits against 4c3a012, six interleaved runs (A,B,B,A,A,B),
benchstatp=0.002:Credentials, 64 entries, no matchHow the scrub got there:
Contains+ReplaceAll, which allocates nothing when nothing matches.Tests
access\u005ftoken) is filled."inject"rule intercepts only for a claim with a non-empty credential and splices otherwise (origin certificate on the spliced leg). The SOCKS5 door splices through it.Locationand from a gzip-negotiated body;Rangerequest cannot split the value.InterceptModeJSON round trip and refusals;Validaterefusessecreton"inject".headerandquery;queryneeds a placeholder;bodyneedsquery; equality sees query and body changes."inject"parses, and needsegress_caliketrue.Review and evidence
Review. Two rounds of reuse/simplification/efficiency/altitude and per-file style ledgers, plus three adversarial correctness passes, ran on this branch; each finding was fixed with a test or skipped with a recorded reason.
Range, escaping and nested-JSON issues fixed in 4c3a012, and verified 4c3a012 on its own isolated instance.Gates on 7faf41d:
GOWORK=off make go-lint: 10 ×0 issues.(5 modules × linux/darwin, plusfmt --diff).asl -forwarder=false ./...is clean on darwin and linux for sandboxd, sdk/go and e2e.go test -race: sandboxd in full on macOS; egress, pool, types and config in agolang:1.27.1linux/arm64 container.Bare metal. The bare-metal testbed ran
scripts/credentials-e2e.sh, extended here with a second pool whose only rule is{"host":"*","intercept":"inject"}. Result on the head commit:The inject leg proves the origin received the value without reading its echo, since the scrub erases echoes:
timestampquery credential turns the origin's/time/validanswer totrue.content-lengthmatches the filled form and JSON bodies.It also checks:
claim:SERP;Not covered on hardware: an already-open spliced tunnel staying spliced after a
PATCH. It follows from the per-CONNECT decision and is documented.