INT-228 + INT-226: Attribute Snowflake Cortex calls to Snowflake and bill them as token counts - #26
Open
anassg-lago wants to merge 4 commits into
Open
INT-228 + INT-226: Attribute Snowflake Cortex calls to Snowflake and bill them as token counts#26anassg-lago wants to merge 4 commits into
anassg-lago wants to merge 4 commits into
Conversation
Snowflake bills Cortex in credits, at a per-credit rate that depends on edition, region and contract and is published in no API, view or account table the SDK could read. There is no per-token rate to find now and no later refresh that could supply one, so token counts are the complete answer for this provider rather than a degraded one — the same structural, permanent miss TOKEN_BILLED_PROVIDERS already covers for Databricks. "snowflake" stays out of _VENDOR_MAP deliberately, and that absence is the load-bearing half: Cortex serves claude-sonnet-4-5 and openai-gpt-5 under those very names, so a vendor prefix would let a near-miss model string match Anthropic's or OpenAI's own OpenRouter rate — a confident mispricing of a call Snowflake charged in credits, rather than an honest miss. Neither Snowflake surface enters _OPENAI_SHAPED_APIS and "snowflake" stays out of _INPUT_INCLUDES_CACHE_READ: both are additive, measured on real rows in INT-224, and the notes say so where the next reader will look.
Cortex is OpenAI-wire-compatible, so customers reach it with the ordinary openai.OpenAI client pointed at a Snowflake base_url. Nothing in the response names Snowflake, so _infer_provider answered "openai" and every event went out labelled as OpenAI usage for tokens Snowflake charged in credits. Checked against the live OpenRouter catalogue, every model id this surface serves misses, so price mode already fell back to token events — what the stamp buys is correct attribution, silence on a miss that can never be fixed, and closing the accident where a Cortex model renamed to a bare vendor spelling starts pricing at OpenAI's public rate. The single `if` becomes an ordered (path, provider) table: this is the second surface to need a base_url override and Ramp is next. First match wins, so entries stay most-specific-first, and a test asserts every provider the table can produce is one emit() bills as token counts. Matches the path /api/v2/cortex/, never the snowflakecomputing.com host — the same reason the Databricks row matches /ai-gateway/mlflow/ and not /ai-gateway/. The SQL API this SDK's own gateway reader drives lives on that host, and a warehouse query is not model inference. The stamp is asserted on a wrapped call, streaming and non-streaming, not on the helper in isolation: pinning only the helper is how the missing stream hint survived a green suite once already.
The base-URL fixtures carried the capture account's real hostname, `YKTMXSA-YHC49987.snowflakecomputing.com` — an addressable host identifying a live account, on its way into a public repo. INT-224 added a fixture-hygiene guard for exactly this pattern, but that guard globs `*.json` under the fixtures directory, so test source is outside what it can see. Swapped for `example-account.snowflakecomputing.com`, the placeholder the hygiene tests already whitelist. Nothing about the assertions changes: the hint table matches on the `/api/v2/cortex/` path, never the host.
The INT-224 entry quoted the real value a capture arrived with, `USER$RAFSARK` — the same class of identifier the hygiene guard it describes exists to keep out of this repo, published in the changelog instead. The sentence needs the shape, `USER$<login>`, not a real login; that shape is still there. Riding on this branch rather than its own PR: it is one line of docs, the review queue is the bottleneck, and it is the same scrub as the commit before it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes INT-228 and INT-226. Both tickets in one PR: the token-billed set does nothing until the wrapper stamps
"snowflake", and stamping without the set entry would make every Cortex call report a structural price miss on the error hook. Shipping them together means no intermediate state is wrong.What was broken
Cortex answers chat completions at
https://<account>.snowflakecomputing.com/api/v2/cortex/v1/chat/completions, so customers reach it with the ordinary OpenAI client and a base URL. The response is an ordinary chat completion — nothing in it names Snowflake — soinferProvider/_infer_providerreturned"openai"and every event went out labelled as OpenAI usage for tokens Snowflake charged in credits.The severity is mislabelling, not a silent under-bill, and the brief said otherwise. Checked against the live OpenRouter catalogue on 2026-08-25 (430 exact keys), every model id this surface actually serves —
claude-sonnet-4-5,claude-opus-4-5,llama3.1-70b,openai-gpt-4.1,openai-gpt-5,snowflake-arctic-embed-m— misses, so price mode already fell back to token events plus an error report per call. Brief §5 has been corrected. What the stamp buys:gpt-4.1does hit that catalogue, so a Cortex model renamed to a bare vendor spelling would have started pricing at OpenAI's public rate against usage Snowflake bills in credits.What changed
ifbecame an ordered(path, provider)table. Second surface to need a base-URL override, and Ramp is next; a thirdifis where this goes wrong. First match wins, so entries stay most-specific-first. A test asserts every provider the table can produce is oneemit()bills as token counts — otherwise a hint quietly converts a priceable call into a permanent price miss./api/v2/cortex/, never the host — the same reason the Databricks row matches/ai-gateway/mlflow/and not/ai-gateway/. The SQL API this SDK's own gateway reader drives lives on that host at/api/v2/statements, and a warehouse query is not model inference. Requiring the trailing slash also keeps/api/v2/cortexsomethingout, and costs nothing: a working OpenAI base URL always carries a segment aftercortex."snowflake"added toTOKEN_BILLED_PROVIDERS, and deliberately NOT to the vendor map. Snowflake bills Cortex in credits at an edition/region/contract rate published in no API, view or account table the SDK could read. Cortex serves Anthropic's and OpenAI's models under their real names, so a vendor prefix would letclaude-opus-4.8through Cortex match Anthropic's own rate — a confident mispricing rather than an honest miss. Both the absence and the reason are now pinned by a test.mode: "tokens"anywhere — the provider set is what decides, so a customer running price mode globally already behaves correctly.TOKENSequals the sum of everyTOKENS_GRANULARvalue on 24 of 24 captured rows; the wire agrees). Notes added at both sets so the next reader does not "complete" them by wire shape.Tests
15 new tests per repo, mirrored one-for-one, all named the same. Removing just the Snowflake row from the table fails 8 of them in each repo — same 8, verified in both.
The stamp is asserted on a wrapped call, streaming and non-streaming, not on the helper in isolation: pinning only the helper is how the missing stream hint survived a green suite once already. Also covered: a Snowflake host that is not Cortex (no false stamp), a plain OpenAI client (byte-identical behaviour), a params object reused across two calls (nested
stream_optionsuntouched), a fully-qualifieddb.schema.model(customer's spelling preserved, no price match), a client whose base-URL getter throws, and asnowflakerow under global price mode (token events, nothing on the error hook).Rebase notes
Branched off
main, not off the two open Snowflake PRs. Independent of both:total_tokensadditive cache) owns the token-shape half of INT-228's acceptance criteria — Cortex reportscached_tokensadditively on an OpenAI wire, and that fix is what stops those tokens being folded intooutput. Nothing here changes it. The fixtures in this PR are deliberately uncached, so the numbers read the same with or without that branch.gateway/. No overlap.Merge order does not matter; no conflicts expected in either direction.
Not in scope
Streaming already injects
stream_options.include_usage, and the hint already has no default on any stream path (a compile error in JS) — both predate this PR and are re-asserted rather than changed. Cortex's documented per-model limitations (audio input, vision, tool calling,max_completion_tokens) are request-shaping concerns the wrapper never touches; nothing here can regress them.