Skip to content

INT-228 + INT-226: Attribute Snowflake Cortex calls to Snowflake and bill them as token counts - #26

Open
anassg-lago wants to merge 4 commits into
mainfrom
int-228
Open

INT-228 + INT-226: Attribute Snowflake Cortex calls to Snowflake and bill them as token counts#26
anassg-lago wants to merge 4 commits into
mainfrom
int-228

Conversation

@anassg-lago

@anassg-lago anassg-lago commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Closes INT-228 and INT-226. Both tickets in one PR: the token-billed set does nothing until the wrapper stamps "snowflake", and stamping without the set entry would make every Cortex call report a structural price miss on the error hook. Shipping them together means no intermediate state is wrong.

What was broken

Cortex answers chat completions at https://<account>.snowflakecomputing.com/api/v2/cortex/v1/chat/completions, so customers reach it with the ordinary OpenAI client and a base URL. The response is an ordinary chat completion — nothing in it names Snowflake — so inferProvider/_infer_provider returned "openai" and every event went out labelled as OpenAI usage for tokens Snowflake charged in credits.

The severity is mislabelling, not a silent under-bill, and the brief said otherwise. Checked against the live OpenRouter catalogue on 2026-08-25 (430 exact keys), every model id this surface actually serves — claude-sonnet-4-5, claude-opus-4-5, llama3.1-70b, openai-gpt-4.1, openai-gpt-5, snowflake-arctic-embed-mmisses, so price mode already fell back to token events plus an error report per call. Brief §5 has been corrected. What the stamp buys:

  1. Correct attribution on every event.
  2. Silence on a miss that is structural and permanent, instead of a false alarm per request.
  3. Closing the accident: a bare gpt-4.1 does hit that catalogue, so a Cortex model renamed to a bare vendor spelling would have started pricing at OpenAI's public rate against usage Snowflake bills in credits.

What changed

  • The single if became an ordered (path, provider) table. Second surface to need a base-URL override, and Ramp is next; a third if is where this goes wrong. First match wins, so entries stay most-specific-first. A test asserts every provider the table can produce is one emit() bills as token counts — otherwise a hint quietly converts a priceable call into a permanent price miss.
  • Matches the path /api/v2/cortex/, never the host — the same reason the Databricks row matches /ai-gateway/mlflow/ and not /ai-gateway/. The SQL API this SDK's own gateway reader drives lives on that host at /api/v2/statements, and a warehouse query is not model inference. Requiring the trailing slash also keeps /api/v2/cortexsomething out, and costs nothing: a working OpenAI base URL always carries a segment after cortex.
  • "snowflake" added to TOKEN_BILLED_PROVIDERS, and deliberately NOT to the vendor map. Snowflake bills Cortex in credits at an edition/region/contract rate published in no API, view or account table the SDK could read. Cortex serves Anthropic's and OpenAI's models under their real names, so a vendor prefix would let claude-opus-4.8 through Cortex match Anthropic's own rate — a confident mispricing rather than an honest miss. Both the absence and the reason are now pinned by a test.
  • Token counts only. No credit rate, no config field, no cost event, and no forced mode: "tokens" anywhere — the provider set is what decides, so a customer running price mode globally already behaves correctly.
  • Neither Snowflake surface enters the OpenAI-shaped sets. Both are additive, established from real rows in INT-224 (TOKENS equals the sum of every TOKENS_GRANULAR value on 24 of 24 captured rows; the wire agrees). Notes added at both sets so the next reader does not "complete" them by wire shape.

Tests

15 new tests per repo, mirrored one-for-one, all named the same. Removing just the Snowflake row from the table fails 8 of them in each repo — same 8, verified in both.

The stamp is asserted on a wrapped call, streaming and non-streaming, not on the helper in isolation: pinning only the helper is how the missing stream hint survived a green suite once already. Also covered: a Snowflake host that is not Cortex (no false stamp), a plain OpenAI client (byte-identical behaviour), a params object reused across two calls (nested stream_options untouched), a fully-qualified db.schema.model (customer's spelling preserved, no price match), a client whose base-URL getter throws, and a snowflake row under global price mode (token events, nothing on the error hook).

Rebase notes

Branched off main, not off the two open Snowflake PRs. Independent of both:

  • Account for additive cache counts in the total_tokens guard #23 (and its JS twin) (total_tokens additive cache) owns the token-shape half of INT-228's acceptance criteria — Cortex reports cached_tokens additively on an OpenAI wire, and that fix is what stops those tokens being folded into output. Nothing here changes it. The fixtures in this PR are deliberately uncached, so the numbers read the same with or without that branch.
  • INT-221: Snowflake Cortex REST adapter #24 (and its JS twin) (INT-221 REST adapter) touches only gateway/. No overlap.

Merge order does not matter; no conflicts expected in either direction.

Not in scope

Streaming already injects stream_options.include_usage, and the hint already has no default on any stream path (a compile error in JS) — both predate this PR and are re-asserted rather than changed. Cortex's documented per-model limitations (audio input, vision, tool calling, max_completion_tokens) are request-shaping concerns the wrapper never touches; nothing here can regress them.

Snowflake bills Cortex in credits, at a per-credit rate that depends on
edition, region and contract and is published in no API, view or account
table the SDK could read. There is no per-token rate to find now and no
later refresh that could supply one, so token counts are the complete
answer for this provider rather than a degraded one — the same structural,
permanent miss TOKEN_BILLED_PROVIDERS already covers for Databricks.

"snowflake" stays out of _VENDOR_MAP deliberately, and that absence is the
load-bearing half: Cortex serves claude-sonnet-4-5 and openai-gpt-5 under
those very names, so a vendor prefix would let a near-miss model string
match Anthropic's or OpenAI's own OpenRouter rate — a confident mispricing
of a call Snowflake charged in credits, rather than an honest miss.

Neither Snowflake surface enters _OPENAI_SHAPED_APIS and "snowflake" stays
out of _INPUT_INCLUDES_CACHE_READ: both are additive, measured on real rows
in INT-224, and the notes say so where the next reader will look.
Cortex is OpenAI-wire-compatible, so customers reach it with the ordinary
openai.OpenAI client pointed at a Snowflake base_url. Nothing in the
response names Snowflake, so _infer_provider answered "openai" and every
event went out labelled as OpenAI usage for tokens Snowflake charged in
credits. Checked against the live OpenRouter catalogue, every model id this
surface serves misses, so price mode already fell back to token events —
what the stamp buys is correct attribution, silence on a miss that can
never be fixed, and closing the accident where a Cortex model renamed to a
bare vendor spelling starts pricing at OpenAI's public rate.

The single `if` becomes an ordered (path, provider) table: this is the
second surface to need a base_url override and Ramp is next. First match
wins, so entries stay most-specific-first, and a test asserts every
provider the table can produce is one emit() bills as token counts.

Matches the path /api/v2/cortex/, never the snowflakecomputing.com host —
the same reason the Databricks row matches /ai-gateway/mlflow/ and not
/ai-gateway/. The SQL API this SDK's own gateway reader drives lives on
that host, and a warehouse query is not model inference.

The stamp is asserted on a wrapped call, streaming and non-streaming, not
on the helper in isolation: pinning only the helper is how the missing
stream hint survived a green suite once already.
The base-URL fixtures carried the capture account's real hostname,
`YKTMXSA-YHC49987.snowflakecomputing.com` — an addressable host identifying
a live account, on its way into a public repo. INT-224 added a fixture-hygiene
guard for exactly this pattern, but that guard globs `*.json` under the
fixtures directory, so test source is outside what it can see.

Swapped for `example-account.snowflakecomputing.com`, the placeholder the
hygiene tests already whitelist. Nothing about the assertions changes: the
hint table matches on the `/api/v2/cortex/` path, never the host.
The INT-224 entry quoted the real value a capture arrived with, `USER$RAFSARK`
— the same class of identifier the hygiene guard it describes exists to keep
out of this repo, published in the changelog instead. The sentence needs the
shape, `USER$<login>`, not a real login; that shape is still there.

Riding on this branch rather than its own PR: it is one line of docs, the
review queue is the bottleneck, and it is the same scrub as the commit before it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant