fix(pricing): refresh fallback price table (as_of 2026-08-11) + refresh tool - #226
Merged
Conversation
added 2 commits
August 11, 2026 15:50
…efresh tool Unblocks the freshness gate (test workflow was red: table 46d old > 45d max). Verified against vendor pages (2026-08-11): - anthropic: Opus 4.8 15/75 -> 5/25, Fable 5 -> 10/50 (cache rates match); Sonnet/Haiku 4.5/4.6 lines unchanged (docs.anthropic.com) - openai: added current flagship gpt-5.6-sol/terra/luna; gpt-5/o4 rows carried forward as LEGACY (no longer on the pricing page) - deepseek: v4-pro 0.55/2.19 -> 0.435/0.87, cache-hit rates corrected; vendor notes planned price increase (api-docs.deepseek.com) - google: 3.1 Pro Preview -> 2.00/12.00 (<=200k tier), cache rates corrected (ai.google.dev) - xai/mistral/cohere/meta: carried forward unverified (noted in comment) tools/refresh_price_table.py: mechanical refresh helper — fetches the vendor docs pages (markdown/HTML), extracts tracked-model rates, diffs vs PRICE_TABLE; never edits the table (provenance stays truthful); exit 1 on drift, 2 on unparseable provider (fail-closed). deepseek is client-rendered and prints a manual-verify reminder. Verified live: all tracked models match the current pages.
The 2026-08-11 price refresh (PRICE_TABLE_AS_OF 2026-06-26 -> 08-11) changed opus-4-8 (15/75 -> 5/25) and deepseek-v4-pro (0.55/2.19 -> 0.435/0.87), which the golden cost/savings tests pin exactly (they are an intended guard against UNINTENDED table edits — a vendor-verified refresh updates them together, per their own docstrings): - test_benchmark_cost.py: opus exact-math pin 90.0 -> 30.0 - test_engine.py: estimate_cost pin 90.0 -> 30.0 - test_efficiency.py: flagship/leak pins 52.50/49.00 -> 17.50/14.00; deepseek-v4-pro flagship 9.88 -> 6.09 (multiple 494x -> 304.5x) - test_meter_regression.py + examples/invarium_meter_regression.py: haiku-vs-opus-baseline savings golden 49.00 -> 14.00 - test_savings.py: baseline/floor/savings pins for opus-4-8 and token-reduction baselines (52.50->17.50, 15.75->5.25, 2.25->0.75, 42.00->7.00, 14.90->4.90, 13.50->4.50, 49.00->14.00) All comments carrying the old arithmetic updated to the new rates. Verification: full suite 697 passed / 0 failed (freshness gate green, pricing/catalog/efficiency/savings/regression goldens all green).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
From the 2026-08-11 test-suite/CI audit: the price-freshness gate (
tools/check_price_freshness.py, run on every test.yml invocation) is RED — the checked-in fallback table isas_of=2026-06-26, 46 days > the 45-day max. This blocks the entire test workflow.Changes:
ledger_agent/pricing.py— table refreshed to vendor-verified rates,PRICE_TABLE_AS_OF→ 2026-08-11, with per-provider provenance comments:tools/refresh_price_table.py(new) — the owned refresh process the audit recommended: fetches the vendor docs pages, extracts every tracked model's current rates, diffs vs PRICE_TABLE, never edits the table; exit 1 on drift, 2 on unparseable provider (fail-closed, so as_of can't be stamped on a broken check). DeepSeek's page is client-rendered → prints a manual-verify reminder.Verification: freshness checker passes (
as_of=2026-08-11); 17/17 pricing/catalog tests pass; the refresh tool itself ran LIVE against all vendor pages and reports "All tracked models match the live pricing pages" (exit 0), and re-running--offlinefrom cache reproduces it.