feat: estimate and report run cost per model

[arunSunnyKVS]

Problem

Reports track total token usage but there is no way to turn that into a cost — the number people actually want to see.

The blocker was not the price data. TokenTracker kept one combined total for the whole run, with no record of which model spent what. That is unpriceable in principle: judgeLlm can differ from attackerLlm, and their rates differ by more than an order of magnitude. The same 100k tokens could be 3 cents or 60 cents depending on a split we were not recording.

There is also a second, unrelated price table already in the tree — a hardcoded Claude-only map in core/src/autonomous/lib/budget.ts used by opfor hunt. That one is correct for hunt (it runs Claude models through the Claude SDK) and is deliberately left alone.

Solution

1. Attribute tokens to models. TokenTracker now keeps a per-model breakdown alongside the existing flat totals. totals is unchanged in behaviour, so the CLI summary, HTML report and extension popup keep working untouched.

The awkward part: a built AI SDK model no longer knows the provider it was configured with — openai-compatible reports itself as custom.chat, which is useless for a price lookup. Rather than thread LlmConfig through a dozen signatures, createModel records identity in a WeakMap that every recording site resolves from the model object it already holds. Run phase (attacker/judge) is derived from the context label withRetry already accepted for logging.

2. Price it. scripts/build-pricing.ts prunes LiteLLM's community price map (2,986 entries) to the providers opfor can reach and vendors it as an inlined module — 557 entries, 43 KB, no Node imports so the extension bundles it.

Lookup tries a short ladder of model-name forms (azure/gpt-4o-mini → gpt-4o-mini → prefix-stripped), then verifies the matched row belongs to the configured provider. Without that guard a mistyped Groq model silently matches OpenAI's row and gets priced at OpenAI's rates — a confidently wrong number, which is worse in a security report than no number.

Design decisions worth reviewing

• Vendored, not fetched at runtime. opfor runs in air-gapped CI and locked-down networks, and a security artifact should produce the same numbers when re-run months later. An optional refresh flag can come later; the network is not on the critical path.
• build:pricing --check is not a blocking CI gate. Unlike build:catalog:check (stale only when this repo changes), this table goes stale when a third party reprices a model. Gating merges on that lets outside events red the build. Run it on a schedule and open a PR instead.
• Unpriced ≠ free. A model we cannot price is reported as such; the total is labelled a lower bound and the missing model is named. Never silently counted as $0.
• "Testing cost", not "total cost". The target's own inference spend is not observable from opfor and is excluded by design.

Changes

core — new src/pricing/

• types.ts — ModelPrice, ModelCost, RunCost
• providerAliases.ts — opfor provider → litellm_provider names; single source of truth shared by the generator and the lookup, so the table cannot contain rows the lookup will never accept
• priceTable.generated.ts — vendored table (generated)
• lookupPrice.ts — candidate ladder + provider guard
• estimateCost.ts — estimateRunCost, formatUsd

core — modified

• execute/tokenTracker.ts — per-model breakdown; totals unchanged
• providers/modelIdentity.ts (new) + providers/factory.ts — identity registry
• lib/llmRetry.ts — model on RetryOptions; roleFromContext
• evaluators/judge.ts, generate/generateAttacks.ts, generate/generateNextTurn.ts, llm/openaiCompatible.ts, run/judge.ts — attribute usage at each recording site
• execute/{types,runAll,runAllBrowser,evaluatorLoop}.ts, report/{types,buildReport,render}.ts — carry and render cost
• browser.ts, package.json — export the pricing surface

runners/cli — commands/run.ts prints testing cost with the per-model split

root — scripts/build-pricing.ts, npm run build:pricing

tests — pricing.test.ts (new), modelIdentity.test.ts (new), tokenTracker.test.ts (extended). 50 new tests; 238 total, 0 failures.

Review fixes (second commit)

Rebased onto master after #230 — the only conflict was render.ts, where #230 rewrote the exec strip this PR adds the cost card to. Both changes kept.

• build:pricing --check was permanently broken. The generator emitted compact JSON.stringify output while the pre-commit Prettier hook reformatted the committed file, so the two never matched. --check reported "stale" even immediately after regenerating, and the documented "unchanged → no-op" fast path never fired — every regeneration produced a 557-line diff. The script now formats its output with the repo's Prettier config before comparing or writing. Both paths verified working.
• MCP baseline scans were invisible to cost. They judge every tool description and resource, but ran without a token tracker — so every MCP run understated cost while complete: true claimed full coverage (no tokens arrived, so nothing could be flagged unpriced). runAll now threads the run tracker through runBaselineScans.
• RunCost.complete now documents that it means "nothing we saw was unpriced", not "nothing was missed".
• costSubLabel labels a role-less bucket unknown rather than mixed.
• Dropped the PRICE_TABLE_VERSION re-export from lookupPrice (AGENTS.md: no barrel re-exports).
• Added roleFromContext tests pinned to the literal context: labels used at call sites, so renaming a log string can't silently drop the role split.
• Added the missing end-to-end test: createModel → withRetry → correct bucket, exercising the WeakMap identity registry that every other test bypasses with hand-built identities.

Issue

N/A

How to test

npm install && npm run build
npm test                            # 233 pass / 0 fail
npm run build:pricing -- --check    # table matches upstream

Then any run prints cost:

opfor run --config tests/e2e/agents/vanilla-chat/opfor.config.json

Worth exercising specifically:

• Set judgeLlm to a different (pricier) model than attackerLlm and confirm the two are priced separately — this is the case the old combined total could not express.
• Point attackerLlm.model at a made-up model name and confirm the run reports it as not priced and marks the total a lower bound, rather than reporting $0.
• Open the HTML report: a Testing Cost card sits beside Token Usage, with a per-evaluator figure inline.

Screenshots

CLI output from a verification run (attacker and judge on different models):

Token usage: 187 input / 42 output (229 total)
Testing cost: ≈$0.000073
   deepseek/deepseek-v4-flash [attacker]: ≈$0.000035
   anthropic/claude-haiku-4-5 [judge]: ≈$0.000038

Accuracy check — estimates vs what a LiteLLM proxy actually billed, across three vendors:

model	ours	proxy reported	delta
deepseek/deepseek-v4-flash	$0.000019	$0.000019	0.0%
anthropic/claude-haiku-4-5	$0.0001	$0.0001	0.0%
openai/gpt-4o-mini	$0.000012	$0.000012	0.0%

Known limitations

• Repeat (cached) input tokens are not counted separately yet, so all input is priced at the full rate. Multi-turn runs against providers that discount repeated context are over-estimated. Deferred deliberately — it needs per-provider verification of whether cached tokens are reported inside or alongside the input total, and getting that backwards is a 2× error.
• Four LLM call sites still record no tokens at all (telemetry/curation.ts ×2, lib/summarizeSessionContext.ts, lib/generateJsonObject.ts). Pre-existing, not introduced here, but it propagates into cost as a slight undercount for runs using telemetry curation. This is why complete: true is documented as the narrower claim. Worth a follow-up. (The MCP baseline-scan gap in this same class is fixed here.)
• The default model for the anthropic provider (claude-3-5-haiku-20241022) has been retired upstream and has no direct Anthropic price, so it reports as unpriced. Unrelated to this PR — that default is about two generations old and probably wants bumping regardless.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added estimated testing costs to run summaries, evaluator results, reports, and CLI output.
  • Costs include per-model token usage, attacker/judge breakdowns, pricing coverage, and unpriced-model notices.
  • Added provider-aware pricing lookup, USD formatting, and browser-accessible cost information.
  • Added cost tracking for baseline scans and evaluator activity.

• Documentation

  • Documented testing-cost reporting, pricing behavior, and known limitations across CLI and browser integrations.

• Tests

  • Added coverage for pricing, model identification, and per-model token tracking.

[coderabbitai]

Warning

Review limit reached

@arunSunnyKVS, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 21 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0610be9a-00d1-40e8-a0b9-d178464a6358

📥 Commits

Reviewing files that changed from the base of the PR and between f2bc2cb and 5ecf1cb.

⛔ Files ignored due to path filters (1)

• core/src/pricing/priceTable.generated.ts is excluded by !**/*.generated.*

📒 Files selected for processing (32)

• README.md
• core/package.json
• core/src/browser.ts
• core/src/evaluators/judge.ts
• core/src/execute/baselineScanner.ts
• core/src/execute/evaluatorLoop.ts
• core/src/execute/runAll.ts
• core/src/execute/runAllBrowser.ts
• core/src/execute/tokenTracker.ts
• core/src/execute/types.ts
• core/src/generate/generateAttacks.ts
• core/src/generate/generateNextTurn.ts
• core/src/lib/llmRetry.ts
• core/src/llm/openaiCompatible.ts
• core/src/pricing/estimateCost.ts
• core/src/pricing/lookupPrice.ts
• core/src/pricing/providerAliases.ts
• core/src/pricing/types.ts
• core/src/providers/factory.ts
• core/src/providers/modelIdentity.ts
• core/src/report/buildReport.ts
• core/src/report/render.ts
• core/src/report/types.ts
• core/src/run/judge.ts
• core/tests/modelIdentity.test.ts
• core/tests/pricing.test.ts
• core/tests/tokenTracker.test.ts
• docs/browser-extension.md
• docs/cli.md
• package.json
• runners/cli/src/commands/run.ts
• scripts/build-pricing.ts

Walkthrough

The PR adds model-level token attribution, provider-aware pricing lookup, run-cost estimation, and cost reporting. It propagates cost data through evaluator results, run summaries, reports, browser exports, CLI output, and documentation.

Changes

Model-attributed cost reporting

Layer / File(s)	Summary
Pricing data and cost calculation core/src/pricing/*, scripts/build-pricing.ts, core/tests/pricing.test.ts, core/src/browser.ts, core/package.json, package.json	Adds pricing types, provider aliases, LiteLLM price-table generation, provider-aware lookup, USD estimation, browser exports, build tooling, and pricing tests.
Model identity and token aggregation core/src/providers/*, core/src/execute/tokenTracker.ts, core/tests/modelIdentity.test.ts, core/tests/tokenTracker.test.ts	Registers provider and model identity, attributes token usage by model and role, preserves unknown usage, propagates child tracker data, and tests aggregation behavior.
LLM call attribution core/src/lib/llmRetry.ts, core/src/llm/openaiCompatible.ts, core/src/evaluators/judge.ts, core/src/generate/*, core/src/run/judge.ts, core/src/execute/baselineScanner.ts	Passes model and attacker or judge role metadata through retry, completion, attack generation, and baseline judging paths into token tracking.
Cost propagation and presentation core/src/execute/*, core/src/report/*, runners/cli/src/commands/run.ts, README.md, docs/*	Adds model token breakdowns and estimated costs to evaluator results and run summaries, renders them in reports and CLI output, and documents pricing behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• KeyValueSoftwareSystems/agent-opfor#154: Introduced baseline scanning and run token-tracking paths extended by this PR.
• KeyValueSoftwareSystems/agent-opfor#163: Added related browser exports and provider model integration.
• KeyValueSoftwareSystems/agent-opfor#224: Added token-tracking infrastructure extended here with model attribution and cost estimation.

Suggested reviewers: jithin23-kv

Sequence Diagram(s)

sequenceDiagram
  participant ModelFactory
  participant LLMCall
  participant TokenTracker
  participant CostEstimator
  participant Report
  ModelFactory->>LLMCall: provide configured model identity
  LLMCall->>TokenTracker: record tokens with model and role
  TokenTracker-->>CostEstimator: provide per-model breakdown
  CostEstimator-->>Report: provide estimated RunCost
  Report-->>Report: render testing cost and pricing coverage

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 78.95% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: estimating and reporting run cost per model.
Description check	✅ Passed	The description covers the problem, solution, changes, issue status, testing steps, screenshots, and known limitations in sufficient detail.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 2

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

⚔️ Resolve merge conflicts 💡

• Resolve merge conflict in branch feat/run-cost-estimation

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/run-cost-estimation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (4)

core/src/execute/runAllBrowser.ts (1)

192-195: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract the repeated usage/cost assignment into a helper.

The pattern result.tokenUsageByModel = tracker.breakdown; result.cost = estimateRunCost(result.tokenUsageByModel); appears three times in this function: once for the partial-failure result, once for the completed evaluator result, and once for the overall report summary. Extract a small helper, for example attachUsageAndCost(target, tracker), so a future change to how cost is derived only needs to happen once.

♻️ Proposed helper extraction

+function attachCost<T extends { tokenUsage?: unknown; tokenUsageByModel?: unknown; cost?: unknown }>(
+  target: T,
+  tracker: TokenTracker
+): void {
+  (target as any).tokenUsage = tracker.totals;
+  (target as any).tokenUsageByModel = tracker.breakdown;
+  (target as any).cost = estimateRunCost(tracker.breakdown);
+}

Also applies to: 235-238, 241-247

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/src/execute/runAllBrowser.ts` around lines 192 - 195, In the
runAllBrowser function, extract the repeated tokenUsageByModel and cost
assignment into a shared helper such as attachUsageAndCost(target, tracker).
Replace the duplicated logic for the partial-failure result, completed evaluator
result, and overall report summary with calls to this helper, while preserving
each result’s existing tokenUsage assignment and cost calculation behavior.

core/src/browser.ts (1)

72-78: 🧹 Nitpick | 🔵 Trivial

Confirm the vendored price table's size impact on the extension bundle.

These new exports pull the full vendored price table (over 100 rows, per pricing.test.ts) into the browser-safe entry point used by the Chrome MV3 extension. Confirm the resulting bundle size increase is acceptable for the extension's size constraints, since MV3 extensions can have stricter packaging limits than a Node CLI.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/src/browser.ts` around lines 72 - 78, Measure the Chrome MV3 extension
bundle after adding the pricing exports from estimateRunCost, formatUsd,
lookupPrice, and PRICE_TABLE_VERSION, including the vendored PRICE_TABLE. Verify
the size increase remains within the extension’s packaging constraints; if it
does not, avoid exposing the full price table through the browser entry point
while preserving the required pricing API.

core/src/pricing/lookupPrice.ts (1)

51-58: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add registry-sourced coverage to the pricing guard.

providerAccepts treats an unlisted usage.provider the same as openai-compatible and accepts any price table key, so new TokenTracker usage can be priced even without an alias entry. Derive VENDORED_LITELLM_PROVIDERS from providerRegistryData / alias validation or add unknown to the alias map so unattributed usage cannot match priced rows.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/src/pricing/lookupPrice.ts` around lines 51 - 58, Update providerAccepts
to distinguish unknown providers from explicitly unverifiable providers: derive
the recognized provider set from providerRegistryData/alias validation, or add
an explicit unknown entry in LITELLM_PROVIDER_ALIASES, and reject unlisted
providers so unattributed usage cannot match priced rows while preserving
openai-compatible behavior.

core/src/execute/evaluatorLoop.ts (1)

251-255: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract a shared helper for attaching token usage, per-model breakdown, and cost. All three sites repeat the identical three-line pattern — read a tracker's totals and breakdown, then call estimateRunCost — to populate a result or summary object. The shared root cause is the lack of one helper function; each site currently must be kept in sync by hand.

• core/src/execute/evaluatorLoop.ts#L251-L255: replace the inline block with a call to a shared helper, e.g. applyCostAndUsage(evResult, evalTracker).
• core/src/execute/evaluatorLoop.ts#L230-L234: replace the inline block with the same shared helper, e.g. applyCostAndUsage(partialResult, evalTracker).
• core/src/execute/runAll.ts#L145-L150: replace the inline block with the same shared helper, e.g. applyCostAndUsage(report.summary, tokenTracker), and let the helper own the "only set fields when usage exists" guard so all three sites share one consistent zero-usage rule.

♻️ Proposed shared helper

// e.g. in core/src/execute/tokenTracker.ts or core/src/pricing/estimateCost.ts
export function applyCostAndUsage<
  T extends { tokenUsage?: TokenUsage; tokenUsageByModel?: ModelTokenUsage[]; cost?: RunCost },
>(target: T, tracker?: TokenTracker): void {
  if (!tracker) return;
  const totals = tracker.totals;
  if (totals.totalTokens <= 0) return;
  target.tokenUsage = totals;
  target.tokenUsageByModel = tracker.breakdown;
  target.cost = estimateRunCost(target.tokenUsageByModel);
}

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core/src/execute/evaluatorLoop.ts` around lines 251 - 255, Extract and export
a shared applyCostAndUsage helper that accepts a result/summary target and
optional tracker, returns without changes when no tracker or total usage exists,
and otherwise assigns totals, per-model breakdown, and estimated cost. Replace
the inline blocks at core/src/execute/evaluatorLoop.ts lines 230-234 and
251-255, and core/src/execute/runAll.ts lines 145-150, with calls to this helper
using partialResult, evResult, and report.summary respectively.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/src/report/render.ts`:
- Around line 527-535: The cost displays in core/src/report/render.ts require
the same incomplete-cost handling: at lines 527-535, update the
executive-summary Testing Cost card to show “Unpriced” when
summary.cost.complete is false and totalUsd is zero, use a ≥ prefix for non-zero
partial lower bounds, and retain ≈ only for complete totals; apply the
equivalent branching at line 242 for the evaluator-header cost span. Use the
existing summary.cost fields and formatUsd path without changing complete,
non-zero behavior.

In `@scripts/build-pricing.ts`:
- Around line 157-159: Validate the parsed upstream price map in the
build-pricing flow before passing it to prune(). Define or reuse a Zod schema
for the expected map and row structure, parse the JSON value through that schema
instead of relying on the Record cast, and surface malformed rows with an
actionable validation error.

---

Nitpick comments:
In `@core/src/browser.ts`:
- Around line 72-78: Measure the Chrome MV3 extension bundle after adding the
pricing exports from estimateRunCost, formatUsd, lookupPrice, and
PRICE_TABLE_VERSION, including the vendored PRICE_TABLE. Verify the size
increase remains within the extension’s packaging constraints; if it does not,
avoid exposing the full price table through the browser entry point while
preserving the required pricing API.

In `@core/src/execute/evaluatorLoop.ts`:
- Around line 251-255: Extract and export a shared applyCostAndUsage helper that
accepts a result/summary target and optional tracker, returns without changes
when no tracker or total usage exists, and otherwise assigns totals, per-model
breakdown, and estimated cost. Replace the inline blocks at
core/src/execute/evaluatorLoop.ts lines 230-234 and 251-255, and
core/src/execute/runAll.ts lines 145-150, with calls to this helper using
partialResult, evResult, and report.summary respectively.

In `@core/src/execute/runAllBrowser.ts`:
- Around line 192-195: In the runAllBrowser function, extract the repeated
tokenUsageByModel and cost assignment into a shared helper such as
attachUsageAndCost(target, tracker). Replace the duplicated logic for the
partial-failure result, completed evaluator result, and overall report summary
with calls to this helper, while preserving each result’s existing tokenUsage
assignment and cost calculation behavior.

In `@core/src/pricing/lookupPrice.ts`:
- Around line 51-58: Update providerAccepts to distinguish unknown providers
from explicitly unverifiable providers: derive the recognized provider set from
providerRegistryData/alias validation, or add an explicit unknown entry in
LITELLM_PROVIDER_ALIASES, and reject unlisted providers so unattributed usage
cannot match priced rows while preserving openai-compatible behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5ccfe88f-beee-4796-b2ce-203d96a81c45

📥 Commits

Reviewing files that changed from the base of the PR and between 04d0690 and b5ce1e5.

⛔ Files ignored due to path filters (1)

• core/src/pricing/priceTable.generated.ts is excluded by !**/*.generated.*

📒 Files selected for processing (28)

• core/package.json
• core/src/browser.ts
• core/src/evaluators/judge.ts
• core/src/execute/evaluatorLoop.ts
• core/src/execute/runAll.ts
• core/src/execute/runAllBrowser.ts
• core/src/execute/tokenTracker.ts
• core/src/execute/types.ts
• core/src/generate/generateAttacks.ts
• core/src/generate/generateNextTurn.ts
• core/src/lib/llmRetry.ts
• core/src/llm/openaiCompatible.ts
• core/src/pricing/estimateCost.ts
• core/src/pricing/lookupPrice.ts
• core/src/pricing/providerAliases.ts
• core/src/pricing/types.ts
• core/src/providers/factory.ts
• core/src/providers/modelIdentity.ts
• core/src/report/buildReport.ts
• core/src/report/render.ts
• core/src/report/types.ts
• core/src/run/judge.ts
• core/tests/modelIdentity.test.ts
• core/tests/pricing.test.ts
• core/tests/tokenTracker.test.ts
• package.json
• runners/cli/src/commands/run.ts
• scripts/build-pricing.ts

[coderabbitai]

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/src/pricing/types.ts`:
- Around line 56-65: Update the RunCost completeness contract and its
calculation so complete is true only when every expected call reported usage and
every reported model was priced; otherwise mark the result as incomplete. Ensure
formatCostDisplay in core/src/report/render.ts renders the total with ≥,
including for runs with omitted usage or zero recorded tokens, and add
regression coverage for both cases.

In `@docs/browser-extension.md`:
- Line 91: Update the “Token usage and testing cost” documentation to remove the
claim that the target has nothing to bill. State instead that target-side
inference costs are excluded from Opfor’s measurements and are not observable or
metered by Opfor, while preserving the explanation of attacker and judge LLM
costs.
- Line 91: Update the cost description near “Token usage and testing cost” to
accurately state that the configured LLM is used for both attacker and judge
roles, replacing the claim that separate attacker and judge LLMs are configured.

In `@README.md`:
- Line 116: Update README.md lines 116-116 to avoid claiming every run reports
complete cost; update docs/cli.md lines 220-220 to scope cost reporting to
instrumented LLM calls; update README.md lines 130-130 to mention unrecorded
calls as another lower-bound condition; and update docs/cli.md lines 249-249 to
state that incomplete usage capture can make summary.cost.totalUsd a lower
bound.
- Line 118: Update the opening output fences to specify the text language:
change the fence at README.md lines 118-118 and docs/cli.md lines 222-222 to use
the text identifier, preserving the existing output content.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ab162b8c-832e-4738-9ec0-69758a051cfb

📥 Commits

Reviewing files that changed from the base of the PR and between bf46049 and f2bc2cb.

⛔ Files ignored due to path filters (1)

• core/src/pricing/priceTable.generated.ts is excluded by !**/*.generated.*

📒 Files selected for processing (32)

• README.md
• core/package.json
• core/src/browser.ts
• core/src/evaluators/judge.ts
• core/src/execute/baselineScanner.ts
• core/src/execute/evaluatorLoop.ts
• core/src/execute/runAll.ts
• core/src/execute/runAllBrowser.ts
• core/src/execute/tokenTracker.ts
• core/src/execute/types.ts
• core/src/generate/generateAttacks.ts
• core/src/generate/generateNextTurn.ts
• core/src/lib/llmRetry.ts
• core/src/llm/openaiCompatible.ts
• core/src/pricing/estimateCost.ts
• core/src/pricing/lookupPrice.ts
• core/src/pricing/providerAliases.ts
• core/src/pricing/types.ts
• core/src/providers/factory.ts
• core/src/providers/modelIdentity.ts
• core/src/report/buildReport.ts
• core/src/report/render.ts
• core/src/report/types.ts
• core/src/run/judge.ts
• core/tests/modelIdentity.test.ts
• core/tests/pricing.test.ts
• core/tests/tokenTracker.test.ts
• docs/browser-extension.md
• docs/cli.md
• package.json
• runners/cli/src/commands/run.ts
• scripts/build-pricing.ts

🚧 Files skipped from review as they are similar to previous changes (28)

• core/package.json
• package.json
• core/src/run/judge.ts
• core/src/generate/generateNextTurn.ts
• core/src/execute/baselineScanner.ts
• core/src/evaluators/judge.ts
• core/src/execute/runAll.ts
• core/src/providers/factory.ts
• core/src/browser.ts
• core/src/execute/runAllBrowser.ts
• core/tests/tokenTracker.test.ts
• core/src/providers/modelIdentity.ts
• scripts/build-pricing.ts
• core/src/pricing/estimateCost.ts
• core/tests/modelIdentity.test.ts
• core/src/llm/openaiCompatible.ts
• core/src/generate/generateAttacks.ts
• core/src/report/buildReport.ts
• core/src/pricing/providerAliases.ts
• core/src/lib/llmRetry.ts
• runners/cli/src/commands/run.ts
• core/src/pricing/lookupPrice.ts
• core/src/execute/tokenTracker.ts
• core/src/execute/types.ts
• core/tests/pricing.test.ts
• core/src/execute/evaluatorLoop.ts
• core/src/report/render.ts
• core/src/report/types.ts