KeyValueSoftwareSystems · arunSunnyKVS · Aug 6, 2026 · Aug 6, 2026 · Aug 6, 2026 · Aug 6, 2026
diff --git a/AGENTS.md b/AGENTS.md
@@ -277,10 +277,14 @@ There is no longer a separate `generate` step. `opfor run --config <file>` does
 
 **MCP targets** additionally run baseline pre-flight scans (`runBaselineScans`) before the evaluator loop — these enumerate `tools/list` + `resources/list` and judge them for poisoning / leakage independent of any evaluator. Throughout the run, `runAll` fans lifecycle events to registered `RunListener`s (progress reporting, NDJSON streaming) rather than only a callback.
 
-**Cancellation.** `RunAllOptions` accepts an optional `signal?: AbortSignal`. When aborted, the evaluator loop finishes the in-flight attack, skips remaining evaluators/attacks, and returns a partial report with `stopReason: "user-interrupted"`. The CLI wires this to SIGINT (first Ctrl+C = graceful stop, second = force kill). The SDK can reuse the same mechanism for programmatic cancellation.
+**Cancellation.** `RunAllOptions` accepts an optional `signal?: AbortSignal`. It is threaded down to `runAttack` and checked **before each turn**, not just between attacks — otherwise a single high-`turns` attack would run to completion before the stop took effect. When aborted, the in-flight turn finishes, the attack is finalized (judging whatever transcript exists), remaining evaluators/attacks are skipped, and a partial report is returned with `stopReason: "user-interrupted"`. The CLI wires this to SIGINT (first Ctrl+C = graceful stop, second = force kill). The SDK can reuse the same mechanism for programmatic cancellation.
+
+The browser extension reaches the same granularity by a different route: `DomTarget.send()` checks `state.OPFOR_STOP` around each send/extract and throws an error tagged `code: "OPFOR_STOP"`, which `runAllBrowser` recognizes as a clean stop (structurally, since core cannot import the extension's class) and turns into a partial report rather than an unhandled throw.
 
 **Token usage tracking.** `runAll` creates a `TokenTracker` (see `core/src/execute/tokenTracker.ts`) and threads it through the evaluator loop → attack drivers → `withRetry` / `generateText` / `chatCompletionJsonContent` call sites. Every LLM call auto-records its `usage` (input/output tokens). Per-evaluator child trackers aggregate into evaluator-level totals (`EvaluatorResult.tokenUsage`); the parent accumulates run-level totals (`UnifiedRunReport.summary.tokenUsage`). The CLI prints a summary line, the HTML report shows a stat card, and the JSON report includes the data for CI. `runAllBrowser` follows the same pattern so the extension popup can show a token count.
 
+**Cost is cache-aware.** `inputTokens` is the **inclusive** total — providers report it as `noCache + cacheRead + cacheWrite`. `estimateRunCost` (`core/src/pricing/estimateCost.ts`) therefore _divides_ it across those tiers and prices each at its own published rate; adding a cache charge on top of the full input charge would bill cached tokens twice. The tracker reads the split from the AI SDK's provider-agnostic `usage.inputTokenDetails`, so Anthropic's `cache_read_input_tokens` and OpenAI/DeepSeek's `prompt_tokens_details.cached_tokens` both work without provider-specific code. Two invariants worth preserving when touching this: the fresh count is always _derived_ from `inputTokens` (a split that doesn't add up is dropped rather than trusted), and a tier with no published rate falls back to the full input rate — never to free.
+
 `runAllBrowser` is the same loop in browser-safe form: takes preloaded `EvaluatorSpec[]` + a pre-built `AgentTarget` (e.g. `DomTarget`), skips disk reads.
 
 ---

diff --git a/README.md b/README.md
@@ -124,9 +124,9 @@ Testing cost: $0.18
 
 This is **opfor's own spend** — the attacker and judge LLMs. It excludes your target's inference cost, which opfor cannot see from the outside. The per-model split is the useful part: the judge is often the bigger share, and pointing it at a cheaper model is usually the easiest saving.
 
-Prices come from a snapshot of LiteLLM's public price map that ships with the package, so runs work offline and a report re-rendered later produces the same figure. Two caveats worth knowing:
+Prices come from a snapshot of LiteLLM's public price map that ships with the package, so runs work offline and a report re-rendered later produces the same figure. Cached input is billed at the provider's cache rate — multi-turn attacks re-send the conversation each turn, and that repeated prefix is often ~100× cheaper than fresh text, so the figure tracks the real bill rather than a worst case. Caveats worth knowing:
 
-- **Multi-turn runs read high.** Providers discount repeated context, and a multi-turn attack re-sends the conversation each turn — opfor prices every input token at full rate, so the real bill is usually lower.
+- **Caching is only credited when it's reported.** A provider that doesn't break out cached tokens, or a model with no published cache rate, is charged at the full input rate — an over-estimate, chosen over quietly under-reporting.
-Prices come from a snapshot of LiteLLM's public price map that ships with the package, so runs work offline and a report re-rendered later produces the same figure. Cached input is billed at the provider's cache rate — multi-turn attacks re-send the conversation each turn, and that repeated prefix is often ~100× cheaper than fresh text, so the figure tracks the real bill rather than a worst case. Caveats worth knowing:
-
- **Multi-turn runs read high.** Providers discount repeated context, and a multi-turn attack re-sends the conversation each turn — opfor prices every input token at full rate, so the real bill is usually lower.
- **Caching is only credited when it's reported.** A provider that doesn't break out cached tokens, or a model with no published cache rate, is charged at the full input rate — an over-estimate, chosen over quietly under-reporting.
+Prices come from a snapshot of LiteLLM's public price map that ships with the package, so runs work offline and a report re-rendered later produces the same figure. Cache-read and cache-write input are billed at their separate published rates. In multi-turn attacks, a repeated prefix can be much cheaper when it is reported as a cache read. Cache writes can cost more than fresh input. Caveats worth knowing:
+
+- **Caching is only credited when it's reported.** A provider that doesn't break out cached tokens, or a model with no published cache rate, is charged at the full input rate — an over-estimate, chosen over quietly under-reporting.
-Prices come from a snapshot of LiteLLM's public price map that ships with the package, so runs work offline and a report re-rendered later produces the same figure. Cached input is billed at the provider's cache rate — multi-turn attacks re-send the conversation each turn, and that repeated prefix is often ~100× cheaper than fresh text, so the figure tracks the real bill rather than a worst case. Caveats worth knowing:
-
- **Multi-turn runs read high.** Providers discount repeated context, and a multi-turn attack re-sends the conversation each turn — opfor prices every input token at full rate, so the real bill is usually lower.
- **Caching is only credited when it's reported.** A provider that doesn't break out cached tokens, or a model with no published cache rate, is charged at the full input rate — an over-estimate, chosen over quietly under-reporting.
+Prices come from a snapshot of LiteLLM's public price map that ships with the package, so runs work offline and a report re-rendered later produces the same figure. Cache-read and cache-write input are billed at their separate published rates. In multi-turn attacks, a repeated prefix can be much cheaper when it is reported as a cache read. Cache writes can cost more than fresh input. Caveats worth knowing:
+
+- **Caching is only credited when it's reported.** A provider that doesn't break out cached tokens, or a model with no published cache rate, is charged at the full input rate — an over-estimate, chosen over quietly under-reporting.
 - **Unknown models are never counted as free.** A model missing from the price table is reported as unpriced and the total is marked a lower bound, rather than silently reading as $0.
 - **A few helper calls aren't metered yet.** Trace curation, session summarisation and one JSON helper don't report token usage, so their spend is missing from the total. Treat the figure as a floor.
 

diff --git a/core/src/execute/attackRunner.ts b/core/src/execute/attackRunner.ts
@@ -31,11 +31,19 @@ export interface AttackDriver<TInput, TOutput> {
 /**
  * Template Method for running one attack: the invariant skeleton every attack
  * kind shares. The `driver` supplies the kind-specific behavior.
+ *
+ * `signal`, when given, is checked before each turn so a long multi-turn attack
+ * (e.g. `turns: 100`) can be interrupted between turns rather than only between
+ * whole attacks/evaluators — `runAll`'s cancellation contract promises "finishes
+ * in-flight work", and a single attack's turn loop is exactly the granularity
+ * that needs to honor it.
  */
 export async function runAttack<TInput, TOutput>(
-  driver: AttackDriver<TInput, TOutput>
+  driver: AttackDriver<TInput, TOutput>,
+  signal?: AbortSignal
 ): Promise<AttackResult> {
   for (let turnNo = driver.startTurn; turnNo <= driver.totalTurns; turnNo++) {
+    if (signal?.aborted) break;
     const input = await driver.buildTurn(turnNo);
     const output = await driver.execute(input);
     driver.record(turnNo, input, output);

diff --git a/core/src/execute/evaluatorLoop.ts b/core/src/execute/evaluatorLoop.ts
@@ -179,7 +179,14 @@ export async function runEvaluatorAttacks(
       try {
         result =
           attack.kind === "mcp"
-            ? await runMcpAttack(attack, mcpTarget!, attackModel, judgeLlmConfig, evalTracker)
+            ? await runMcpAttack(
+                attack,
+                mcpTarget!,
+                attackModel,
+                judgeLlmConfig,
+                evalTracker,
+                signal
+              )
             : await runAgentAttack(
                 attack,
                 attackModel,
@@ -191,7 +198,8 @@ export async function runEvaluatorAttacks(
                   targetConfig: config.target,
                   telemetry: config.telemetry,
                   tokenTracker: evalTracker,
-                }
+                },
+                signal
               );
       } catch (err) {
         const makeFailedResult = (reason: string): AttackResult =>

diff --git a/core/src/execute/mcpAttackDriver.ts b/core/src/execute/mcpAttackDriver.ts
@@ -169,7 +169,8 @@ export async function runMcpAttack(
   target: McpTarget,
   attackModel: LanguageModel,
   judgeLlm: LlmConfig,
-  tokenTracker?: TokenTracker
+  tokenTracker?: TokenTracker,
+  signal?: AbortSignal
 ): Promise<AttackResult> {
   if (!attack.toolName) {
     return {
@@ -185,6 +186,7 @@ export async function runMcpAttack(
     };
   }
   return runAttack(
-    new McpAttackDriver(attack, target, attack.toolName, attackModel, judgeLlm, tokenTracker)
+    new McpAttackDriver(attack, target, attack.toolName, attackModel, judgeLlm, tokenTracker),
+    signal
   );
 }
diff --git a/core/src/execute/runAgentLoop.ts b/core/src/execute/runAgentLoop.ts
@@ -19,9 +19,11 @@ export async function runAgentAttack(
   attackIndex: string,
   patterns: AttackPattern[],
   target: AgentTarget,
-  context?: AgentAttackContext
+  context?: AgentAttackContext,
+  signal?: AbortSignal
 ): Promise<AttackResult> {
   return runAttack(
-    new AgentAttackDriver(attack, attackModel, judgeModel, attackIndex, patterns, target, context)
+    new AgentAttackDriver(attack, attackModel, judgeModel, attackIndex, patterns, target, context),
+    signal
   );
 }
diff --git a/core/src/execute/runAllBrowser.ts b/core/src/execute/runAllBrowser.ts
@@ -209,6 +209,17 @@ export async function runAllBrowser(
           pushPartialResult(stopReason);
           break evaluatorLoop;
         }
+        // The extension's DomTarget throws a plain Error tagged `code: "OPFOR_STOP"`
+        // on user cancel/pause (see runners/extension/domTarget.js) — core can't
+        // import that class, so recognize it structurally instead. Without this,
+        // a user-cancelled run threw past every handler below and returned no
+        // report at all, silently dropping the TokenTracker totals collected so far.
+        if ((err as { code?: string })?.code === "OPFOR_STOP") {
+          stopReason = err instanceof Error ? err.message : "Run stopped by user.";
+          notify({ type: "run_stopped", reason: stopReason });
+          pushPartialResult(stopReason);
+          break evaluatorLoop;
+        }
         throw err;
       }
 

diff --git a/core/src/execute/tokenTracker.ts b/core/src/execute/tokenTracker.ts
@@ -27,6 +27,26 @@ export interface TokenUsage {
   totalTokens: number;
 }
 
+/**
+ * How one call's input tokens split across cache tiers.
+ *
+ * `inputTokens` is the **inclusive** total — providers report it as
+ * `noCache + cacheRead + cacheWrite`, not as the fresh-text count alone. Pricing
+ * therefore has to divide that total between the tiers, never add a cache charge
+ * on top of it (which would bill cached tokens twice).
+ *
+ * Carried only so {@link estimateRunCost} can apply each tier's rate; it is not
+ * reported on its own.
+ */
+export interface InputCacheSplit {
+  /** Input tokens processed fresh, billed at the full input rate. */
+  noCache: number;
+  /** Input tokens served from cache, billed at the provider's (much lower) read rate. */
+  cacheRead: number;
+  /** Input tokens written to cache, billed at the provider's write rate. */
+  cacheWrite: number;
+}
+
 /** Token usage attributed to one provider/model pair. */
 export interface ModelTokenUsage extends TokenUsage {
   /** `"<provider>:<model>"`, or `"unknown"` for usage that could not be attributed. */
@@ -37,6 +57,14 @@ export interface ModelTokenUsage extends TokenUsage {
   roles: string[];
   /** Number of LLM calls recorded against this model. */
   calls: number;
+  /**
+   * Cache split of {@link TokenUsage.inputTokens}, present only when this model
+   * actually hit a cache. Exists so each tier can be priced at its own rate; the
+   * three fields sum to `inputTokens`. See {@link InputCacheSplit}.
+   */
+  noCacheInputTokens?: number;
+  cacheReadInputTokens?: number;
+  cacheWriteInputTokens?: number;
 }
 
 /** Optional provenance supplied alongside a usage recording. */
@@ -65,20 +93,50 @@ export const LlmUsageSchema = z
     inputTokens: z.number().int().min(0).optional().default(0),
     outputTokens: z.number().int().min(0).optional().default(0),
     totalTokens: z.number().int().min(0).optional().default(0),
+    // Provider-agnostic cache split, supplied by the AI SDK as part of `usage`.
+    // Absent on providers (or call paths) that don't report it — see the
+    // transform below, which then treats every input token as uncached.
+    inputTokenDetails: z
+      .object({
+        noCacheTokens: z.number().int().min(0).optional(),
+        cacheReadTokens: z.number().int().min(0).optional(),
+        cacheWriteTokens: z.number().int().min(0).optional(),
+      })
+      .passthrough()
+      .optional(),
   })
   .passthrough()
-  .transform((u) => ({
-    inputTokens: u.inputTokens,
-    outputTokens: u.outputTokens,
-    totalTokens: u.totalTokens > 0 ? u.totalTokens : u.inputTokens + u.outputTokens,
-  }));
+  .transform((u) => {
+    const cacheRead = u.inputTokenDetails?.cacheReadTokens ?? 0;
+    const cacheWrite = u.inputTokenDetails?.cacheWriteTokens ?? 0;
+    const cached = cacheRead + cacheWrite;
+    // `inputTokens` already includes the cached tokens, so the fresh count is
+    // always the remainder. Derived rather than read from `noCacheTokens`: cost
+    // divides inputTokens between the tiers, so a reported split that doesn't add
+    // up would bill tokens the call never used. For a well-formed provider the
+    // two agree — the AI SDK builds inputTokens as noCache + cacheRead +
+    // cacheWrite — so deriving costs nothing and makes the invariant hold.
+    //
+    // A split claiming more cached tokens than there was input can't be divided
+    // at all; drop it and let the call price at the full input rate, the same
+    // conservative direction taken for an unpriced model.
+    const usable = cached > 0 && cached <= u.inputTokens;
+    return {
+      inputTokens: u.inputTokens,
+      outputTokens: u.outputTokens,
+      totalTokens: u.totalTokens > 0 ? u.totalTokens : u.inputTokens + u.outputTokens,
+      // Omitted when nothing was cached, so a non-caching run records and prices
+      // exactly as it did before this field existed.
+      ...(usable ? { cache: { noCache: u.inputTokens - cached, cacheRead, cacheWrite } } : {}),
+    };
+  });
 
 /**
  * Validate and normalize a raw usage object (from any provider/SDK) into a
  * clean {@link TokenUsage}. Returns `undefined` when the input is falsy or
  * fails validation so callers can safely discard garbage.
  */
-export function parseUsage(raw: unknown): TokenUsage | undefined {
+export function parseUsage(raw: unknown): (TokenUsage & { cache?: InputCacheSplit }) | undefined {
   if (!raw || typeof raw !== "object") return undefined;
   const result = LlmUsageSchema.safeParse(raw);
   return result.success ? result.data : undefined;
@@ -94,6 +152,11 @@ interface ModelBucket {
   input: number;
   output: number;
   total: number;
+  // Cache tiers of `input`. Calls that report no split count entirely as
+  // noCache, so these three always sum to `input`.
+  noCache: number;
+  cacheRead: number;
+  cacheWrite: number;
 }
 
 /**
@@ -119,7 +182,12 @@ export class TokenTracker {
    * call site degrades to today's behavior rather than losing tokens.
    */
   record(
-    usage?: { inputTokens?: number; outputTokens?: number; totalTokens?: number },
+    usage?: {
+      inputTokens?: number;
+      outputTokens?: number;
+      totalTokens?: number;
+      cache?: InputCacheSplit;
+    },
     attribution?: RecordAttribution
   ): void {
     if (!usage) return;
@@ -132,15 +200,16 @@ export class TokenTracker {
     this.output += out;
     this.total += resolvedTotal;
 
-    this.recordToBucket(inp, out, resolvedTotal, attribution);
+    this.recordToBucket(inp, out, resolvedTotal, attribution, usage.cache);
   }
 
   /** Fold one call's usage into its per-model bucket, creating the bucket on first sight. */
   private recordToBucket(
     inp: number,
     out: number,
     total: number,
-    attribution?: RecordAttribution
+    attribution?: RecordAttribution,
+    cache?: InputCacheSplit
   ): void {
     const identity = resolveModelIdentity(attribution?.model);
     const key = identity ? modelKey(identity) : UNKNOWN_MODEL_KEY;
@@ -156,6 +225,9 @@ export class TokenTracker {
         input: 0,
         output: 0,
         total: 0,
+        noCache: 0,
+        cacheRead: 0,
+        cacheWrite: 0,
       };
       this.buckets.set(key, bucket);
     }
@@ -166,6 +238,11 @@ export class TokenTracker {
     bucket.input += inp;
     bucket.output += out;
     bucket.total += total;
+    // No split reported → the whole call was uncached, keeping the three tiers
+    // summing to `input` even when only some calls to this model were cached.
+    bucket.noCache += cache?.noCache ?? inp;
+    bucket.cacheRead += cache?.cacheRead ?? 0;
+    bucket.cacheWrite += cache?.cacheWrite ?? 0;
   }
 
   /** Current accumulated totals. Uses the provider-supplied total when available. */
@@ -193,6 +270,15 @@ export class TokenTracker {
         inputTokens: b.input,
         outputTokens: b.output,
         totalTokens: b.total,
+        // Omitted when this model never hit a cache, so the shape is unchanged
+        // for runs where caching never applied.
+        ...(b.cacheRead > 0 || b.cacheWrite > 0
+          ? {
+              noCacheInputTokens: b.noCache,
+              cacheReadInputTokens: b.cacheRead,
+              cacheWriteInputTokens: b.cacheWrite,
+            }
+          : {}),
       }))
       .sort((a, b) => b.totalTokens - a.totalTokens || a.key.localeCompare(b.key));
   }
@@ -222,6 +308,7 @@ class ChildTracker extends TokenTracker {
       inputTokens?: number;
       outputTokens?: number;
       totalTokens?: number;
+      cache?: InputCacheSplit;
     },
     attribution?: RecordAttribution
   ): void {

diff --git a/core/src/pricing/estimateCost.ts b/core/src/pricing/estimateCost.ts
@@ -6,9 +6,11 @@
  * whenever `complete` is false, and the UI must say so — a report that quietly
  * treats an unknown model as $0 understates spend while looking authoritative.
  *
- * Repeat ("cached") input tokens are not yet separated by the token counter, so
- * all input is priced at the full input rate. For multi-turn runs against
- * providers that discount repeated context, that makes this an over-estimate.
+ * Input is priced per cache tier. `inputTokens` is the inclusive total, so the
+ * tiers are *divided out of* it rather than charged on top — adding a cache
+ * charge to the full input charge would bill every cached token twice. A tier
+ * whose rate the price table doesn't publish falls back to the full input rate,
+ * keeping the same never-quietly-free stance as an unpriced model.
  */
 
 import type { ModelTokenUsage } from "../execute/tokenTracker.js";
@@ -33,11 +35,25 @@ function costOne(usage: ModelTokenUsage): ModelCost {
 
   if (!found) return base;
 
+  const { inputPerToken, outputPerToken, cacheReadPerToken, cacheWritePerToken } = found.price;
+
+  // Split the (inclusive) input total across cache tiers. A model that reported
+  // no split priced every input token at the full rate before this existed, and
+  // still does: cacheRead/cacheWrite fall to 0 and noCache absorbs the total.
+  const cacheRead = usage.cacheReadInputTokens ?? 0;
+  const cacheWrite = usage.cacheWriteInputTokens ?? 0;
+  const noCache =
+    usage.noCacheInputTokens ?? Math.max(0, usage.inputTokens - cacheRead - cacheWrite);
+
+  // `??` not `||`: a published rate of 0 is real (some providers don't charge
+  // for cache writes) and must not be mistaken for a missing rate.
   return {
     ...base,
     usd:
-      usage.inputTokens * found.price.inputPerToken +
-      usage.outputTokens * found.price.outputPerToken,
+      noCache * inputPerToken +
+      cacheRead * (cacheReadPerToken ?? inputPerToken) +
+      cacheWrite * (cacheWritePerToken ?? inputPerToken) +
+      usage.outputTokens * outputPerToken,
     source: "table",
     matchedKey: found.matchedKey,
   };

diff --git a/core/src/pricing/types.ts b/core/src/pricing/types.ts
@@ -14,10 +14,11 @@ export interface ModelPrice {
   outputPerToken: number;
   /**
    * USD per cached (repeat) input token, when the provider publishes one.
-   * Not applied yet — token counting does not separate repeat tokens today.
+   * Falls back to {@link inputPerToken} when absent, so an unpublished cache
+   * rate over-estimates rather than under-charges.
    */
   cacheReadPerToken?: number;
-  /** USD per cache-write token, when the provider publishes one. Not applied yet. */
+  /** USD per cache-write token, when the provider publishes one. Same fallback. */
   cacheWritePerToken?: number;
 }