Skip to content

docs: correct cancellation and stop behaviour across the docs - #238

Merged
jithin23-kv merged 1 commit into
masterfrom
docs/cancellation-and-stop-behaviour
Aug 6, 2026
Merged

docs: correct cancellation and stop behaviour across the docs#238
jithin23-kv merged 1 commit into
masterfrom
docs/cancellation-and-stop-behaviour

Conversation

@arunSunnyKVS

@arunSunnyKVS arunSunnyKVS commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Problem

Follow-up to #236. That PR changed cancellation from attack-granular to turn-granular and changed what a stopped extension run leaves behind, but the user-facing docs still described the old behaviour. A sweep of every first-party doc against the merged code turned up three gaps:

  1. docs/cli.md promised the wrong thing. The "Graceful shutdown (Ctrl+C)" table said the first Ctrl+C "finishes the in-flight attack" — which described the bug fix: cancellation turn-granularity, extension report gaps, and cache-aware cost pricing #236 fixed as if it were the contract. A user reading it would still expect a turns: 100 attack to run to completion before stopping.
  2. Stop and Pause were undocumented. docs/browser-extension.md walks through the run flow but never mentioned either control, despite both being prominent buttons and fix: cancellation turn-granularity, extension report gaps, and cache-aware cost pricing #236 changing what a stopped run produces (it now yields a report with token usage and cost rather than nothing). The neighbouring limitation — "No pause / resume across sessions" — reads as though pause doesn't exist at all; it actually meant a paused run doesn't survive uninstall.
  3. AGENTS.md overstated SDK support. It said the SDK "can reuse the same mechanism" for programmatic cancellation, which reads as though it already does. runners/sdk accepts no signal option.

Solution

  • docs/cli.md — corrected the shutdown table to say turn, and added what that buys you: stopping costs one turn's latency (a model call plus a target call) however high turns is set. Also noted that the interrupted attack is still judged on the transcript it collected, so partial multi-turn work isn't discarded.
  • docs/browser-extension.md — documented Stop vs Pause and what each leaves behind, and rewrote the limitation to say what it means.
  • AGENTS.md — states plainly that signal isn't surfaced by the SDK, so plumbing it through is the work required to offer programmatic cancellation.

Verified against the code, not recollection

The first draft of the extension section claimed stopping "never leaves the target mid-exchange". That's false: runners/extension/domTarget.js tests the stop flag after the send and before the extract, so a stop can abandon a message already sent. The text now says so, and notes the turn is dropped rather than recorded half-finished.

The Pause/Stop table was checked against finalizeUserInterruption in runners/extension/orchestrator.js — pause persists a snapshot and writes no result (so the in-progress evaluator has no verdict), cancel persists a partial result carrying cancelledJudgment's CANCELLED verdict.

Also checked, deliberately unchanged

  • skills/*/report-schema.md — documents the skill's own output shape (totalEvaluators, criticalFindings), not UnifiedRunReport; never covered cost.
  • docs/sdk.md's totalCostUsd — belongs to the autonomous hunt pipeline, fed by the Agent SDK's own reported cost, untouched by fix: cancellation turn-granularity, extension report gaps, and cache-aware cost pricing #236's pricing change.
  • CHANGELOG.md — release-please generated from conventional commits; hand-editing would be wrong.
  • runners/{cli,sdk,mcp}/README.md, CONTRIBUTING.md — no mentions of cost, tokens, or cancellation.

Changes

  • docs/cli.md — Ctrl+C behaviour
  • docs/browser-extension.md — Stop/Pause section; limitation reworded
  • AGENTS.md — SDK cancellation status

Docs only; no source or test changes.

Issue

N/A

How to test

Read the three diffs against the behaviour merged in #236 — specifically runAttack's per-turn signal check in core/src/execute/attackRunner.ts, and finalizeUserInterruption / domTarget.js's stop checkpoints in runners/extension/.

npm test (304 pass) and npm run typecheck are unaffected but were run to confirm nothing regressed.

Screenshots

N/A

Summary by CodeRabbit

  • Documentation
    • Clarified browser extension behavior when pausing or stopping runs: the current turn finishes, incomplete turns are discarded, and pause preserves a resumable snapshot.
    • Documented that stopping produces a final report, marks the active evaluation as cancelled, and retains token and cost reporting.
    • Clarified that paused runs are lost if the extension is uninstalled.
    • Updated CLI interruption guidance: the first Ctrl+C completes the current turn, writes a partial report, and evaluates the interrupted attack transcript.
    • Documented current SDK cancellation limitations.

A sweep of every first-party doc against the behaviour changed in this PR
turned up three stale or missing pieces:

- docs/cli.md promised Ctrl+C "finishes the in-flight attack". It now
  finishes the in-flight turn, which is the point of the change — the old
  wording described the bug as if it were the contract.
- docs/browser-extension.md never documented Stop or Pause at all, though
  both are prominent controls and this PR changed what a stopped run
  leaves behind. Added what each produces, and corrected the neighbouring
  limitation that read as though pause did not exist.
- AGENTS.md said the SDK "can reuse the same mechanism" for cancellation,
  which reads as though it already does; runners/sdk accepts no signal.

Verified against the code rather than recollection: the extension checks
for a stop between sending a message and reading the reply, so it can
abandon a sent turn — the docs now say so instead of claiming turns
always complete.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0bf936ef-87dc-429d-8814-a78422919741

📥 Commits

Reviewing files that changed from the base of the PR and between 317eb27 and 0096871.

📒 Files selected for processing (3)
  • AGENTS.md
  • docs/browser-extension.md
  • docs/cli.md

Walkthrough

The documentation now describes SDK cancellation limits, browser extension pause and stop behavior, and CLI interruption handling between turns.

Changes

Cancellation documentation

Layer / File(s) Summary
Cancellation and interruption behavior
AGENTS.md, docs/browser-extension.md, docs/cli.md
The documentation states that the SDK does not expose RunAllOptions.signal, browser pauses preserve resumable snapshots while stops produce final reports, and the CLI handles Ctrl+C between turns with partial reporting.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Suggested reviewers: jithin23-kv

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the documentation updates for cancellation and stop behavior.
Description check ✅ Passed The description covers the problem, solution, affected files, issue status, testing steps, and screenshots status.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/cancellation-and-stop-behaviour

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jithin23-kv
jithin23-kv merged commit 03d375f into master Aug 6, 2026
8 checks passed
@jithin23-kv
jithin23-kv deleted the docs/cancellation-and-stop-behaviour branch August 6, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants