Skip to content

feat: three-mode safety policy with decoupled classifier#3806

Draft
trungutt wants to merge 9 commits into
docker:mainfrom
trungutt:feat/safety-policy-three-modes
Draft

feat: three-mode safety policy with decoupled classifier#3806
trungutt wants to merge 9 commits into
docker:mainfrom
trungutt:feat/safety-policy-three-modes

Conversation

@trungutt

Copy link
Copy Markdown
Contributor

Why

Shell command approval today is a coarse two-mode toggle:

  • safer on a shell toolset: the classifier asks about anything
    it doesn't positively recognise as safe.
  • safer off (or --yolo): everything auto-approves.

That's it. There's no way to say "I trust you on read-only calls,
but ask me before you destroy something". A user who finds strict
mode annoying has only one escape hatch — full auto-approve — and
loses the safety net for genuinely destructive commands.

The classifier itself is also entangled with the approval verdict.
It reads the session's safety policy, decides whether to ask, and
emits its verdict directly. That makes it impossible to reuse the
classifier's judgement anywhere else (UI badges, telemetry, other
gating logic) without duplicating its rules, and adding a new
approval mode means editing the classifier.

What this PR does

Introduce three explicit safety modes and split the classifier
into a pure labeller so the mode is what actually gates the call.

                     ┌─────────┬─────────────┬─────────┐
                     │  safe   │ destructive │ unknown │
        ┌────────────┼─────────┼─────────────┼─────────┤
        │ Strict     │   ASK   │     ASK     │   ASK   │
        │ Balanced   │  ALLOW  │     ASK     │   ASK   │
        │ Autonomous │  ALLOW  │    ALLOW    │  ALLOW  │
        └────────────┴─────────┴─────────────┴─────────┘
  • Strict — today's default. Prompt on every tool call.
  • Balanced — new middle tier. The read-only path (ls, git status,
    docker ps, …) flows through silently; destructive and unrecognised
    calls still prompt.
  • Autonomous — legacy yolo. Auto-approve everything.

Custom Allow/Ask/Deny rules layer on top and always win over the
mode. A distinction the runtime enforces: a rule the user added at
runtime (session-scoped) beats even Autonomous, while a rule the
agent author baked into the YAML is advisory and yields to
Autonomous.

The classifier that used to make the verdict now only labels the
call. Downstream (the runtime, hosting applications, telemetry) all
see the same three values — safe / destructive / unknown — and
each layer decides what to do with them independently.

                      ┌────────────────────┐        ┌────────────────────┐
   shell call    ───▶ │  safer_shell hook  │ ─────▶ │  mode × label      │ ─▶ allow / ask
                      │  (label only)      │ label  │  decision table    │
                      └────────────────────┘        └────────────────────┘
                        safe / destructive           reads session mode
                        / unknown

Notes for the reviewer

  • The safer: true opt-in on shell toolsets is removed from the
    schema. Any agent that declares a shell toolset gets the
    classifier automatically — it never blocks, only labels.
  • Resume verbs the CLI/API can send back grow two entries so the
    confirmation UI can offer "approve + switch to Balanced" and
    "approve + switch to Autonomous" alongside the existing single-call
    approve.
  • No behavioural change for existing --yolo users: yolo maps to
    Autonomous and every call still auto-approves.
  • Strict is unchanged from today's no-yolo default — every call
    prompts. The only new behaviour is Balanced.

Replace the two-mode safer/unsafe shell approval model with three
explicit safety modes and a pure-labeller classifier.

- Modes: Strict (ask always), Balanced (auto-approve classifier-safe
  calls, ask on destructive/unknown), Autonomous (auto-approve every
  call).
- Classifier is now a pure labeller emitting safe / destructive /
  unknown as metadata; the runtime consumes it via a mode x label
  decision table.
- The per-toolset 'safer: true' opt-in is gone. Any agent that
  declares a shell toolset gets the classifier automatically.
- Custom Allow/Ask/Deny rules override the mode; session-scoped Ask
  rules beat Autonomous, YAML-declared Ask rules are advisory and
  yield to it.
@trungutt
trungutt requested a review from docker-agent July 23, 2026 14:09
docker-agent

This comment was marked as resolved.

docker-agent

This comment was marked as resolved.

docker-agent

This comment was marked as resolved.

@aheritier aheritier added area/config For configuration parsing, YAML, environment variables area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/tui For features/issues/fixes related to the TUI kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Jul 23, 2026
docker-agent

This comment was marked as resolved.

trungutt added 5 commits July 23, 2026 16:52
SetSafetyPolicy writes under s.mu but concurrent readers in the
dispatcher (permissionDecision, hook input construction) read the
field bare, racing with 'approve + switch to Balanced/Autonomous'
resumes on sibling in-flight calls. Add a locked GetSafetyPolicy()
accessor mirroring IsToolsApproved() and route all runtime reads
through it.

The debug logs in the dispatcher and session_manager ResumeSession
path are dropped; they existed only for local development and were
the origin of the racy field reads.
SubSessionConfig only carried ToolsApproved. Balanced-mode parents
would spawn delegated agents, transferred tasks, and skill runs
with an empty SafetyPolicy, silently downgrading them to Strict —
Balanced sub-sessions would prompt on every read-only call while
their parent flowed through silently.

Carry SafetyPolicy alongside ToolsApproved and propagate it via
session.WithSafetyPolicy at every sub-session construction site.
…surface

Add Decision, key binding (B), OptionsHelp entry, and Resume
mapping for ApproveBalanced alongside the existing ApproveAutonomous
so 'approve + switch to Balanced' is reachable from the
confirmation dialog, matching the existing 'approve + switch to
Autonomous' shortcut.
Doc-comment still described the old model (opt-in via 'safer: true' on
the toolset; hook routes destructive shell commands to confirmation
regardless of --yolo). The classifier is now auto-registered whenever
an agent has a shell toolset and is a pure labeller — it never blocks,
only attaches blast_radius metadata that the runtime consumes.
Cut narrative prose that restates what the code does or duplicates
adjacent godoc. No behaviour change.
@trungutt
trungutt requested a review from docker-agent July 23, 2026 15:09

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

This PR cleanly implements the three-mode safety policy (Strict/Balanced/Autonomous) with a well-designed decoupled classifier. The core logic in pkg/runtime/toolexec/permissions.go is correct: the mode × label decision table, the session-vs-team ForceAsk precedence, and the legacy ToolsApproved sync are all sound.

One confirmed gap: the new Balanced mode is wired in the full TUI but is missing from the leantui keyboard handler — see the inline comment below.

Comment thread pkg/leantui/update.go

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

This PR is well-structured: the three-mode safety architecture (Strict / Balanced / Autonomous), the decoupled classifier labeller, and the custom rule precedence model (session ForceAsk beats mode; team ForceAsk is advisory and yields to Autonomous) are all implemented correctly. One functional gap was found in the CLI interaction path.

Comment thread pkg/cli/printer.go
trungutt added 3 commits July 23, 2026 17:31
The PR claimed 'Strict is unchanged from today's no-yolo default —
every call prompts.' In practice the old no-yolo default did auto-
approve tools annotated ReadOnlyHint; removing that short-circuit
broke headless callers (--exec, MCP serve, A2A) that relied on
read-only tools flowing through without a user to prompt.

Reintroduce the short-circuit right before the fallback ask so
Balanced/Autonomous keep going through the mode table and Strict
regains the read-only escape hatch.
The full TUI dialog exposes ApproveBalanced via 'B'; leantui only
offered [s] for Autonomous. Add [b] so leantui users can opt into
Balanced from a confirmation prompt.
Two-function chain where the second was the sole caller of the first.
Merge into a single LabelFromAnnotations that takes the ToolAnnotations
struct directly, matching the shape of the caller.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config For configuration parsing, YAML, environment variables area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/tui For features/issues/fixes related to the TUI kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants