Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
25f79ea
feat: group gateway CLI sessions by profile and agent, and render the…
chhhee10 Jul 28, 2026
8b6882f
fix: surface convention policies in the dashboard and fix the CLI col…
chhhee10 Jul 29, 2026
b73a993
fix: list a shared project/user policy directory once, not twice as f…
chhhee10 Jul 29, 2026
8655e47
feat: record convention policies in policies-config.json
chhhee10 Jul 29, 2026
101e0da
fix: record convention policies at user scope only, never in a projec…
chhhee10 Jul 29, 2026
c94cc47
fix: indent the folder icon with its label in the projects tree
chhhee10 Jul 29, 2026
93a37d7
fix: stop breaking Claude worktrees, and score the canary on ground t…
chhhee10 Jul 29, 2026
518985a
feat: surface "policy ran, but this CLI cannot enforce here" in the a…
chhhee10 Jul 29, 2026
52f2dad
fix: send deny in the shape five CLI events actually parse
chhhee10 Jul 29, 2026
0cd0842
feat: install Hermes pre_verify, its real turn-end gate
chhhee10 Jul 29, 2026
b035dc5
fix: never overwrite a policies-config.json that does not parse
chhhee10 Jul 29, 2026
94339d7
Revert installing Hermes pre_verify, keep what the audit established
chhhee10 Jul 29, 2026
aac85e8
fix: restore Pi's tool gate, unbreak worktrees here, fix dashboard cwd
chhhee10 Jul 29, 2026
dbc9e2e
docs: fill CHANGELOG PR references for #622
chhhee10 Jul 29, 2026
a06ccb9
fix: dedupe overlapping policy dirs, isolate discovery failures (Code…
chhhee10 Jul 29, 2026
9cba1ca
fix: stop a policy running twice, and a malformed settings key aborti…
NiveditJain Jul 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 0 additions & 12 deletions .claude/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -240,18 +240,6 @@
]
}
],
"WorktreeCreate": [
{
"hooks": [
{
"type": "command",
"command": "command -v node >/dev/null 2>&1 || { echo '[failproofai] node not found; install Node >=20.9' >&2; exit 2; }; node \"$CLAUDE_PROJECT_DIR/scripts/dev-hook.mjs\" --hook WorktreeCreate",
"timeout": 60,
"__failproofai_hook__": true
}
]
}
],
"WorktreeRemove": [
{
"hooks": [
Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

53 changes: 42 additions & 11 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -367,17 +367,48 @@ internal tool calls don't fire Hermes hooks — gate the *spawn* at `pre_tool_ca
| `post_tool_call` | `PostToolUse` | observation | Observe / sanitize. |
| `on_session_start` | `SessionStart` | observation | — |
| `on_session_end` | `SessionEnd` | observation | — |
| `subagent_stop` | `SubagentStop` | ✅ block | Subagent-return gate. |

**Limitations vs. Claude semantics.** Hermes has **no turn-end `Stop` event** — its
lifecycle is session-oriented and it can't force another turn into a session that's
ending — so `HERMES_EVENT_MAP` never emits `Stop` and the 5 `require-*-before-stop`
builtins never fire for it (inapplicable, not broken). It also lacks `UserPromptSubmit`
(only per-LLM-call `pre_llm_call`), `PreCompact`/`Notification`, etc. `instruct()`
degrades to **allow + logged note** — Hermes has no additional-context channel, so the
evaluator emits a non-blocking `{"decision":"allow", reason}` and surfaces the note on
stderr. In exchange Hermes has capabilities others lack (`transform_tool_result`,
`pre_gateway_dispatch`, `pre_llm_call`) — out of scope for now.
| `subagent_stop` | `SubagentStop` | observation | **NOT a gate** — see the correction below. |
| `pre_verify` | *(not installed)* | ✅ block | Real turn-end gate upstream — **we deliberately do not install it**; see below. |

**Corrections (2026-07-29).** Three claims that stood here were wrong, each verified
against upstream `hermes-agent` @ `5771a6e`. `agent/shell_hooks.py:567-621`
(`_parse_response`) is **event-gated**: it returns a verdict for `pre_tool_call` and
`pre_verify` only, and falls through to `return None` for everything else. So:

1. **`subagent_stop` was documented "✅ block" and is not one.** The parser never
returns a verdict for it, and the call site discards the return anyway
(`tools/delegate_tool.py:2677`, a bare `invoke_hook(...)`). Any customer policy
denying on SubagentStop had **zero** enforcement for as long as that row stood.
2. **"Hermes has no turn-end `Stop` event" was false.** `pre_verify` is exactly that.
Upstream fires it once per turn when the agent has edited code and is about to
finish (`agent/conversation_loop.py:6754`); its parser accepts our Claude Stop
shape verbatim — `{decision:"block",reason}` reads as "block the stop", i.e. keep
going — and the reason is injected as a synthetic user message before the loop
re-enters (`conversation_loop.py:6774-6800`). Installing it is one entry in
`config.yaml`; it is a shell hook like the other five, no Python involved.

**We deliberately do not install it** (product decision, 2026-07-29). So
`HERMES_EVENT_MAP` emits no `Stop` and the 5 `require-*-before-stop` builtins
remain **inapplicable** on Hermes — but that is now a choice, not a platform
limit, and this is what it would buy if revisited. Three upstream conditions
would apply: it fires only on turns that landed `write_file`/`patch`
(`agent/tool_result_classification.py:9` — a `terminal`-only turn does not qualify,
so a chat-only gateway may never see it), it is capped at 3 nudges per turn
(`agent/verify_hooks.py:21`, operator-overridable), and on Hermes older than
~2026-06-30 the config key fails `VALID_HOOKS` and is **warn-and-skipped silently**
(`agent/shell_hooks.py:325`).
3. **"No additional-context channel" was false.** `pre_llm_call` consumes
`{"context": str}` via the parser's fallthrough (`shell_hooks.py:617-621`). We do
not install it, so `instruct()` still degrades to allow + a stderr note — but that
is now a gap we chose, not a limit of the platform.

Hermes still lacks `UserPromptSubmit` (only per-LLM-call `pre_llm_call`),
`PreCompact`/`Notification`, etc. In exchange it has capabilities others lack
(`transform_tool_result`, `pre_gateway_dispatch`, `pre_llm_call`) — out of scope for now.

The authoritative, machine-readable version of this table is now
`src/hooks/enforcement-capability.ts`, which a test asserts against. Prefer it over this
prose: this table is exactly the artifact that drifted.

**Tool-input canonicalization.** `HERMES_TOOL_MAP` canonicalizes tool *names* and
`HERMES_TOOL_INPUT_MAP` the *argument keys*. Verified against a live `~/.hermes/state.db`:
Expand Down
119 changes: 118 additions & 1 deletion __tests__/actions/get-hooks-config.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
// @vitest-environment node
import { describe, it, expect, vi, beforeEach } from "vitest";
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";

vi.mock("@/src/hooks/hooks-config", () => ({
readHooksConfig: () => ({ enabledPolicies: [] }),
// The action walks up to the project root like enforcement does; these tests
// point cwd straight at the project root, so identity is the right stub.
findProjectConfigDir: (start: string) => start,
}));

vi.mock("@/src/hooks/manager", () => ({
Expand Down Expand Up @@ -91,3 +97,114 @@ describe("getHooksConfigAction — clis payload", () => {
expect(config.clis.find((c) => c.id === "codex")!.label).toBe("OpenAI Codex");
});
});

// ── Convention policies ──────────────────────────────────────────────────────
//
// These are registered by the filesystem, never by policies-config.json, so a
// dashboard that only reads config renders nothing and a working policy looks
// absent. That was the reported bug: `failproofai policies` listed four
// convention files while the dashboard's configure view showed none.
describe("getHooksConfigAction — convention policies", () => {
let tmp: string;

beforeEach(() => {
tmp = mkdtempSync(join(tmpdir(), "fp-convention-"));
// The action prefers FAILPROOFAI_LAUNCH_CWD over process.cwd(); if it is
// set in the shell or the CI image, the cwd spy below is bypassed and every
// test here silently resolves against the wrong root.
vi.stubEnv("FAILPROOFAI_LAUNCH_CWD", "");
});

afterEach(() => {
rmSync(tmp, { recursive: true, force: true });
vi.unstubAllEnvs();
});

function writePolicyDir(root: string, files: Record<string, string>) {
const dir = join(root, ".failproofai", "policies");
mkdirSync(dir, { recursive: true });
for (const [name, body] of Object.entries(files)) {
writeFileSync(join(dir, name), body, "utf8");
}
return dir;
}

const POLICY_SRC = `
import { customPolicies, allow } from "failproofai";
customPolicies.add({
name: "enforce-formal-review",
description: "Require a formal review",
match: { events: ["PreToolUse", "Stop"] },
fn: async () => allow(),
});
`;

it("discovers project-scope policy files and parses their hooks", async () => {
writePolicyDir(tmp, { "enforce-formal-review-policies.mjs": POLICY_SRC });
const spy = vi.spyOn(process, "cwd").mockReturnValue(tmp);
try {
const config = await getHooksConfigAction();
Comment thread
coderabbitai[bot] marked this conversation as resolved.
const entry = config.conventionPolicies.find(
(e) => e.file === "enforce-formal-review-policies.mjs",
);
expect(entry).toBeDefined();
expect(entry!.scope).toBe("project");
expect(entry!.policies).toEqual([
{
name: "enforce-formal-review",
description: "Require a formal review",
eventScope: "PreToolUse, Stop",
},
]);
} finally {
spy.mockRestore();
}
});

it("skips files that do not match the *policies.{js,mjs,ts} convention", async () => {
writePolicyDir(tmp, {
"enforce-formal-review-policies.mjs": POLICY_SRC,
// Silently skipped by the loader too — listing it would tell the user a
// file is active when it enforces nothing.
"block-foo.mjs": POLICY_SRC,
});
const spy = vi.spyOn(process, "cwd").mockReturnValue(tmp);
try {
const config = await getHooksConfigAction();
const files = config.conventionPolicies.map((e) => e.file);
expect(files).toContain("enforce-formal-review-policies.mjs");
expect(files).not.toContain("block-foo.mjs");
} finally {
spy.mockRestore();
}
});

it("returns an empty list when no policy directory exists", async () => {
// HOME must be isolated too, or the developer's own user-scope policies
// leak into the assertion.
const emptyHome = mkdtempSync(join(tmpdir(), "fp-home-"));
const spy = vi.spyOn(process, "cwd").mockReturnValue(tmp);
vi.stubEnv("HOME", emptyHome);
vi.stubEnv("USERPROFILE", emptyHome);
try {
const config = await getHooksConfigAction();
expect(config.conventionPolicies).toEqual([]);
} finally {
spy.mockRestore();
rmSync(emptyHome, { recursive: true, force: true });
}
});

it("does not list the same file twice when cwd is the home directory", async () => {
writePolicyDir(tmp, { "team-policies.mjs": POLICY_SRC });
const cwdSpy = vi.spyOn(process, "cwd").mockReturnValue(tmp);
vi.stubEnv("HOME", tmp);
try {
const config = await getHooksConfigAction();
const matching = config.conventionPolicies.filter((e) => e.file === "team-policies.mjs");
expect(matching).toHaveLength(1);
} finally {
cwdSpy.mockRestore();
}
});
});
Loading