Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 22 additions & 20 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -670,28 +670,30 @@ jobs:
done
fi

MISSING=()
for NAME in "${NAMES[@]}"; do
# The registry is a read-through cache, so a just-published version
# can take a moment to be visible everywhere. Check immediately,
# then back off 10s / 30s / 1m / 2m before calling it missing —
# long enough that propagation is not mistaken for a failed publish,
# short enough that a genuinely failed publish is still reported in
# the same run rather than hours later by a user.
FOUND=""
for DELAY in 0 10 30 60 120; do
[[ "$DELAY" -gt 0 ]] && sleep "$DELAY"
if npm view "$NAME@$PUBLISH_VERSION" version >/dev/null 2>&1; then
FOUND=1
break
# npm may accept a publish but keep a large package in asynchronous
# processing for much longer than normal registry propagation. The
# 1.0.6-beta.0 root package took 11m25s to become queryable after npm
# printed `+ failproofai@...`; the previous 3m40s window therefore
# reported a false version split and skipped install verification.
# Check every still-missing name in each round so several genuinely
# missing packages cannot multiply this bounded wait.
MISSING=("${NAMES[@]}")
for DELAY in 0 10 30 60 120 180 300 300; do
[[ "$DELAY" -gt 0 ]] && sleep "$DELAY"
STILL_MISSING=()
for NAME in "${MISSING[@]}"; do
if npm view "$NAME@$PUBLISH_VERSION" version --prefer-online >/dev/null 2>&1; then
echo " ok $NAME@$PUBLISH_VERSION"
else
STILL_MISSING+=("$NAME")
fi
done
if [[ -n "$FOUND" ]]; then
echo " ok $NAME@$PUBLISH_VERSION"
else
echo " MISSING $NAME@$PUBLISH_VERSION"
MISSING+=("$NAME")
fi
MISSING=("${STILL_MISSING[@]}")
[[ ${#MISSING[@]} -eq 0 ]] && break
done

for NAME in "${MISSING[@]}"; do
echo " MISSING $NAME@$PUBLISH_VERSION"
done

if [[ ${#MISSING[@]} -gt 0 ]]; then
Expand Down
37 changes: 37 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,48 @@
# Changelog

## 1.0.6 — 2026-09-16

Stable native Hermes policy-enforcement release, validated across CLI and
Telegram sessions, default and named profiles, instruction delivery, hard
denials, daemon-backed evaluation, and end-to-end event ingestion.

### Added

- Add a native Hermes plugin backed by the existing `failproofaid` warm worker. Policies now intercept Hermes tool calls before execution, deliver model-visible `instruct()` guidance with a bounded retry ledger, and preserve hard blocking for `deny()` decisions.
- Add structured `policyEvaluation` / `policyResult` daemon messages for native integrations, including canonical tool, policy, match, reason, and latency metadata.

### Fixes

- Install, remove, and report health for every active Hermes home, including the default profile, upstream nested profiles, and config-bearing `~/.hermes-<name>` installations.
- Require an exact native-policy capability check before changing any Hermes plugin or profile configuration. Missing, unhealthy, outdated, or protocol-incompatible daemons are rejected with setup instructions before fail-closed enforcement can be enabled.
- Apply Hermes evaluation timeouts as one total socket deadline and keep advisory instruction retries bounded so an unavailable state store or repeatedly unchanged action cannot deadlock a turn.
- Make npm release verification tolerate registry propagation delay while still requiring the root package and all platform daemon packages to publish at the same version.

## 1.0.6-beta.1 — 2026-09-15

### Fixes

- Require an exact `policyEvaluation` / `policyResult` capability probe before direct Hermes installation changes plugin or profile configuration. A healthy older daemon that only supports shell-hook requests is now rejected with upgrade instructions instead of enabling a fail-closed plugin it cannot serve.
- Discover config-bearing `~/.hermes-<name>` installations alongside the default and upstream nested profiles, so install, uninstall, health checks, and audit coverage reach every Hermes home without mistaking empty backup directories for active profiles.
- Make daemon readiness an integration capability rather than a Hermes command special-case, so future daemon-only plugins inherit the same pre-install lockout protection while CLI-backed integrations keep their local evaluator fallback.
- Apply Hermes `evaluation_timeout_ms` as one total socket deadline instead of resetting the full timeout for each partial send or receive.
- Allow npm up to 16 minutes 40 seconds to expose an accepted release while checking every package in parallel per retry round. The `1.0.6-beta.0` root package took 11 minutes 25 seconds to become visible after `npm publish` succeeded, causing the workflow to report a false version split while all five packages had actually published.

## 1.0.6-beta.0 — 2026-09-15

### Docs

- Translate the English documentation changes from #788 and #791 into all 14 locales, including the new evaluation pages and SDK event redaction on the custom-agents page, and point translated fragment links at their translated headings (#797)

### Added

- Add a native, profile-local Hermes plugin that evaluates policies through the existing failproofaid warm worker. Hermes `instruct()` decisions now interrupt the first matching tool attempt with model-visible guidance, then use a persistent bounded retry ledger so advisory policy cannot deadlock the turn; `deny()` remains a hard block.
- Add structured `policyEvaluation` / `policyResult` messages to the versioned local daemon protocol for native integrations, including canonical tool, policy, match, reason, and latency metadata.

### Changed

- Hermes installation now copies and enables the managed plugin in every profile, migrates only legacy FailproofAI shell hooks, refuses to overwrite unmanaged plugin directories, and reports incomplete or duplicate profile installations as unhealthy.

### Dependencies

- yaml 2.9.0 → 2.9.1, and rustls 0.23.43 → 0.23.45 (with rustls-webpki 0.103.13 → 0.103.15) in `Cargo.lock`, closing RUSTSEC-2026-0285 (5.3, fixed in 0.23.45). The advisory turned the Supply Chain gate red on `main` itself, not through any PR's change; rustls is transitive-only, via `reqwest` in `failproofaid` and `fpai-collect` (#803)
Expand Down
92 changes: 55 additions & 37 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,13 +321,12 @@ we run in-repo. Hermes is a **dual-pillar** integration: an **audit** adapter
(`src/audit/cli-adapters/hermes.ts`, reads `~/.hermes/state.db` directly) **and** a
**live-hook** integration (`hermes` in `INTEGRATIONS`).

Hermes uses a **Claude/Codex-style external shell-hook system**, but its config is
**YAML** (`~/.hermes/config.yaml`) under a `hooks:` map — the only integration whose
settings file is YAML, so `integrations.ts` has a comment-preserving `readYamlDoc`/
`writeYamlDoc` layer (the `yaml` package's `Document` API) that rewrites only the
`hooks:` key, preserving the operator's other settings and any comments **outside**
that block. (Comments *inside* the `hooks:` map are not preserved — we rebuild the
key from `doc.toJS()` — but failproofai owns that block, so there's nothing to keep.)
Hermes enforcement uses the shipped **native Python plugin** under each profile's
`plugins/failproofai/` directory. The profile's YAML config enables it through
`plugins.enabled: [failproofai]`. `integrations.ts` copies the plugin atomically,
marks the directory as FailproofAI-managed, refuses to overwrite an unmanaged
directory with the same name, and uses the `yaml` package's comment-preserving
`Document` API for config changes.

Settings file paths:

Expand All @@ -336,21 +335,21 @@ Settings file paths:
| user | `~/.hermes/config.yaml` |

Hermes is **user-scope only** — there is no project config, so `getSettingsPath`
ignores scope/cwd. Hermes exposes no `$HERMES_PROJECT_DIR`; the installed command uses
the resolved binary path (`"${binaryPath}" --hook <event> --cli hermes`) — since
Hermes is user-scope only, no `npx` project form applies. `timeout` is in **seconds** (30).

**Consent (headless gateway).** Hermes prompts once per unique `(event, command)` hook
before running it. The gateway has no TTY, so install also writes
`hooks_auto_accept: true` into config.yaml (uninstall removes it). Tradeoff: this
auto-accepts *any* hook the operator adds — a deliberate choice for headless operation.
A more targeted alternative is to pre-seed `~/.hermes/shell-hooks-allowlist.json` with
just our `(event, command)` pairs; deferred as a future refinement.

**Block contract** (verified live): Hermes reads a `{"decision":"block","reason"}` JSON
object on **stdout** and **ignores exit codes**. So `policy-evaluator.ts` has a
`cli === "hermes"` deny branch (ahead of the Stop block) that emits that shape
unconditionally for every event — one branch covers PreToolUse/PostToolUse/SubagentStop.
ignores scope/cwd. Every default and named profile receives its own plugin copy and
enablement entry. Installed-state detection requires both the complete managed plugin
directory and the config entry; a missing file, disabled plugin, newly-created profile,
or leftover legacy shell hook is reported as unhealthy.

The plugin talks directly to `failproofaid` over the existing owner-only Unix socket.
The daemon relays a versioned `policyEvaluation` request to the warm TypeScript worker
and returns a structured `policyResult` with `allow`, `deny`, or `instruct`, policy
names, reason, matched policies, canonical tool name, and evaluation duration. The
normal native path does not parse CLI stdout and does not spawn a process per event.

The contract is pinned to upstream Hermes 0.21.3 / commit
`4d55ca91656ac5f83e1506679b7f81e0238e5e16`: plugin manifest v2, `PluginState.data_dir`,
keyword hook payloads, and `pre_tool_call` correlation IDs are all present there.
Callbacks accept `**kwargs` for additive compatibility.

**Platform independence & subagents.** The gateway is one Hermes process and
`pre_tool_call` fires on the *tool event*, not the source — so a single install
Expand All @@ -362,17 +361,36 @@ internal tool calls don't fire Hermes hooks — gate the *spawn* at `pre_tool_ca

**Per-event capability matrix:**

| Hermes event | Canonical (`HERMES_EVENT_MAP`) | Veto / mutate? | Notes |
|--------------------|--------------------------------|----------------|-------|
| `pre_tool_call` | `PreToolUse` | ✅ block | The core deny point — stops the tool before it runs. |
| `post_tool_call` | `PostToolUse` | observation | Observe / sanitize. |
| `on_session_start` | `SessionStart` | observation | — |
| `on_session_end` | `SessionEnd` | observation | — |
| `subagent_stop` | `SubagentStop` | observation | **NOT a gate** — see the correction below. |
| `pre_verify` | *(not installed)* | ✅ block | Real turn-end gate upstream — **we deliberately do not install it**; see below. |

**Corrections (2026-07-29).** Three claims that stood here were wrong, each verified
against upstream `hermes-agent` @ `5771a6e`. `agent/shell_hooks.py:567-621`
| Hermes plugin hook | Canonical (`HERMES_EVENT_MAP`) | Effect | Notes |
|--------------------|--------------------------------|--------|-------|
| `pre_tool_call` | `PreToolUse` | ✅ block | Deny always blocks. Instruct blocks once for model-visible delivery, then the next API iteration may proceed. |
| `post_tool_call` | `PostToolUse` | observation | Return value is ignored by Hermes. |
| `pre_llm_call` | plugin-local | context | Adds the stable instruction protocol hint to the current turn. |
| `on_session_start` | `SessionStart` | observation | Forwarded to the evaluator. |
| `on_session_end` | `SessionEnd` | observation + cleanup | Forwarded, then clears that session's instruction ledger. |
| `on_session_reset` / `on_session_finalize` | plugin-local | cleanup | Clears stale per-session retry state. |
| `subagent_stop` | `SubagentStop` | observation | Not a gate. |
| `pre_verify` | *(not installed)* | none | Hermes exposes a bounded turn-end gate, but FailproofAI does not map it yet. |

**Bounded `instruct()` delivery.** The first matching instruction in a
profile/session/task/turn/policy/tool scope is persisted to the profile-local SQLite
ledger before `pre_tool_call` returns a block. Re-entry with the same
`api_request_id` remains blocked, which prevents sibling calls from consuming the
permit. A later API request acknowledges the instruction and proceeds. At most two
distinct instruction scopes may interrupt one turn by default. Missing correlation
IDs or ledger failure fail open for advisory instructions so a broken ledger cannot
deadlock Hermes; a real `deny()` remains a hard block. Evaluator or protocol failure
defaults to fail closed and may be changed per profile with
`plugins.entries.failproofai.settings.failure_mode: allow`.

Install migrates only shell-hook entries carrying the FailproofAI marker or legacy
`failproofai --hook` command. It preserves unrelated hooks and removes the old broad
`hooks_auto_accept` setting only when no hook remains. The native plugin itself needs
no shell-hook consent prompt.

**Legacy shell-adapter findings (retained for migration context).** Three earlier
claims were corrected against upstream `hermes-agent` @ `5771a6e`.
`agent/shell_hooks.py:567-621`
(`_parse_response`) is **event-gated**: it returns a verdict for `pre_tool_call` and
`pre_verify` only, and falls through to `return None` for everything else. So:

Expand All @@ -398,10 +416,10 @@ against upstream `hermes-agent` @ `5771a6e`. `agent/shell_hooks.py:567-621`
(`agent/verify_hooks.py:21`, operator-overridable), and on Hermes older than
~2026-06-30 the config key fails `VALID_HOOKS` and is **warn-and-skipped silently**
(`agent/shell_hooks.py:325`).
3. **"No additional-context channel" was false.** `pre_llm_call` consumes
`{"context": str}` via the parser's fallthrough (`shell_hooks.py:617-621`). We do
not install it, so `instruct()` still degrades to allow + a stderr note — but that
is now a gap we chose, not a limit of the platform.
3. **"No additional-context channel" was false.** Native `pre_llm_call` callbacks
consume `{"context": str}`. The plugin uses that channel for a stable protocol
hint, while each concrete `instruct()` reason is delivered through the blocked
`pre_tool_call` result that Hermes inserts into model-visible history.

Hermes still lacks `UserPromptSubmit` (only per-LLM-call `pre_llm_call`),
`PreCompact`/`Notification`, etc. In exchange it has capabilities others lack
Expand Down
6 changes: 3 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ resolver = "3"
members = ["crates/*"]

[workspace.package]
version = "1.0.6-beta.0"
version = "1.0.6"
edition = "2024"
license-file = "LICENSE"
repository = "https://github.com/FailproofAI/failproofai"
4 changes: 3 additions & 1 deletion __tests__/ci/release-pipeline.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,9 @@ describe("publish.yml", () => {
expect(verify.if).toContain("dry_run != 'true'");
// The registry is a read-through cache — propagation must not read as a
// failed publish, and a failed publish must not wait forever.
expect(verify.run).toContain("for DELAY in 0 10 30 60 120");
expect(verify.run).toContain("for DELAY in 0 10 30 60 120 180 300 300");
expect(verify.run).toContain('MISSING=("${NAMES[@]}")');
expect(verify.run).toContain('STILL_MISSING+=("$NAME")');
Comment thread
chhhee10 marked this conversation as resolved.
});

it("installs the published packages from the registry, once per platform", () => {
Expand Down
Loading