Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
8a6a755
✨ (core): Add runtime_audit, the sink that ships an outcome to the ru…
Chisanan232 Aug 14, 2026
9f405ac
✨ (core): Add AUDIT_SINK_FORWARDED for a record that reaches the runtime
Chisanan232 Aug 14, 2026
515603e
✨ (core): Resolve record_result and on_tool_end on the SDK's interceptor
Chisanan232 Aug 14, 2026
f22f5e2
🐛 (core): Warn about the audit gap only when there is one
Chisanan232 Aug 14, 2026
44d6a45
✅ (test): Give the native double the send_event and GovernanceEvent i…
Chisanan232 Aug 14, 2026
7023eb4
✅ (test): Bind the forwarded declaration to the native boundary
Chisanan232 Aug 14, 2026
9740b8c
✅ (test): Forbid deferring audit recording to AAASM-5750 as well
Chisanan232 Aug 14, 2026
1b9227c
✅ (test): Pin the audit payload against the real GovernanceEvent
Chisanan232 Aug 14, 2026
95699e4
📝 (docs): Correct every remaining claim that no audit evidence is kept
Chisanan232 Aug 14, 2026
70bbdcf
🐛 (test): Stop the referent scan dying on a file that vanishes mid-walk
Chisanan232 Aug 14, 2026
5ec5b0b
📝 (docs): Scope an unqualified absolute the touched line carried
Chisanan232 Aug 14, 2026
45e9400
📝 (docs): Correct nine further claims an independent sweep found
Chisanan232 Aug 14, 2026
f26573e
📝 (core): Withdraw the Observed claim; a handoff is not evidence
Chisanan232 Aug 14, 2026
fcba3e1
📝 (adapters): Scope the audit claim to the paths that actually record
Chisanan232 Aug 14, 2026
76c8faa
📝 (docs): Name the eight adapters that record nothing on a deny
Chisanan232 Aug 14, 2026
b4c8d21
📝 (core): Cite AAASM-5783 as the open work behind the Observed gap
Chisanan232 Aug 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,16 @@ ordered by latency cost (lowest first) and detection authority (highest first):

1. **SDK layer (in-process)** — *this repo*. The SDK applies pre-execution allow/deny
on tool calls via the native shim over `aa-sdk-client`. Fastest path; requires SDK
adoption. It does **not** emit audit events: the adapters offer every governed
outcome to an audit hook, but on every interceptor this SDK ships that hook does not
resolve, so nothing is recorded — for allowed calls as much as denied ones
(AAASM-5731). Do not describe this layer as producing an audit trail.
adoption. It hands audit events to the runtime **only over a connected runtime**:
the adapters offer a governed outcome to an audit hook, and
`RuntimeQueryInterceptor` writes it to the native event channel. Two limits are
load-bearing and must not be dropped when describing this: the send is
unacknowledged, so a handoff is **not** evidence and never ADR 0033 §6
*Observed* — AAASM-5783 is open on the downstream half and must land before
that changes; and only `google_adk`, `pydantic_ai` and `openai_agents` record on the
**denied** path — the other eight governed adapters return or raise first. With no
reachable runtime nothing is recorded at all (AAASM-5750). Never describe this
layer as producing an audit trail.
2. **Sidecar proxy (`aa-proxy`)** — MitM of outbound HTTPS; enforces network-egress
policy with no code changes. (Lives in the monorepo.)
3. **eBPF (`aa-ebpf*`)** — kernel uprobes; catches everything, including bypass
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,15 @@

Python SDK for **AI Agent Assembly** — a governance-native runtime for AI agents. One `init_assembly()` call wires your agent into the policy gateway and applies pre-execution allow/deny on tool calls, without changing how the agent itself is written.

> **The SDK layer produces no audit evidence of its own.** The framework adapters offer the outcome of every governed call to an audit hook on the governance interceptor — but on every interceptor this SDK ships, that hook **does not resolve**, so nothing is emitted. This covers **allowed** calls as much as denied ones. Enforcement is unaffected: a policy DENY still blocks the tool. `init_assembly()` warns about it and reports `audit_sink` on the returned context; supply your own handler with a `record_result` or `on_tool_end` to retain the record ([AAASM-5731](https://lightning-dust-mite.atlassian.net/browse/AAASM-5731)).
> **The SDK hands records to the runtime; it does not give you an audit trail.** The framework adapters offer a governed call's outcome to an audit hook on the governance interceptor, and over a connected runtime that hook writes it to the native event channel — the same one agent registration uses. That is a handoff, **not** evidence: the send is unacknowledged, so this SDK cannot tell you the record arrived, and does not claim it did. Downstream, [AAASM-5783](https://lightning-dust-mite.atlassian.net/browse/AAASM-5783) is open on `report_event` payloads reaching neither the live stream nor the durable entry — until it lands, no SDK can claim ADR 0033 §6 *Observed*. Without a reachable runtime there is no channel at all and nothing is emitted.
>
> **Denied calls are mostly not covered.** Only `google_adk`, `pydantic_ai` and `openai_agents` build a record on the denied path. The other eight governed adapters — `crewai`, `llamaindex`, `haystack`, `agno`, `smolagents`, `microsoft_agent_framework`, `mcp` and `langchain` — return or raise before their record helper, so a deny there produces no record for any sink to carry. Enforcement is unaffected either way: a policy DENY still blocks the tool. `init_assembly()` warns when no record can be sent and reports `audit_sink` on the returned context ([AAASM-5750](https://lightning-dust-mite.atlassian.net/browse/AAASM-5750)).

## Why use it

- **Framework adapters** for LangChain, LangGraph, CrewAI, OpenAI Agents, Pydantic AI, Google ADK, Haystack, Smolagents, Agno, LlamaIndex, Microsoft Agent Framework, and MCP servers — drop in, no SDK rewrites required.
- **Pre-execution policy enforcement** via the `FrameworkAdapter` ABC — block disallowed tool calls before they hit the LLM.
- **Agent lineage** — parent / root / team identity is registered with the gateway and carried on every policy check. (An audit *trail* is not part of what this SDK layer delivers — see the note above.)
- **Agent lineage** — parent / root / team identity is registered with the gateway and carried on every policy check. (An audit *trail* from this SDK layer depends on a reachable runtime — see the note above.)
- **Native PyO3 fast path** (optional) — drop into a Rust runtime client when you need sub-millisecond policy checks.
- **Typed throughout** — Pydantic models for every gateway payload, mypy strict on adapter base and registry.

Expand Down
45 changes: 23 additions & 22 deletions agent_assembly/adapters/_shared/tool_governance.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@
verdict, honour a ``pending`` approval round-trip, deny by raising when the
verdict is ``deny``, otherwise run the original inside a spawn-context scope.
Either way the outcome is *offered* to the audit hook before the flow ends
(AAASM-5665) — offered, not recorded: on every interceptor this SDK ships the hook
does not resolve, so nothing is retained on either path. See
:func:`_record_async_tool_result` for the measurement. That shared body — previously duplicated verbatim in both
(AAASM-5665). Whether the hook resolves depends on the interceptor: over a
connected runtime it does, and the record is forwarded to the runtime's evidence
pipeline (AAASM-5750); without one it does not, and nothing is retained on either
path. See :func:`_record_async_tool_result` for the measurement. That shared body — previously duplicated verbatim in both
adapters (the cross-file duplication SonarCloud flagged on PR #269, AAASM-4746) —
lives here so each adapter keeps only its framework-specific glue.

Expand Down Expand Up @@ -178,22 +179,22 @@ async def _record_async_tool_result(
apart from a tool that ran and returned that same text.

Whether anything is recorded depends entirely on the ``callback_handler``.
Both hooks are duck-typed, and on every interceptor the SDK ships *neither
resolves*: ``RuntimeQueryInterceptor`` defines only ``check_tool_start`` and
delegates the rest to ``GatewayClient``, whose surface has no
``record_result`` and no ``on_tool_end``. Measured against the native and
HTTP boundaries, this function therefore finds no hook and emits nothing on
the shipped path — for allowed calls as much as denied ones.

Under ADR 0033 §6 that makes SDK-side recording **Planned** (AAASM-5750),
not *Unmeasured*: §6 reserves ``Unmeasured`` for an action no control
inspected, where nothing is known, and here exactly where the record stops
has been measured. It is certainly not *Observed*, which needs a durable
event attributed to the action. Every handler the SDK ships declares this in
``audit_sink`` (see :mod:`agent_assembly.core.audit_sink`), ``init_assembly``
warns about it, and a caller that supplies its own handler does get the
record. Wiring a sink into the SDK's own interceptor is a separate
capability.
Both hooks are duck-typed. Over a connected runtime
``RuntimeQueryInterceptor.record_result`` resolves and writes the record to
the native event channel (AAASM-5750). That is a handoff and **not** ADR 0033
§6 *Observed*: the send is unacknowledged, so nothing here establishes a
durable event attributed to the action, and AAASM-5783 is open on the
downstream half of that. Without a runtime neither hook resolves and this
function emits nothing, measured against the native and HTTP boundaries.

Note the scope of "allowed or denied" here: *this* shared flow calls the hook
on both paths, which is why ``google_adk`` and ``pydantic_ai`` cover denies.
Most adapters do not route through it and return or raise before their own
record helper, so their denied calls produce no record at all.

Which of the two a run is in is declared in ``audit_sink`` (see
:mod:`agent_assembly.core.audit_sink`) and warned about by ``init_assembly``;
a caller that supplies its own handler gets the record either way.
"""
denial_flag = {"denied": denied} if denied else {}

Expand Down Expand Up @@ -295,9 +296,9 @@ async def run_governed_async_tool(
# Offer the deny to the audit hook before raising (AAASM-5665).
# Previously this raised straight past the record call below, so a
# denied call could not reach an audit sink even when the caller had
# supplied one. See _record_async_tool_result on why the SDK's own
# interceptor still resolves no hook, so the shipped path emits nothing
# here either (AAASM-5731).
# supplied one. See _record_async_tool_result for where the record then
# goes: the SDK's own interceptor resolves the hook over a connected
# runtime and forwards it, and resolves nothing without one (AAASM-5750).
#
# Best-effort, and the guard is load-bearing: the hook is duck-typed
# from caller-supplied code, and inserting a call here where none used
Expand Down
6 changes: 4 additions & 2 deletions agent_assembly/adapters/haystack/patch.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,10 @@
The interceptor contract mirrors the other tool-call adapters (CrewAI, Pydantic AI):
a ``check_tool_start`` pre-execution gate that returns ``allow`` / ``deny`` /
``pending``, an optional ``wait_for_tool_approval`` for the pending flow, and a
post-execution ``record_result`` / ``on_tool_end`` audit hook — which no interceptor
this SDK ships resolves, so the outcome is offered and not recorded (AAASM-5731).
post-execution ``record_result`` / ``on_tool_end`` audit hook — which the SDK's own
interceptor resolves over a connected runtime, handing the outcome to the
runtime's event channel, and does not resolve without one. Note this adapter does
not reach that hook on the denied path: it raises first (AAASM-5750).
Under the fail-closed
``enforce`` posture an unknown or malformed verdict denies (AAASM-3107).
"""
Expand Down
12 changes: 8 additions & 4 deletions agent_assembly/adapters/langchain/callback_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,14 @@ def audit_sink(self) -> AuditSinkDisposition:
wrapped, and this handler sits on the *other* side of the split from the
interceptors it wraps. ``on_tool_end`` is defined here, so the adapters'
audit-hook lookup **does** resolve on this object — the record is built
and handed over. It is then forwarded to the interceptor's own
``on_tool_end``, and on every interceptor this SDK ships there is none,
so the record stops here: accepted and dropped, which is ``discarded``,
not ``absent``.
and handed over — and it is then forwarded to the interceptor's own
``on_tool_end``.

So the answer is the wrapped interceptor's, with one substitution: when
the wrapped one resolves no hook at all, the record still reaches *this*
object before stopping, which is ``discarded`` and not ``absent``. The
distinction is not cosmetic — ``absent`` says nothing constructs the
event, and here something does.

A caller-supplied interceptor that really records is reported as such:
this SDK does not claim anything about a handler it did not build, in
Expand Down
6 changes: 4 additions & 2 deletions agent_assembly/adapters/llamaindex/adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ class LlamaIndexAdapter(FrameworkAdapter):

Wires the SDK-layer pre-execution allow/deny onto the LlamaIndex
tool-execution path (``FunctionTool.call`` / ``acall``), and offers each
outcome to the audit hook — which no interceptor this SDK ships resolves, so
nothing is recorded from here (AAASM-5731). The
outcome to the audit hook — which the SDK's own interceptor resolves over a
connected runtime, handing the record to the runtime's event channel, and
does not resolve without one. Only the *allowed* path reaches it here: a deny
raises before the record helper, so it produces no record (AAASM-5750). The
framework package is imported as ``llama_index.core``; the patch targets the
concrete tool methods the agent loop actually invokes (the base methods are
abstract).
Expand Down
54 changes: 35 additions & 19 deletions agent_assembly/core/assembly.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,12 +124,16 @@ class AssemblyContext:
# (AAASM-4547, mirroring the Node SDK's ``ctx.registered``).
registered: bool = True
# What the governance interceptor the adapters were handed does with the
# hook-layer audit record for a governed tool call (AAASM-5731). Anything
# other than ``"caller-supplied"`` means governed actions produce NO audit
# evidence from this SDK, so no claim of attributability or after-the-fact
# review holds on the SDK path. The programmatic counterpart of the stderr
# warning ``_warn_audit_not_recorded`` emits, so the gap is detectable in
# code and not only by reading stderr.
# hook-layer audit record for a governed tool call (AAASM-5731).
# ``"forwarded"`` means the record is handed to the runtime's event channel
# (a handoff, not evidence — the send is unacknowledged);
# ``"absent"`` and ``"discarded"`` both mean governed actions produce NO
# audit evidence from this SDK, so no claim of attributability or
# after-the-fact review holds on that path; ``"caller-supplied"`` means this
# SDK makes no claim. The programmatic counterpart of the stderr warning
# ``_warn_audit_not_recorded`` emits for the two evidence-free values, so
# which case a run is in is detectable in code and not only by reading
# stderr.
audit_sink: AuditSinkDisposition = AUDIT_SINK_ABSENT
_lock: Lock = field(default_factory=Lock, init=False, repr=False)
_is_shutdown: bool = field(default=False, init=False, repr=False)
Expand Down Expand Up @@ -325,7 +329,14 @@ def init_assembly(
# AAASM-5731 — surface an audit path that retains nothing, on the
# default path with nothing opted into. Emitted after registration so it
# reflects the interceptor the adapters were actually handed.
if audit_sink != AUDIT_SINK_CALLER_SUPPLIED:
#
# The condition enumerates the dispositions that warrant a warning rather
# than excluding the one that does not (AAASM-5750). Written as
# ``!= AUDIT_SINK_CALLER_SUPPLIED`` it warned about every value that was
# not the caller's own, which silently included ``forwarded`` the moment
# that value existed — telling a caller whose records do reach the runtime
# that they produce none.
if audit_sink in (AUDIT_SINK_ABSENT, AUDIT_SINK_DISCARDED):
_warn_audit_not_recorded(audit_sink)

context = AssemblyContext(
Expand Down Expand Up @@ -397,15 +408,18 @@ def _warn_agent_unregistered(detail: str) -> None:


def _warn_audit_not_recorded(disposition: AuditSinkDisposition) -> None:
"""Emit a loud, unconditional stderr warning that no audit record is kept.
"""Emit a loud stderr warning that no audit record is kept on this run.

The framework adapters offer the outcome of every governed tool call to an
audit hook on the interceptor they were handed. On every interceptor this SDK
ships that hook does not resolve, so nothing is emitted — for **allowed**
calls as much as denied ones — and the caller had no way to learn that short
of reading the interceptor. Enforcement is genuinely unaffected, which is
exactly why the gap is easy to miss: denies still deny, and the governed call
returns normally.
audit hook on the interceptor they were handed. Over a connected runtime that
hook resolves and forwards the record (AAASM-5750); without one it does not
resolve at all, so nothing is emitted — for **allowed** calls as much as
denied ones — and the caller had no way to learn that short of reading the
interceptor. Enforcement is genuinely unaffected, which is exactly why the gap
is easy to miss: denies still deny, and the governed call returns normally.

Fires only for the dispositions that leave no evidence; see the enumeration at
the call site in :func:`init_assembly`.

Written straight to ``sys.stderr`` for the same reason as
:func:`_warn_agent_unregistered`: ``logging`` configuration cannot silence it.
Expand All @@ -425,18 +439,20 @@ def _warn_audit_not_recorded(disposition: AuditSinkDisposition) -> None:
"the interceptor it forwards to exposes no on_tool_end"
if disposition == AUDIT_SINK_DISCARDED
else "no audit hook (record_result / on_tool_end) resolves on the governance "
"interceptor this SDK builds, so no record is even attempted"
"interceptor this SDK built, because no runtime is reachable for it to "
"send a record to, so no record is even attempted"
)
sys.stderr.write(
"[agent-assembly] WARNING: hook-layer audit records are NOT retained "
f"(audit sink '{disposition}'): {mechanism}. Governed tool calls — ALLOWED "
"ones as well as denied ones — therefore produce NO audit evidence from "
"this SDK, and nothing on this path can be attributed or reviewed after "
"the fact. Enforcement is unaffected: a policy DENY still blocks a tool "
"call, and the proxy / eBPF layers remain authoritative. Supply your own "
"handler with a record_result or on_tool_end to retain the record, and "
"inspect the 'audit_sink' attribute on the returned assembly context to "
"detect this programmatically (AAASM-5731).\n"
"call, and the proxy / eBPF layers remain authoritative. Connect a runtime so "
"the SDK's own sink resolves, or supply your own handler with a "
"record_result or on_tool_end, and inspect the 'audit_sink' attribute on "
"the returned assembly context to detect this programmatically "
"(AAASM-5731, AAASM-5750).\n"
)


Expand Down
Loading