Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,81 @@
## [0.19.0] - 2026-09-30

Closes the SDK-side bypasses found auditing `DEF-MP-TS12-ENF-01`
(QA cycle RUN_ID 20260929T1338): the gate answering the agent
"allowed" on calls it had actually blocked or never checked.

### Surface (breaking)

- `NullRunRuntime.execute(..., mode="inline")` removed. It
returned a synthesised local `allow` **without contacting the
gateway**, so budget, rate limit and tool-block policies were
all skipped — the SDK's own `explanation` string said as much.
The only guard was a sensitivity check, which meant the safety
of a tool call depended on whether someone had remembered to
mark it sensitive. It now raises `NullRunConfigError`
(`error_code="NR-S001"`) at the top of `execute`, before any
context resolution. **There is no replacement**: every call
goes through `/execute`. If you were using `mode="inline"` to
avoid a round-trip, drop the argument — `mode="auto"` (the
default) already always contacts the gateway.
- `nullrun.runtime.register_strict_mode_forced`,
`nullrun.runtime.is_strict_mode_forced` and the module-level
`_STRICT_MODE_FORCED` set removed. They existed only to force
strict mode past the inline fast path. `register_strict_mode_forced`
already had zero callers (the `@sensitive` decorator its own
docstring referenced no longer exists in the SDK); dead security
machinery reads as a live mechanism and invites a bypass being
wired back up.
- `MCPAdapter(runtime=None)` no longer means "do not gate".
`call_tool` was conditional on `self._runtime is not None`, and
`runtime` defaulted to `None` — so a default-constructed adapter
(which is what the module's own documented example builds) called
the MCP server with no `/execute` round-trip at all. The operator
got a contextvar that a *later* `@protect` wrapper might read on
its *next* `/check`: post-hoc annotation, not enforcement. The
umbrella `mcp_destructive_policy` / `mcp_readonly_policy`
therefore applied to a locally-declared function but not to a
remote MCP call, on the same agent, in the same loop.
`runtime=None` now means "resolve the global runtime", on the
same terms `@protect` resolves it. Resolution is lazy — at
`call_tool`, not at construction — so the adapter stays
constructible in fixtures and doc snippets without
`nullrun.init()`. A missing API key raises rather than degrading
to an ungated call, matching `@protect`'s fail-loud invariant.
Passing `runtime=` explicitly still works and still wins.

`mode` itself is unchanged on the wire — it is still sent, and the
backend still ignores it (`transport.py`: "Wire-present but unused
by backend"). Its only real function was deciding whether to skip
enforcement.

The per-runtime sensitivity registry (`add_sensitive_tool`,
`register_sensitive_tools`, `remove_sensitive_tool`,
`is_sensitive_tool`, `get_sensitive_tools`) is **unchanged** — it
is a separate documented surface, and no longer has a consumer
inside `execute`. See ADR-061.

### Fixed

- A real gate decision is no longer treated as a transport
fail-open. The LangChain/LangGraph callback swallowed
`NullRunBlockedException` alongside transport errors; enforcement
exceptions are now recorded and re-raised through a
thread-local deferred handoff to the `@protect` boundary, which
is the only place the framework lets an exception abort.
- One definition of a synthetic decision. `decorators.py` and
`runtime.py` each had their own test for "is this source
synthetic", and the two disagreed on `AUTH_ERROR`; both now call
`is_fallback_decision_source`.
- A skipped pre-flight gate is countable. `check_control_plane`
and `check_workflow_budget` still no-op when no workflow can be
resolved (correct — a never-bound key has no control-plane state
and no per-workflow budget), but the no-op now increments
`control_plane_no_workflow_total` /
`budget_preflight_no_workflow_total` at DEBUG instead of being
indistinguishable from normal operation. Behaviour is unchanged:
it still never raises.

## [0.18.5] - 2026-09-26

Consolidated release that bundles three rounds of work landed on
Expand Down
35 changes: 25 additions & 10 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ build-backend = "hatchling.build"
name = "nullrun"
# Full release history lives in CHANGELOG.md; only the current version
# is pinned here.
version = "0.18.5"
version = "0.19.0"
# Kept under the 200-char preview threshold so the full line is visible
# without an "expand" click. The headline is the canonical §1 statement
# from positioning.md — "runtime decision layer for tool-using AI agents"
Expand Down Expand Up @@ -103,15 +103,30 @@ dev = [
# their data. Wrapping ``pytest -n auto`` in ``coverage run`` only
# traces the coordinator process and produces a false 0% report.
"pytest-cov>=5.0",
# The SDK eagerly imports `nullrun.instrumentation.langgraph`
# (from `nullrun.decorators`, imported by `nullrun.__init__` at
# collection time), which itself does `from langchain_core.callbacks
# import BaseCallbackHandler`. Without this dep, *every* test in
# the suite errors at pytest collection, not at a specific test.
# CI installs `[dev]` only, so the test extras need to cover the
# import chain. `langchain-core` is the smallest dep that makes
# the import succeed; the `langgraph` and `langchain` extras pull
# in heavier stacks that the unit tests don't need.
# `langchain-core` is a dev dep so CI exercises the REAL LangChain
# path, not the SDK's `object` fallback.
#
# It was originally here for a different reason, and that reason
# stopped being true: the SDK used to import
# `nullrun.instrumentation.langgraph` eagerly (reached from
# `nullrun.__init__` at collection time), which did an unguarded
# `from langchain_core.callbacks import BaseCallbackHandler`. With
# no such dep, *every* test in the suite errored at pytest
# collection. That import is now guarded — DEF-MP-TS12-SDK-05
# (RUN_ID 20260929T1338) made it degrade to an `object` base
# instead, and collection without langchain-core now succeeds
# (verified: 1449 tests collected, exit 0, with `langchain_core`,
# `langchain` and `langgraph` all blocked at import).
#
# The dep is still required, because the guarded fallback is a
# DEGRADED mode: with langchain-core absent, the tests that pin
# `BaseCallbackHandler is the real BaseCallbackHandler`
# (`test_langgraph_optional`) and that pin LangChain's
# swallow-exceptions contract (`test_langchain_enforcement`) would
# SKIP rather than run, and the real integration would go untested.
# `langchain-core` is the smallest dep that covers both; the
# `langgraph` and `langchain` extras pull in heavier stacks the
# unit tests don't need.
"langchain-core>=0.3,<1.0",
# `tests/test_integrations_fastapi.py` does `from fastapi import ...`
# at module top-level, so pytest collection aborts the entire suite
Expand Down
2 changes: 1 addition & 1 deletion src/nullrun/__version__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@
string and the SDK_MIN_VERSION constant.
"""

__version__ = "0.18.5"
__version__ = "0.19.0"
__platform_version__ = "1.0.0"
74 changes: 64 additions & 10 deletions src/nullrun/decorators.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ def researcher(q):
set_trace_id,
)
from nullrun.runtime import NullRunRuntime, get_runtime
from nullrun.transport import is_fallback_decision_source

# Sentinel used when a gate fires outside a workflow context.
UNKNOWN_WORKFLOW_ID = "__nullrun_unknown__"
Expand Down Expand Up @@ -634,6 +635,16 @@ def _protect_body(args: tuple[Any, ...], kwargs: dict[str, Any], unify_block: bo
# backend decides allow/block/require-approval.
_run_tool_policy_gate(runtime, fn, args, kwargs)

# 5. Drain any enforcement decision that a framework
# callback could not enforce. LangChain swallows
# exceptions raised from a callback handler, so a real
# block discovered in `on_llm_start` (budget exhausted,
# workflow KILL/PAUSE) cannot abort the LLM call from
# inside the callback. The callback stashes it instead;
# this boundary — which CAN abort — raises it. Runs after
# the gates so the primary decision always wins.
_raise_deferred_enforcement(fn.__name__)

yield runtime
except BaseException as exc: # noqa: BLE001
error = exc
Expand Down Expand Up @@ -737,6 +748,48 @@ def sync_wrapper(*args: Any, **kwargs: Any) -> Any:
return sync_wrapper # type: ignore[return-value]


def _raise_deferred_enforcement(tool_name: str) -> None:
"""Raise a gate decision a framework callback could not enforce.

LangChain discards exceptions raised from a callback handler — it
logs ``Error in <handler> callback`` and continues. So when
``NullRunCallback.on_llm_start`` calls ``check_workflow_budget``
and gets a real block (budget exhausted, workflow KILL/PAUSE), it
cannot abort the LLM call from inside the callback.

The callback stashes the decision instead; this runs at the
``@protect`` boundary, which can abort, and re-raises the oldest
unraised one.

The import is deliberately LAZY and failure-tolerant. The stashing
side lives in ``instrumentation.langgraph``, which is only imported
when LangChain instrumentation is in play; importing it here would
couple the core decorator path to the LangChain adapter. A missing
module simply means nothing was stashed.

Never invoked on the transport-fail-OPEN path: only REAL gate
decisions are stashed (see ``record_deferred_enforcement``).
"""
try:
from nullrun.instrumentation.langgraph import (
drain_deferred_enforcement,
)
except Exception: # noqa: BLE001 — optional adapter, never a hard dep
return
deferred = drain_deferred_enforcement()
if deferred is None:
return
logger.error(
"@protect for %r: raising enforcement decision deferred from a "
"framework callback (%s: %s) — the callback itself could not "
"abort the call.",
tool_name,
type(deferred).__name__,
deferred,
)
raise deferred


def _run_tool_policy_gate(
runtime: Any,
fn: Callable[..., Any],
Expand Down Expand Up @@ -908,16 +961,17 @@ def _run_tool_policy_gate(
# typed transport-error arms above are the canonical path.
if isinstance(result, dict):
decision_source = result.get("decision_source", "")
if isinstance(decision_source, str) and (
decision_source.startswith("FALLBACK_")
or decision_source
in {
TransportErrorSource.NETWORK_ERROR,
TransportErrorSource.GATEWAY_ERROR,
TransportErrorSource.BREAKER_OPEN,
TransportErrorSource.AUTH_ERROR,
}
):
# DEF-MP-TS12-ENF-01: this arm used to test
# `startswith("FALLBACK_")` — an UPPERCASE prefix that no code
# path in the transport produces, since the real value is
# `DecisionSource.FALLBACK == "fallback"`. The clause could
# therefore never fire, and it still carried
# `TransportErrorSource.AUTH_ERROR`, which the same predicate
# in `runtime.check_workflow_budget` had already had removed
# (Fix D). One shared definition now, in
# `transport.is_fallback_decision_source`, so the two copies
# cannot drift apart again.
if is_fallback_decision_source(decision_source):
if fail_open:
logger.warning(
f"tool policy gate for {fn.__name__!r} returned "
Expand Down
Loading
Loading