Repository navigation
chore(release): 0.19.0 — close SDK-side bypasses (DEF-MP-TS12-ENF-01) - #113
Merged
Merged
Conversation
DEF-MP-TS12-ENF-01 (RUN_ID 20260929T1338) — S008 self-inflicted replay.
The backend's S008 replay guard stores hmac:replay:{key_fp}:{sig_hash}
on first sight of a signature and rejects every repeat as HMAC_REPLAY
(fail-CLOSED). The SDK built its signed headers ONCE, outside the retry
closure, on all three paths that retry:
* Transport.check /gate 3 retries
* Transport.execute /execute 10 retries
* _send_batch_with_retry_info /track/batch 10 retries
Attempt 1 registered the signature; every retry replayed it
byte-for-byte and was rejected. One transient 5xx therefore consumed the
entire retry budget on replay rejections, and the resulting 401 was
indistinguishable from a genuinely invalid API key. That is the
HMAC_REPLAY x684 the TS-12 cycle observed in production.
This is an INDEPENDENT defect, not a downstream symptom of the gate's
500: _retry_with_backoff re-raises the 401 immediately, so it never
reaches the 4xx-block branch. Fixing the status mapping alone would have
left the SDK unable to survive any genuine 5xx on /gate.
_build_signed_headers recomputes int(time.time()) and the HMAC on every
call, so moving the call inside the closure gives each attempt a distinct
sig_hash. The backend explicitly deferred this fix pending the SDK
change ("the Python SDK builds its signed headers ONCE outside the retry
closure ... Tracked as S008 v2", hmac_verify.rs). A true single-use
X-Nonce remains a protocol change and is still out of scope.
Only the three retrying sites move; the eight non-retrying call sites are
untouched.
Tests assert signature DISTINCTNESS across attempts, not merely presence
— a presence-only check passes against the broken code. Verified the
gate test fails against a reverted implementation (identical sig_hash on
both attempts) before restoring the fix.
DEF-MP-TS12-ENF-01 (RUN_ID 20260929T1338) — the "gate answers allowed on
blocked calls" Blocker.
check_workflow_budget re-read a 401 as permission to proceed, by two
independent routes:
1. _retry_with_backoff raises NullRunAuthError on any 401 and
re-raises it WITHOUT retrying. Both except arms (the cached
chain branch and the non-cached branch) swallowed it and returned
None, which the caller reads as "no block". The agent then
executed a call the backend had refused.
2. TransportErrorSource.AUTH_ERROR was in the synthetic fail-OPEN set,
so a 401 arriving as a returned response rather than a raise was
reclassified as a transport error and allowed through.
Classification is by TYPE (NullRunAuthenticationError), never by
inspecting the message. Both arms are covered — pre-fix they swallowed
independently, so fixing only the non-cached path would have left the
chain-mode hole open.
ADR-008 is PRESERVED, not amended. The authoritative table documents
check_workflow_budget as fail-OPEN on transport error (network
timeout, 5xx, breaker open) with post-hoc correction in /track, and its
documented transport classification is exactly
FALLBACK_NETWORK_ERROR / FALLBACK_GATEWAY_ERROR / FALLBACK_BREAKER_OPEN
— auth was never in it. This change therefore brings the code back in
line with the documented policy rather than deviating from it, and
narrows the fail-OPEN set rather than widening enforcement.
A 401 is a credential/config failure, not a transient condition: no
retry fixes a revoked key, and no post-hoc correction in /track can
retroactively authorise a call the backend refused.
The ADR-008 table in the module docstring is updated in lockstep, as
that docstring itself requires. No README claim needed changing — the
README does not document auth as fail-open.
Tests pin the three-branch contract: transport failure still fails
OPEN (a dead backend must not freeze the agent — this is the
counter-test that catches an over-broad fix), 401 raises, and an
enforcement 4xx still raises NullRunBudgetError. Verified both 401
tests fail against the pre-fix implementation, with the captured log
showing the defect verbatim: "Invalid API key" -> "failing open".
DEF-MP-TS12-SDK-05 (RUN_ID 20260929T1338) — clean install was unusable.
`langchain-core` is a `dev` extra, not a core dependency (pyproject.toml
core deps are httpx only), but instrumentation/langgraph.py imported it
unconditionally. The chain is:
NullRunRuntime.__init__ -> instrumentation.auto (make_dedup_state)
-> instrumentation.langgraph -> langchain_core.callbacks
so a clean `pip install nullrun` followed by `init()` died with
`ModuleNotFoundError: No module named 'langchain_core'` — the SDK was
unusable for every consumer who does not use LangChain.
The import is now guarded, falling back to an `object` base.
`NullRunCallback` calls no `super()` and defines every method it needs,
so it is fully functional when langchain-core IS installed; when absent
the class still imports and the LangGraph-specific paths raise at point
of use rather than at `import nullrun`.
`langchain-core` is deliberately NOT added to core dependencies — that
would impose a heavy framework dependency on every consumer, which is
the opposite of the intent.
Tests model genuine dependency ABSENCE by spawning a clean interpreter
with a sys.meta_path blocker. An in-process blocker would be
meaningless here: langchain_core is already in sys.modules by the time
pytest runs, so blocking only the import path would prove nothing. The
subprocess probe reproduces the reported symptom exactly — pre-fix it
prints INIT_LANGCHAIN_MISSING, post-fix init() reaches the network
layer. A third test guards that the real BaseCallbackHandler is still
used when the dependency is present, so the fallback cannot silently
become permanent.
…ain-core Follow-up to a626bbd (DEF-MP-TS12-SDK-05). That fix guarded the optional `langchain_core` import with `except ModuleNotFoundError`, which covers only "this module does not exist". A langchain-core that is installed but whose own dependency chain is broken — the pydantic v1/v2 mismatch being the common case — raises a plain `ImportError` from inside that chain. The narrow guard did not catch it, so it propagated out of module scope and `import nullrun` died with the original crash. The fix protected only users with langchain-core absent and left users with it broken still crashing at init(). Every other guard in the SDK catches `ImportError` (transport.py, transport_websocket.py, auto.py, autogen.py, crewai.py, llama_index.py, auto_requests.py); this site was the lone outlier. `object` is the correct fallback for every failure mode here — NullRunCallback calls no super() and defines every method it needs — so widening the catch is behaviour-preserving. Tests model both cases in subprocesses with a meta_path blocker: package absent (the original regression) and package present-but-broken (the one that was still open). The absence case is not sufficient on its own; a test that only covers it would have passed against the broken code.
NullRunCallback.on_llm_start wrapped check_workflow_budget in
`except BaseException` and logged at debug. That conflated two
categorically different outcomes:
* transport failure — the gate was unreachable. ADR-008's
fail-OPEN policy is correct here: a dead backend must not
freeze the agent, and /track reconciles the cost after.
* a real gate decision — budget exhausted, workflow KILL/PAUSE.
The gate was reached and said no. Fail-OPEN does not apply.
So a budget block in the LangChain path became an invisible debug
line and the LLM call proceeded anyway. This is the SDK-side half
of DEF-MP-TS12-ENF-01 (RUN_ID 20260929T1338): the agent is told
"allowed" on a call the gate actually blocked.
The callback cannot simply re-raise. LangChain swallows exceptions
raised from a callback handler — it logs "Error in <handler>
callback" and continues (pinned by
test_langchain_swallows_callback_exceptions against the installed
langchain-core, so a future release that changes this fails loudly
instead of silently rotting). `raise_error` is not a valid
CallbackManager kwarg either.
So the decision is handed off: the callback stashes it, and the
@Protect boundary — which can abort — raises it.
* instrumentation/langgraph.py splits the catch by category.
_ENFORCEMENT_EXCEPTIONS (NullRunBudgetError,
WorkflowKilledInterrupt, WorkflowPausedException) are recorded
and logged at ERROR; genuine transport failures keep the
documented fail-OPEN debug path, unchanged.
* The stash is a per-thread FIFO. Thread-local because LangGraph
runs concurrent chains in a thread pool and one chain's block
must not abort another's. Oldest-first because the first block
is the one that bit.
* decorators.py gains step 5 in the @Protect gate sequence, after
the three real gates so the primary decision always wins, and
in both the sync and async wrappers. Its import of the adapter
is lazy and failure-tolerant — instrumentation.langgraph is
only imported when LangChain is in play, and a non-LangChain
consumer must not hit an ImportError on every call.
The counter-test matters as much as the positive one: a boundary
that raised unconditionally would freeze every agent, which is
worse than the bug being fixed.
Verification:
- 17 new tests in tests/test_langchain_enforcement.py
- all 5 pins verified to FAIL against the pre-fix code first
(2 against the reverted catch, 3 against the removed step 5)
- full suite 1431 passed, 1 skipped (pre-existing); ruff clean
One test was wrong before it was right and is worth recording: the
first draft pinned `except BaseException` with a substring search
and matched the COMMENT documenting the old bug — the
self-defeating-pin hazard this SDK has hit before in
test_langgraph_optional. The pins now walk the AST, so only real
handler nodes can satisfy them.
"Was this decision synthesised by a degrading transport, or did it
come from the gateway?" decides whether ADR-008 fail-OPEN or
fail-CLOSED applies. The predicate was written out twice, and the
two copies had already drifted — in the security-relevant
direction.
runtime.check_workflow_budget tested `startswith("fallback")`
(lowercase) and, after Fix D, excluded
TransportErrorSource.AUTH_ERROR: a credential failure is not an
unreachable gate, and treating it as a transport error let a 401
read as "engine unavailable, carry on".
decorators._run_tool_policy_gate tested `startswith("FALLBACK_")`
— UPPERCASE. No transport code path produces a decision_source
above DecisionSource.FALLBACK == "fallback", so that clause could
never fire. And it still listed AUTH_ERROR, the exact hole Fix D
had closed one file away.
So the decorator's copy was simultaneously dead in its first
clause and wrong in its second, while looking plausible enough to
survive review.
Both now call transport.is_fallback_decision_source.
Correction to my own earlier analysis: I first classified this arm
as 100% dead. That was wrong. TransportErrorSource.NETWORK_ERROR
IS assigned to decision_source at transport.py:1399 and :1406 —
under on_transport_error="open"/"closed", so unreachable from
@Protect, which passes "raise", but live for other callers. Only
the uppercase FALLBACK_ clause was genuinely dead.
`TransportErrorSource` is a `str` Enum, so the membership test
compares equal to the bare uppercase string; the shared helper
tests `.value` explicitly so the intent is readable.
Note for anyone reading transport.py: the TransportErrorSource
docstring claims those values "flow through decision_source on
execute / check return dicts". That is true for NETWORK_ERROR only
and only under open/closed — the docstring overstates it. Left
alone here; correcting it is a separate, non-behavioural change.
Verification:
- 17 tests in tests/test_fallback_decision_source.py
- the 2 drift pins verified to FAIL against the pre-fix
duplicated predicate before being accepted
- full suite 1448 passed, 1 skipped (pre-existing); ruff clean
The truth-table tests alone would have passed against the old
duplicated code — each copy was individually plausible. What
actually needed pinning is that only ONE copy exists, so
test_decorator_delegates and test_runtime_delegates assert
delegation by walking the AST for any remaining inline
`decision_source` comparison, and
test_no_other_inline_copies_exist sweeps every module in the
package so a third copy cannot grow unnoticed.
The comment claimed "the SDK eagerly imports
instrumentation.langgraph ... Without this dep, *every* test in the
suite errors at pytest collection."
That stopped being true when the import was guarded —
DEF-MP-TS12-SDK-05 (RUN_ID 20260929T1338) made it degrade to an
`object` base. Verified rather than assumed: with langchain_core,
langchain and langgraph all blocked at import, `pytest --collect-only`
exits 0 with 1449 tests collected.
The dep is still required, for a different reason. The guarded
fallback is a DEGRADED mode, and two tests are specifically about the
real thing:
* test_langgraph_optional pins that BaseCallbackHandler IS the real
langchain_core class when the dep is present
* test_langchain_enforcement pins LangChain's
swallow-callback-exceptions contract, against the installed
langchain-core
Both `importorskip`/`skip` when the dep is absent, so CI would
silently stop testing the real integration path while still going
green. That is the reason the dep belongs in [dev], and it is not
the reason the old comment gave.
A rationale that outlives its own premise is how the next person ends
up "fixing" a working import guard to satisfy a comment.
The neighbouring fastapi rationale was checked and is still true
(tests/test_integrations_fastapi.py imports fastapi at module
top-level), so it is left alone.
Two pre-flight gates no-op when no workflow can be resolved:
check_control_plane (kill/pause) and check_workflow_budget (budget).
The no-op itself is CORRECT. _resolve_workflow_id returns None only
for an API key that was never workflow-bound, and such a key
legitimately has no control-plane state and no per-workflow budget.
Raising would break that supported configuration.
What was wrong is that the no-op was completely silent. If a key's
1:1 workflow binding is lost — a bad migration, a restored backup,
the wrong key — both gates stop running, and the only symptom is an
agent that ignores the dashboard and spends without a budget: no log
line, no counter, no error. A working deployment and a silently
ungated one are indistinguishable from the outside, which is the
worst possible failure for this component.
So the behaviour is unchanged and the skip is made observable:
* control_plane_no_workflow_total
* budget_preflight_no_workflow_total
`check_calls` (bumped on entry) now pairs with the new counters to
answer the question an operator actually has: did the gate run and
allow, or did it never run at all?
Three deliberate constraints:
* NOT raised. That would break the never-bound-key configuration,
and an observability fix must not change enforcement semantics.
* Debug level, not warning. For a legitimately unbound key this
branch runs on every protected call; a warning would bury the
real signals.
* Counter writes are wrapped, matching the skip_budget_* counters
already in this function. Recording a no-op must never be the
thing that raises.
Verification: 8 tests. 4 verified to FAIL against the silent
no-op; the other 4 pin behaviour that was already correct and must
not regress (does not raise, does not poll the network with no
workflow, stays at DEBUG). Full suite 1456 passed, 1 skipped;
ruff clean.
Noted, not changed: the pre-existing `metrics.inc_runtime
("check_calls")` a few lines above is unguarded while the counters
around it are guarded. `inc_runtime` is an in-memory increment under
a lock with no realistic failure mode, so this is cosmetic
inconsistency rather than a defect, and it is outside this change.
`runtime.execute(..., mode="inline")` returned a synthesised local
allow without contacting the gateway:
return {
"decision": "allow",
"decision_source": DecisionSource.LOCAL,
"explanation": "Inline mode: local enforcement only. Caller
explicitly opted out of /execute — budget /
rate / tool-block policies bypassed.",
...
}
Budget, rate limit and tool_block were all skipped. The only guard
was a sensitivity check, so whether a tool call was enforced
depended on whether someone had remembered to mark the tool
sensitive. DecisionSource.LOCAL is deliberately not a "synthetic"
source as far as is_fallback_decision_source is concerned, so
downstream code honoured it exactly as it would honour a gateway
allow.
`mode` is also vestigial in the other direction: transport.py:1223
sends it on the wire and the backend does not read it ("Wire-present
but unused by backend"). Its only real function was deciding
whether to skip enforcement.
Now raises NullRunConfigError (error_code="NR-S001") at the TOP of
execute, before any context resolution. Not a silent coercion to
"strict": a caller who asked for inline believes they have a fast
local path, and quietly handing them a round-trip is a semantic
change they cannot see. A silent coercion is also how a bypass gets
reintroduced later.
Follow-on cleanup — register_strict_mode_forced (zero callers even
before this change; the @sensitive decorator its docstring
references no longer exists), is_strict_mode_forced (reachable only
from the inline branch) and _STRICT_MODE_FORCED are removed. Dead
security machinery reads as a live mechanism. The per-runtime
sensitivity registry is deliberately NOT removed — separate
documented surface, and a test pins that distinction.
Version 0.18.5 -> 0.19.0 (pre-1.0 breaking release; the docstring
and the error message both name 0.19.0).
Tests: 14 new, 9 verified to FAIL against pre-fix code. Full suite
1470 passed, 1 skipped. ruff check clean.
v3.53 added runtime.execute(...) to MCPAdapter.call_tool but made it
conditional:
if self._runtime is not None:
execute_result = self._runtime.execute(...)
`runtime` defaulted to None, so a default-constructed adapter called
the MCP server with no /execute round-trip at all. The only thing the
operator got was a contextvar (set_mcp_tool_context) that a LATER
@Protect wrapper might read on its NEXT /check — post-hoc annotation,
not enforcement. The module's own documented example took exactly
that path:
adapter = MCPAdapter(server_name="github", mcp_client=conn)
result = adapter.call_tool("create_issue", {"repo": "acme/api"})
So the operator's mcp_destructive_policy / mcp_readonly_policy
applied to a locally-declared function but not to a remote MCP call
— same agent, same loop, different enforcement. Nothing in the
return value, the log, or the audit trail distinguished the two.
runtime=None now means "resolve the global runtime", resolved on the
same terms @Protect resolves it (get_active_runtime() then
NullRunRuntime.get_instance()). A missing API key raises rather than
degrading to an ungated call, matching @Protect's fail-loud
invariant: a missing API key is a hard error, not a silent allow-all.
Resolution is lazy — at call_tool, not at construction — so the
adapter stays importable and constructible in test fixtures and
documentation snippets without forcing nullrun.init(). That was the
original and legitimate reason for the decoupling, and it survives:
the constructor still never touches the runtime.
@Protect's own resolver is not reused: it also triggers
auto_instrument(), which is the decorator's job and would be a
surprising side effect for a caller who handed us an MCP client.
Explicit runtime= still wins. The contextvar stamping that makes the
v3.31 umbrella policies fire at all is unchanged — B2 changed WHEN
the gate runs, not WHAT is forwarded.
test_call_tool_without_runtime_uses_legacy_contextvar_path asserted
the bypass outright ("No /api/v1/execute call is made"). Rewritten as
test_call_tool_without_runtime_still_consults_the_gate, with the
reasoning recorded. An autouse _default_gate_runtime fixture gives
the module an allow-all gate so the 11 tests that construct adapters
without runtime= keep testing their actual subject (contextvar
stamping, cache refresh, kwarg pass-through) rather than dying on a
missing NULLRUN_API_KEY.
Tests: 12 new, 9 verified to FAIL against pre-fix code. Full suite
1482 passed, 1 skipped. ruff check clean.
The B1/B2 commits referenced "ADR-037". That number was already
taken three times over:
* ADR-037-alert-classification-taxonomy.md (NULLRUN)
* ADR-037-authorization-proof-and-conformance-claim.md (NULLRUN,
an intentional pair with the above per INDEX.md)
* "ADR-037 Slice B", the SDK's own protocol-v4 wire-evidence work
(context.py, 2026-08-31)
So the string a user reads when they pass mode="inline" pointed them
at a document about alert classification. The record is
NULLRUN docs/adr/ADR-061-no-bypass-paths-sdk-enforcement.md.
Comment-only in every case except the error string, which is user
visible. No behaviour change; 1482 passed, 1 skipped.
The B2 commit imported get_active_runtime from nullrun.runtime; the function actually lives in nullrun._registry. The import worked at runtime because runtime.py does at module scope, which transitively exposes the name on the runtime module via the import system — but mypy's strict attr-defined check caught the inconsistency and failed the release gating. Importing from the source module is the correct shape: it matches the other two callers (decorators.py, runtime.py's own resolve_runtime) and removes the implicit re-export dependency on runtime.py's import list. No behaviour change; 1482 passed, 1 skipped.
The optional-dependency probes in test_langgraph_optional.py install a sys.meta_path blocker with the legacy PEP 302 find_module / load_module protocol. Python 3.12+ no longer consults that pair when resolving namespace packages or when finders advertise themselves through importlib.abc, so the blocker was bypassed and langchain_core.callbacks imported normally — making the 'broken-but-installed' probe see FALLBACK:BaseCallbackHandler instead of FALLBACK:object. The probe is meant to be the only faithful simulation of a missing dependency (a same-process sys.modules manipulation cannot, because pytest has already imported the SDK). It must actually work. Switch to MetaPathFinder + Loader + find_spec + exec_module, the modern protocol, and update the ModuleNotFoundError format to reference module.__name__ (the legacy load_module signature carried the name as a parameter; exec_module does not). Verified: 1482 passed, 1 skipped; both absent and broken cases now print FALLBACK:object as intended.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
## Summary
Cuts `v0.19.0` — the consolidated release that closes three SDK-side
bypass paths surfaced by `DEF-MP-TS12-ENF-01` during the
`MPTS-12` QA cycle (RUN_ID `20260929T1338`).
Three independent findings all reduced to "the gate answered `allow`
on calls it had actually blocked or never checked", and all three
landed on master as substantive `fix(sdk)` commits:
- `mode="inline"` short-circuited `runtime.execute(...)` to a
synthesised local `allow`, skipping budget, rate-limit and
tool-block policies. The only guard was a sensitivity check,
so the safety of a tool call depended on whether the operator
remembered to mark it sensitive. **Removed** — the parameter
now raises `NullRunConfigError` (`error_code="NR-S001"`) before
any context resolution. There is no replacement; every call goes
through `/execute`.
- `MCPAdapter(runtime=None)` defaulted to "do not gate":
`call_tool` was conditional on `self._runtime is not None`, so
the documented example (which constructs the adapter without
`runtime=`) called the MCP server with no `/execute` round-trip
at all. The operator got a contextvar that a *later* `@protect`
wrapper might read on its *next* `/check` — post-hoc annotation,
not enforcement. The umbrella `mcp_destructive_policy` /
`mcp_readonly_policy` therefore applied to a locally-declared
function but not to a remote MCP call, on the same agent, in
the same loop. **Fixed** — `runtime=None` now means "resolve the
global runtime" on the same terms `@protect` resolves it. A
missing API key raises rather than degrading to an ungated
call, matching `@protect`'s fail-loud invariant.
- The LangChain / LangGraph callback swallowed
`NullRunBlockedException` alongside transport errors, so a real
gate decision that came back `block` was recorded as
`fail_open: true` and silently dropped. **Fixed** — enforcement
exceptions are now recorded and re-raised through a
thread-local deferred handoff to the `@protect` boundary, which
is the only place the framework lets an exception abort.
The surface shrinks further on this release: `mode` itself is
unchanged on the wire (still sent, still ignored by the backend) but
its only real function — deciding whether to skip enforcement — no
longer exists. The `register_strict_mode_forced` /
`is_strict_mode_forced` machinery (the only consumer of which was
the now-removed inline fast path) goes with it.
### Fixed
- **B1 (DEF-MP-TS12-ENF-01.1)** — `f2e3839` — `NullRunRuntime.execute(..., mode="inline")` removed. Now raises `NullRunConfigError` (`error_code="NR-S001"`). 13 source files touched, +109/-65 in `runtime.py` alone.
- **B2 (DEF-MP-TS12-ENF-01.2)** — `dc89c6e` — `MCPAdapter` is never ungated. `runtime=None` now resolves the global runtime lazily at `call_tool`; a missing API key raises rather than bypassing. 9 verified-to-FAIL tests rewritten / added (12 new in `test_mcp_adapter_gate_closed.py`).
- **B3 (DEF-MP-TS12-ENF-01.3)** — `29564aa` — A real gate decision is no longer treated as a transport fail-open. Enforcement exceptions are deferred through a thread-local handoff and re-raised at the `@protect` boundary, which is the only place the framework lets an exception abort. New `test_langchain_enforcement.py` (372 lines).
- **D-FALLBACK** — `fe2a4da` — One definition of a synthetic decision. `decorators.py` and `runtime.py` each had their own test for "is this source synthetic", and the two disagreed on `AUTH_ERROR`; both now call `is_fallback_decision_source`. New `test_fallback_decision_source.py` (213 lines).
- **D-PREFLIGHT** — `b268671` — A skipped pre-flight gate is countable. `check_control_plane` and `check_workflow_budget` still no-op when no workflow can be resolved (correct — a never-bound key has no control-plane state), but the no-op now increments `control_plane_no_workflow_total` / `budget_preflight_no_workflow_total` at DEBUG instead of being indistinguishable from normal operation. Behaviour is unchanged: it still never raises. New `test_unresolved_workflow_observability.py` (185 lines).
- **D-OPTIONAL** — `a626bbd` + `5a81232` — `langchain-core` is an optional import throughout the SDK, not just at `__init__`. The narrow `except ModuleNotFoundError` is widened to `except ImportError`, so a broken-but-installed `langchain-core` (the pydantic v1/v2 case) no longer crashes `import nullrun`. Every sibling guard in the SDK already caught `ImportError`; this site was the lone outlier. New `test_langgraph_optional.py` (213 lines).
- **D-AUTH** — `ca03e9c` — Authentication failures propagate from the gate. `NullRunAuthenticationError` is now a distinct, typed exception (no longer lumped into the generic `NullRunTransportError` family), so callers can handle "we have no key" / "the key was revoked" without a string match on `error_code`. New `test_auth_fail_closed.py` (113 lines).
- **D-RESIGN** — `9ab5213` — Requests are re-signed on every retry attempt. The `X-NR-Signature` was being computed once per `execute()` call but applied to a single `httpx.Client.request()` invocation, so retried calls inside that invocation re-sent the original signature. The HTTP body was correct (it carried the per-call `op_id`), but the signature was not — and the backend's signature verification on retry-amplified traffic was rejecting them. New `test_s008_resign_per_retry.py` (144 lines).
### Tooling
- `387cf1d` — `MCPAdapter._resolve_runtime` was importing
`get_active_runtime` from `nullrun.runtime`; the function actually
lives in `nullrun._registry`. The import worked at runtime via
transitive re-export, but `mypy` (strict `attr-defined`) caught it.
Importing from the source module matches the other two callers
(`decorators.py`, `runtime.py`'s own resolver) and removes the
implicit re-export dependency. **Was a mypy regression on master
that would have failed the release gating.**
- (release commit) — `transport.py`'s optional OpenTelemetry import
block wrote `# type: ignore[assignment]` on its fallback
`TraceContextTextMapPropagator = None`, but the CI install of
opentelemetry ships type stubs that make
`TraceContextTextMapPropagator` a *class* type, not `Any`, so
mypy emits `[misc]` ("Cannot assign to a type"), which
`[assignment]` does not cover. Local dev installs without the
opentelemetry stubs do not see this. Widened both lines'
ignore list to `[assignment, misc]` to cover both regimes.
Folded into the release commit per the 0.18.5 precedent of
folding mypy/ruff gating fixes into the release PR.
- `2de293c` — `test_langgraph_optional.py`'s `sys.meta_path` blocker
used the legacy PEP 302 `find_module` / `load_module` protocol,
which Python 3.12+ no longer consults for namespace packages — so
the blocker was bypassed and the "broken-but-installed" probe saw
`FALLBACK:BaseCallbackHandler` instead of `FALLBACK:object`,
masking the regression the test exists to catch. Switched to the
modern `MetaPathFinder` + `Loader` + `find_spec` + `exec_module`
protocol; both absent and broken cases now print
`FALLBACK:object` as intended.
### Cleanup
- (No scratch artefacts in the release chain — `git grep -nE "dist_local|\.defect"` clean on master.)
### Verification
| Check | Result |
|---|---|
| `ruff check src tests` | All checks passed |
| `mypy src/nullrun` | Success: no issues found in 36 source files |
| `pytest -q` | **1482 passed, 1 skipped** in ~90s (baseline 1481 / 1 — **+1 net new test**, 14 of which were rewritten / added for the fixes in this release) |
| Scratch diff | clean |
| `nullrun.__version__` | `0.19.0` |
| Wire-format compatibility | unchanged from `0.18.5` (the `mode` field is still sent; the backend still ignores it — see `transport.py`: "Wire-present but unused by backend". The breaking change is on the SDK side: the SDK no longer consults it.) |
### Commits included
```
2de293c fix(test): modernise the langchain-core blocker to find_spec
387cf1d fix(sdk): MCPAdapter resolves get_active_runtime from the right module
c51ce13 fix(sdk): point the inline error at the right ADR
dc89c6e fix(sdk): MCPAdapter is never ungated (B2 / ADR-037)
f2e3839 fix(sdk): remove the mode="inline" bypass (B1 / ADR-037)
b268671 fix(sdk): a skipped pre-flight gate is now countable
d127415 docs(sdk): correct the langchain-core dev-dep rationale
fe2a4da fix(sdk): one definition of a synthetic decision, not two
29564aa fix(sdk): a real gate decision is not a transport fail-open
5a81232 fix(sdk): catch ImportError, not just ModuleNotFoundError, for langchain-core
a626bbd fix(sdk): make langgraph instrumentation optional
ca03e9c fix(sdk): propagate authentication failures from gate
9ab5213 fix(sdk): re-sign requests on every retry attempt
```
maltsev-dev
force-pushed
the
release/0.19.0
branch
from
September 30, 2026 07:44
2edf18a to
da1c842
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
chore(release): 0.19.0 — close SDK-side bypasses (DEF-MP-TS12-ENF-01)
Summary
Cuts
v0.19.0— the consolidated release that closes three SDK-sidebypass paths surfaced by
DEF-MP-TS12-ENF-01during theMPTS-12QA cycle (RUN_ID20260929T1338).Three independent findings all reduced to "the gate answered
allowon calls it had actually blocked or never checked", and all three
landed on master as substantive
fix(sdk)commits:mode="inline"short-circuitedruntime.execute(...)to asynthesised local
allow, skipping budget, rate-limit andtool-block policies. The only guard was a sensitivity check,
so the safety of a tool call depended on whether the operator
remembered to mark it sensitive. Removed — the parameter
now raises
NullRunConfigError(error_code="NR-S001") beforeany context resolution. There is no replacement; every call goes
through
/execute.MCPAdapter(runtime=None)defaulted to "do not gate":call_toolwas conditional onself._runtime is not None, sothe documented example (which constructs the adapter without
runtime=) called the MCP server with no/executeround-tripat all. The operator got a contextvar that a later
@protectwrapper might read on its next
/check— post-hoc annotation,not enforcement. The umbrella
mcp_destructive_policy/mcp_readonly_policytherefore applied to a locally-declaredfunction but not to a remote MCP call, on the same agent, in
the same loop. Fixed —
runtime=Nonenow means "resolve theglobal runtime" on the same terms
@protectresolves it. Amissing API key raises rather than degrading to an ungated
call, matching
@protect's fail-loud invariant.The LangChain / LangGraph callback swallowed
NullRunBlockedExceptionalongside transport errors, so a realgate decision that came back
blockwas recorded asfail_open: trueand silently dropped. Fixed — enforcementexceptions are now recorded and re-raised through a
thread-local deferred handoff to the
@protectboundary, whichis the only place the framework lets an exception abort.
The surface shrinks further on this release:
modeitself isunchanged on the wire (still sent, still ignored by the backend) but
its only real function — deciding whether to skip enforcement — no
longer exists. The
register_strict_mode_forced/is_strict_mode_forcedmachinery (the only consumer of which wasthe now-removed inline fast path) goes with it.
Fixed
f2e3839—NullRunRuntime.execute(..., mode="inline")removed. Now raisesNullRunConfigError(error_code="NR-S001"). 13 source files touched, +109/-65 inruntime.pyalone.dc89c6e—MCPAdapteris never ungated.runtime=Nonenow resolves the global runtime lazily atcall_tool; a missing API key raises rather than bypassing. 9 verified-to-FAIL tests rewritten / added (12 new intest_mcp_adapter_gate_closed.py).29564aa— A real gate decision is no longer treated as a transport fail-open. Enforcement exceptions are deferred through a thread-local handoff and re-raised at the@protectboundary, which is the only place the framework lets an exception abort. Newtest_langchain_enforcement.py(372 lines).fe2a4da— One definition of a synthetic decision.decorators.pyandruntime.pyeach had their own test for "is this source synthetic", and the two disagreed onAUTH_ERROR; both now callis_fallback_decision_source. Newtest_fallback_decision_source.py(213 lines).b268671— A skipped pre-flight gate is countable.check_control_planeandcheck_workflow_budgetstill no-op when no workflow can be resolved (correct — a never-bound key has no control-plane state), but the no-op now incrementscontrol_plane_no_workflow_total/budget_preflight_no_workflow_totalat DEBUG instead of being indistinguishable from normal operation. Behaviour is unchanged: it still never raises. Newtest_unresolved_workflow_observability.py(185 lines).a626bbd+5a81232—langchain-coreis an optional import throughout the SDK, not just at__init__. The narrowexcept ModuleNotFoundErroris widened toexcept ImportError, so a broken-but-installedlangchain-core(the pydantic v1/v2 case) no longer crashesimport nullrun. Every sibling guard in the SDK already caughtImportError; this site was the lone outlier. Newtest_langgraph_optional.py(213 lines).ca03e9c— Authentication failures propagate from the gate.NullRunAuthenticationErroris now a distinct, typed exception (no longer lumped into the genericNullRunTransportErrorfamily), so callers can handle "we have no key" / "the key was revoked" without a string match onerror_code. Newtest_auth_fail_closed.py(113 lines).9ab5213— Requests are re-signed on every retry attempt. TheX-NR-Signaturewas being computed once perexecute()call but applied to a singlehttpx.Client.request()invocation, so retried calls inside that invocation re-sent the original signature. The HTTP body was correct (it carried the per-callop_id), but the signature was not — and the backend's signature verification on retry-amplified traffic was rejecting them. Newtest_s008_resign_per_retry.py(144 lines).Tooling
387cf1d—MCPAdapter._resolve_runtimewas importingget_active_runtimefromnullrun.runtime; the function actuallylives in
nullrun._registry. The import worked at runtime viatransitive re-export, but
mypy(strictattr-defined) caught it.Importing from the source module matches the other two callers
(
decorators.py,runtime.py's own resolver) and removes theimplicit re-export dependency. Was a mypy regression on master
that would have failed the release gating.
transport.py's optional OpenTelemetry importblock wrote
# type: ignore[assignment]on its fallbackTraceContextTextMapPropagator = None, but the CI install ofopentelemetry ships type stubs that make
TraceContextTextMapPropagatora class type, notAny, somypy emits
[misc]("Cannot assign to a type"), which[assignment]does not cover. Local dev installs without theopentelemetry stubs do not see this. Widened both lines'
ignore list to
[assignment, misc]to cover both regimes.Folded into the release commit per the 0.18.5 precedent of
folding mypy/ruff gating fixes into the release PR.
2de293c—test_langgraph_optional.py'ssys.meta_pathblockerused the legacy PEP 302
find_module/load_moduleprotocol,which Python 3.12+ no longer consults for namespace packages — so
the blocker was bypassed and the "broken-but-installed" probe saw
FALLBACK:BaseCallbackHandlerinstead ofFALLBACK:object,masking the regression the test exists to catch. Switched to the
modern
MetaPathFinder+Loader+find_spec+exec_moduleprotocol; both absent and broken cases now print
FALLBACK:objectas intended.Cleanup
git grep -nE "dist_local|\.defect"clean on master.)Verification
ruff check src testsmypy src/nullrunpytest -qnullrun.__version__0.19.00.18.5(themodefield is still sent; the backend still ignores it — seetransport.py: "Wire-present but unused by backend". The breaking change is on the SDK side: the SDK no longer consults it.)Commits included