feat: implement specs/testing-spec.md — <Testing>, <Test>, assertions, xmd test

[taras]

Motivation

Executable documents can contain probabilistic behavior (LLM sampling, subprocesses, network). specs/testing-spec.md (added in this PR) grounds that behavior in observable results: <Testing>/<Test> directives, fourteen @std/assert-backed assertion components, a TestApi (testing operation), and an xmd test CLI command. None of it existed before this PR.

Approach

Testing is not an engine feature — it is a vocabulary extension composed around a single core execution entrypoint.

Core owns the only entrypoint, and testing composes middleware around it:

import { execute } from "@executablemd/core";
import { useTesting } from "@executablemd/testing";

const tests = yield* useTesting();
const execution = yield* execute(options);
const outcome = yield* execution;          // Result<string>
const results = yield* tests.results;

• Core exports the sole execute entrypoint, delivered through a new test-agnostic Execution context Api. Extensions decorate the execution lifecycle with Execution.around({ execute }) middleware — observing options, wrapping the handle, or mapping its completion — without introducing another execution function. Core has no testing flag and imports no testing concepts.
• DocumentExecution completes with Result<string>: Ok(output) on success, Err(error) on document, infrastructure, or testing failure. Once execute has returned a handle, completion never throws — the entire spawned setup and workflow sit in one error boundary that closes the output stream (with complete or partial rendered text) and resolves Err. Pre-handle failures may still throw. collect unwraps the Result for callers that want throw-on-failure ergonomics.
• useTesting() installs scope-local middleware and returns the results handle: the expansion vocabulary (via core's expandInvocation hook), result collection, root testing activation (Test.around({ testing: () => true }) — the same install <Testing> performs for a subtree, making xmd test ≡ root <Testing>), and a completion policy that converts an otherwise successful execution to Err(TestFailureError) after output closes when tests failed or none were discovered. A core Err passes through unchanged and results (an immutable discovery-order snapshot) stays available after failure. One session per execution scope and one execute() per session, both enforced; every install is removed with the session's Effection scope.
• Vocabulary registration is distinct from activation: installTestingVocabulary alone (used by xmd run) keeps testing mode off — <Test> bodies are skipped entirely, assertions stay usable (diagnostics with --verbose), and an explicit <Testing> boundary still activates its subtree and turns boundary failures or an empty boundary into an Err outcome.
• <Test> isolation: child Effection scope, a test-owned EvalScope lease (a dedicated suspended task inside the parent EvalScope, halted during test teardown so persisted middleware cannot leak into later tests), one stable binding environment merging caller-projected bindings under the current environment, a raise interceptor installed as outer instrumentation so every raised ErrorSegment (command failures, import failures, nested <Test>, <Output>-region errors) fails the current test while later tests run, a directly-yielded 20s timebox (DI seam createTestHandlers({ timeoutMs }) for fast tests), and setup/body/teardown failure classification. A completed test returns only text segments — containment survives throwing ambient raise policies.
• Assert first, format after: assertions run on live values (expressions evaluated over projected+current bindings — no JSON round-trip, so RegExp/undefined/object identity survive; msg is type-checked as a string, never formatted); diagnostics are built only after the outcome is fixed, under guarded fallback.
• Durable testing records preserve replay outcomes: every completed TestResult and every explicit <Testing> boundary outcome is journaled as a testing-owned durable operation (test_result, testing_boundary) during expansion, before the root Close — identified deterministically by source position. On a confirmed full replay (the journal already holds a root Close, so the document is not re-expanded and no effects rerun) the stored records are restored from ExecuteOptions.stream into the current collectors before the completion policy applies, so passing, failing, and zero-test outcomes — root sessions and explicit boundaries alike — replay exactly as they ran live. Live and partial journals hydrate nothing; re-expansion records each result exactly once, in discovery order, with completed records replaying in place. Payloads are validated with type guards on read.
• Core support fixes: ComponentInvocation carries deterministic original-file source positions (frontmatter included; body start computed as a verbatim suffix, never content search), and execution output is a replay-safe stream (retained history + one queue per subscriber) so late and repeated subscribers receive every chunk and the close value.
• Self-testing smoke suite: the smoke guide is composed from feature documents under smoke-test/Guide/. Guide prose stays outside tests; each of the 36 atomic <Test> elements owns its complete scenario — setup, action, capture, and assertion inside the same test, connected directly to the captured actual value (AssertEquals/AssertStrictEquals for deterministic outputs and live bindings, anchored AssertMatch for genuinely dynamic values, AssertNotMatch only for deliberately suppressed content). One behavioral outcome per test; no test depends on a sibling's bindings or effects; lifecycle-dependent scenarios like the daemon test contain startup, readiness, and assertion so teardown is bounded by the atomic test scope. Regular xmd run renders the prose guide (test examples are intentionally skipped); xmd test runs every scenario. testing/tests/smoke.test.ts is infrastructure only: useTesting() around core execute(), the 36 exact test names asserted in discovery order, journal size host-side, and a fresh-session full replay asserted identical with zero appended events. (The nested-Instruction accumulation demo is omitted: the engine's cycle detection rejects Instruction within Instruction, so composition cannot be demonstrated in-document.)
• CLI: xmd run registers the vocabulary around execute; xmd test composes useTesting() around the same execute call. Both inspect the returned Result for exit status (0 on Ok, 1 on any Err).

Scope confirmation

• All changed files relate to the stated purpose
• No drive-by refactors or "while I'm here" cleanups
• No formatting changes mixed with functional changes

New abstractions (if any)

• Each new type/interface/class has 3+ consumers or justification
• No speculative features ("we might need this later")

Execution/ExecutionApi (core execution middleware hook), InvocationContext/InvocationHandling (core expansion hook contract), TestApi/TestResult/BoundaryOutcome/Testing (spec-mandated), ReplayStream (fixes chunk loss for any consumer), SourcePosition (spec-mandated test identity).

New dependencies (if any)

• Package: @std/assert (jsr, already resolved in deno.lock; npm alias @jsr/std__assert via the @jsr scope registry added to .npmrc/bunfig.toml)
• Justification: specs/testing-spec.md mandates assertion components use @std/assert semantics and error types.

New tests (if any)

• core/tests/expand-invocation.test.ts (6): fall-through, claim, handled-empty vs unhandled, raise policy, context passing
• core/tests/source-position.test.ts (6): local/origin translation, frontmatter offset with a body-line copy inside frontmatter (defeats content-search false matches), imported components
• core/tests/replay-stream.test.ts (6): replay-before-subscribe, replay/live boundary exactly-once, multiple subscribers, subscribe-after-close, post-close ignore
• testing/tests/assertions.test.ts (17): visibility rules in/out of testing mode and with --verbose, abort semantics outside tests, expected-children Capture parity + XOR validation, per-kind prop validation, RegExp requirement, live/projected bindings, string-only msg type check, assert-then-format regressions
• testing/tests/testing-mode.test.ts (19): skip-with-zero-side-effects, explicit <Testing> under regular runs, root activation, zero-test failure, nested boundary delegation, ErrorSegment failures (exec/import/nested-test/<Output>-region), containment in throwing-policy regions, projected <Test> bodies reading caller bindings, binding + effect isolation across tests (lease teardown), partial-output-before-failure ordering, timeout halt + continue, source-location identity
• testing/tests/use-testing.test.ts (10): session composition + results snapshot, results after failure, core Err preserved, one execute() per session, one session per scope, middleware removed with the scope, Err-not-throw completion, pre-handle throw path, late subscriber integrity, early caller-scope halt teardown
• testing/tests/replay.test.ts (6): passing/failing/zero-test useTesting() live + full replay, pass/fail/empty explicit <Testing> boundaries on replay, partial replay without duplicated results, results available after replayed failure, unchanged output with zero re-run effects
• testing/tests/cli.test.ts (4): xmd test exit 0/1 with piped report, zero-test exit 1, xmd run skip + exit 0
• testing/tests/smoke.test.ts (1): the smoke guide's 36 embedded atomic tests pass in discovery order, live and on full replay, with identical Result/TestResult snapshots and no appended journal events

Release notes

@executablemd/testing starts at the lockstep version 0.3.1. The publish workflow was regenerated (deno task gen:publish-workflow); testing publishes after core. Manual npm/JSR bootstrap (release-process-spec §6 steps 2–4) is a maintainer step outside this PR: first npm publish by hand, trusted publisher config, and JSR package creation. Follow-up tracked separately: the local/no-section-divider-comments oxlint rule (#109).

[github-actions]

Found 12 redundant comments. Inline suggestions to remove them below.

[github-actions]

PR #108: feat: implement specs/testing-spec.md — , , assertions, xmd test

87 files, +4477 / -1130

Scope

🔴 PR has 5607 lines changed. Split into focused PRs.

🟡 5607 lines changed. PRs under 400 receive more thorough review.

🟡 87 files changed. Are all changes related?

🟡 PR mixes config and source changes.

🟡 New abstraction files: testing/tests/helpers.ts. Verify 3+ consumers.

Structural

🟡 2 console statements.

Slop

• core/src/execute.ts:65 — // execute options (spec §7.1)
• core/src/execute.ts:359 — // execute (spec §7.1)
• testing/src/handlers.ts:107 — // deno-lint-ignore require-yield
• testing/src/journal.ts:27 — // deno-lint-ignore require-yield
• testing/src/journal.ts:45 — // deno-lint-ignore require-yield
• testing/src/test-api.ts:60 — // deno-lint-ignore require-yield
• testing/src/test-api.ts:62 — // deno-lint-ignore require-yield
• testing/src/test-api.ts:66 — // deno-lint-ignore require-yield
• testing/src/use-testing.ts:41 — // deno-lint-ignore require-yield

Static Analysis

✅ Oxlint found no issues.

Correctness

No extraneous code patterns detected.

[taras]

Re the 12 'redundant comment' suggestions: 10 of the flagged lines are deno-lint-ignore require-yield suppression directives (the same pattern used throughout core/src/component-api.ts — one flagged instance is pre-existing code), not prose comments. The remaining two state spec constraints the code cannot show (the no-summary rule for <Testing> reports and the skip guarantee for <Test> in regular execution). Leaving them in place per Code Rule 4.

[github-actions]

Found 6 redundant comments. Inline suggestions to remove them below.

[github-actions]

Found 2 redundant comments. Inline suggestions to remove them below.

[github-actions]

Found 1 redundant comment. Inline suggestions to remove them below.

[github-actions]

Found 3 redundant comments. Inline suggestions to remove them below.

[github-actions]

Found 11 redundant comments. Inline suggestions to remove them below.

[github-actions]

Found 6 redundant comments. Inline suggestions to remove them below.

[taras]

Triage of the b3cf9e3/8ce9430 review passes:

• 10 of 14 flagged lines are deno-lint-ignore require-yield suppression directives (same recurring false positive as before) — kept.
• 3 are mid-comment tail lines of multi-line explanations (containment invariant in handlers.ts, the source-position invariant check in execute.ts, root-activation equivalence in use-testing.ts); applying the single-line suggestions would corrupt load-bearing comments — kept.
• 1 (execute.ts:359 // execute (spec §7.1)) is a genuine decorative section header, but the repo-wide divider sweep was intentionally reverted and is tracked by Enforce Code Rule 10: local/no-section-divider-comments oxlint rule with autofix #109 — removing one header piecemeal would leave the codebase inconsistent, so deferring to that issue.
• Scope notes: testing/tests/helpers.ts has four consumers (assertions, testing-mode, use-testing, replay suites); the two console statements are cli.ts's intended stderr error paths, allow-listed in .oxlintrc.

[github-actions]

Found 9 redundant comments. Inline suggestions to remove them below.