feat(contracts): add native Copilot leaf execution - #2837
feat(contracts): add native Copilot leaf execution#2837Daniel Meppiel (danielmeppiel) wants to merge 1 commit into
Conversation
Add strict local Markdown contracts with offline planning, explicit native-advisory execution, exact artifacts, independent checks, and durable records. Preserve legacy script dispatch and document the bounded first-run and installed-skill journeys. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The contract baseline Git invocation bypasses the repo’s trusted git executable resolver, and there are a couple of small correctness/efficiency issues that should be fixed before merging.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (3)
| Severity | Finding |
|---|---|
src/apm_cli/contracts/process.py — local_git() resolves git via shutil.which(), which bypasses the repo’s trusted executable… |
|
src/apm_cli/utils/yaml_io.py — loads_frontmatter_document() raises "Frontmatter must be a nonempty mapping." but currently… |
|
src/apm_cli/commands/contracts.py — workspace.inspect_workspace(plan) is executed even when planning is false, but the resulting… |
What changed in this PR
Introduces the initial “Contracts v0.1” slice: an explicit, local .contract.md execution path that can be planned offline, run via a native Copilot harness under explicit advisory consent, and assessed via fixed checks with durable local records—without changing legacy script dispatch.
Changes:
- Add contract planning/execution pipeline (
contracts/*) with strict admission, exact-byte baseline/artifact capture, independent per-check workspaces, and atomicrecord.jsonlifecycle. - Add Copilot native contract adapter (
CopilotRuntime.build_contract_request) with supervised MCP inventory and narrowly scoped tool permissions. - Add supporting CLI surface (
apm plan,apm run --on copilot), architecture guards, examples/docs, and a dedicated unit test suite (including real POSIX PTY/pipe coverage).
| File | Description |
|---|---|
| tests/unit/contracts/test_terminal_pty.py | Real PTY/pipe regression coverage for streaming, cancellation, and terminal restoration. |
| tests/unit/contracts/test_reliability.py | Exact-byte baseline/artifact integrity, process supervision, durable record behavior, and failure-mode tests. |
| tests/unit/contracts/test_pty_lifecycle.py | End-to-end Click boundary + PTY interrupt handling using a deterministic fake copilot binary. |
| tests/unit/contracts/test_imports.py | Import selection/identity rules for installed skills, including local vs git-backed integrity behavior. |
| tests/unit/contracts/test_frontend.py | Strict contract parsing/planning refusals, bounded YAML/source diagnostics, and platform gating. |
| tests/unit/contracts/test_example_checks.py | Validates the example checker protocol and fixture frontmatter arguments. |
| tests/unit/contracts/test_engine.py | Contract engine integration tests for capture/check sequencing, outcome precedence, and deadlines. |
| tests/unit/contracts/test_dispatch.py | Ensures --on explicitly selects contract mode and disables legacy fallbacks/update probes. |
| tests/unit/contracts/test_contract_runtime.py | Validates Copilot contract request construction, tool grants, and MCP inventory refusal behavior. |
| tests/unit/contracts/test_contract_policy.py | Ensures contract prerequisite policy discovery is conservative and read-only. |
| tests/unit/contracts/test_cli_lifecycle.py | Confirms plan/run lifecycle properties (no native on plan; consent gating; preflight cancellation semantics). |
| tests/unit/contracts/test_architecture.py | Architecture-linter regression traps for owner bypasses in contracts. |
| src/apm_cli/utils/yaml_io.py | Adds strict frontmatter document parsing with bounded YAML, source locations, and file-identity checks. |
| src/apm_cli/utils/console.py | Improves NO_COLOR behavior and adds controlled plain output + broken-pipe propagation for contract logging. |
| src/apm_cli/utils/atomic_io.py | Adds optional durability (fsync + directory sync on POSIX) to atomic writes. |
| src/apm_cli/runtime/registry.py | Adds supports_contracts capability flag to runtime descriptors. |
| src/apm_cli/runtime/copilot_runtime.py | Implements native Copilot contract invocation with supervised MCP inventory and narrow tool permissions. |
| src/apm_cli/runtime/base.py | Adds default build_contract_request() contract-support boundary on runtime adapters. |
| src/apm_cli/policy/discovery.py | Adds discover_contract_policy() entrypoint routing to the contract prerequisite logic. |
| src/apm_cli/policy/contract_prerequisite.py | Implements offline “positive no-policy” prerequisite check used by contract planning. |
| src/apm_cli/contracts/__init__.py | Introduces contracts package. |
| src/apm_cli/contracts/models.py | Defines contract plan/run data model, outcomes, and observation types. |
| src/apm_cli/contracts/events.py | Provides ordered internal event emission for contract runs. |
| src/apm_cli/contracts/frontend.py | Strict .contract.md parsing and read-only planning/admission. |
| src/apm_cli/contracts/imports.py | Read-only resolution of at most one directly declared, installed self-contained skill import. |
| src/apm_cli/contracts/workspace.py | Exact-byte capture of effective baseline + artifact capture + per-check isolated workspaces. |
| src/apm_cli/contracts/process.py | POSIX process-group supervision and bounded local git operations for baseline behavior. |
| src/apm_cli/contracts/records.py | Check normalization, outcome reduction, and atomic private run record management. |
| src/apm_cli/contracts/engine.py | Orchestrates admit → execute → capture → check → record for one contract run. |
| src/apm_cli/commands/run.py | Extends apm run with explicit --on contract dispatch + contract-only flags. |
| src/apm_cli/commands/plan.py | Adds apm plan command for read-only contract planning. |
| src/apm_cli/commands/contracts.py | Shared contract command boundary and error handling. |
| src/apm_cli/cli.py | Registers plan command and avoids update checks for contract plan/dispatch. |
| scripts/architecture_linter/checks/contract_leaf_runtime.py | Adds architecture rule preventing legacy fallback and split-authority patterns in contracts. |
| scripts/architecture_linter/groups/contracts_tests.py | Includes new contract leaf runtime rule group in contracts test guards. |
| .apm/architecture/owners/contracts-tooling.json | Registers canonical owners for contract source/subject/process/outcome decisions. |
| .gitignore | Ignores local .apm/runs/ run state. |
| .apm/docs-index.yml | Adds docs index entries for contract pages and updated CLI references. |
| docs/astro.config.mjs | Adds “Run a contract” to the consumer navigation. |
| docs/src/content/docs/reference/cli/run.md | Documents contract execution mode, prerequisites, and outcomes alongside legacy scripts. |
| docs/src/content/docs/reference/cli/plan.md | Adds reference docs for offline contract planning. |
| docs/src/content/docs/consumer/run-contracts.md | Adds consumer walkthrough for running the provided contract fixtures. |
| packages/apm-guide/.apm/skills/apm-usage/commands.md | Updates APM usage guidance for apm plan/contract mode flags. |
| packages/apm-guide/.apm/skills/apm-usage/package-authoring.md | Documents the bounded .contract.md source format and constraints. |
| examples/contracts/README.md | Adds example instructions for first contract and skill reuse scenarios. |
| examples/contracts/first-contract/apm.yml | Dependency-free fixture manifest. |
| examples/contracts/first-contract/notes.md | Fixture input notes for handoff generation. |
| examples/contracts/first-contract/handoff.contract.md | Minimal contract fixture producing handoff.json. |
| examples/contracts/first-contract/checks/check_handoff.py | Standard-library-only checker for fixture validation. |
| examples/contracts/handoff-style/apm.yml | Self-contained imported skill manifest for reuse fixture. |
| examples/contracts/handoff-style/SKILL.md | Skill content used as imported context in contract runs. |
| examples/contracts/reuse-contract/apm.yml | Fixture manifest declaring local skill dependency. |
| examples/contracts/reuse-contract/handoff.contract.md | Contract fixture demonstrating installed-skill reuse and parameterized check. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| executable = shutil.which("git") | ||
| if executable is None: | ||
| raise ContractError( | ||
| "Git is required for captured assessment workspaces.", code="git_missing" | ||
| ) |
| metadata = loader.get_single_data() | ||
| if not isinstance(metadata, dict): | ||
| raise FrontmatterSourceError("Frontmatter must be a nonempty mapping.", line=2) | ||
| locations = dict(loader.locations) |
| plan = frontend.plan_contract( | ||
| Path(contract), | ||
| Path.cwd(), | ||
| harness=harness, | ||
| model=model, | ||
| ) | ||
| inventory = workspace.inspect_workspace(plan) | ||
| if planning: | ||
| logger.render_plan(plan, inventory) | ||
| return |
|
Archival disposition approved by the maintainer: closing this APM PR as exploratory execution work that likely belongs under separate ASF ownership. This is a preservation and handoff decision, not a claim that an ASF replacement has shipped; no replacement repository or product is being created by this action. Preserved implementation: commit |



add(contracts): execute and assess local Markdown leaves
TL;DR
Add the first APM Contracts v0.1 slice: inspect a Markdown/frontmatter contract offline, run it through native Copilot, capture one artifact, assess those exact bytes, and retain a local result. Contracts can reuse one installed self-contained skill through existing APM dependency and lock machinery, without turning imports into invocation or authority. Explicit
--on copilotselects the new path; existing script dispatch remains unchanged.Important
This is a bounded, native-advisory capability—not a sandbox, general workflow engine, spending guarantee, or merge approval. “v0.1” names Contracts; APM's release version remains 0.29.1.
Maintainer-requested feature and documentation; no linked issue or CHANGELOG entry.
Problem (WHY)
Approach (WHAT)
needs, one scalarproduces, 1–8 named command checks, and optionally one installed skill.--allow-advisoryfor execution in terminals and pipes.--onselects contract mode even when a script has the same name. Without it, barerun,start, named scripts, prompt fallback, parameters, and existing harness behavior remain on the legacy path. Contract-only flags cannot silently modify scripts; contract mode rejects--param.Implementation (HOW)
cli.py,commands/{run,plan,contracts}.py: explicit dispatch, dedicated errors, no contract update probe, and no legacy fallback after contract failure.contracts/{__init__,frontend,imports}.py,utils/yaml_io.py: bounded, source-located strict frontmatter; existing installed dependency/lock resolution; observed local bytes distinguished from Git-backed integrity.contracts/{models,events,engine}.py: one per-command conductor, revalidated admission, deadline clipping, native/OS result separation, and capture/check/record sequencing.contracts/workspace.py: effective tracked changes/deletions plus explicit untracked resources, raw SHA-256, synthetic Git baseline, independent check copies, and exact retained-artifact/log identity.contracts/{process,records}.py,utils/atomic_io.py: bounded POSIX supervision, original-group cleanup observations, raw check statuses, atomic completion, and incomplete/HALTED repair after finalization failure.runtime/{base,registry,copilot_runtime}.py: additive structured capability;view/apply_patch, exact-file write grant, shell/URL denials, and supervised run-only merged MCP inventory.policy/{discovery,contract_prerequisite}.py: additive offline positive-no-policy prerequisite; configured, unknown, disabled, and unsupported governance cases refuse.core/contract_logger.py,contracts/stream.py,utils/console.py: one append-only stream, phase-aware JSONL decoding, stderr, bounded private logs, no reasoning/raw protocol dumps, copyable paths, NO_COLOR, and closed-pipe handling..apm/architecture/owners/contracts-tooling.json,scripts/architecture_linter/{checks/contract_leaf_runtime,groups/contracts_tests}.py: register and guard new durable decisions..gitignoreexcludes run state.examples/contracts/README.mdand all files underfirst-contract/,handoff-style/,reuse-contract/: standalone handoff, shared stdlib checker/notes, and explicitly installed skill reuse.tests/unit/contracts/: frontend/imports/policy, dispatch/runtime/engine, exact-byte/process/record, example criteria, architecture, stream/logger, and real-process/PTY cases.docs/src/content/docs/{consumer/run-contracts,reference/cli/plan,reference/cli/run}.md,docs/astro.config.mjs,.apm/docs-index.yml, andpackages/apm-guide/.apm/skills/apm-usage/{commands,package-authoring}.md: first-run, source, outcome, limit, navigation, and agent-guidance updates. Root README is unchanged.Review anchors: dispatch boundary, conductor, outcome reducer, and native inventory.
Outcome and record contract
2220210Raw check exits
0/1/2mean pass/failure/incomplete; unknown exits, missing tools, and signals are incomplete. Per-check timeout stays incomplete unless the root watchdog expires. Missing output is not assessed merely to manufacture rejection; successful forced cleanup never upgrades a lingering-child failure.Artifacts remain at
.apm/runs/<run-id>/artifacts/, withrecord.jsonand boundedtranscript.log; original output paths are not overwritten. Records preserve source/input/import/output/check identities and raw process observations. Requested and observed models are separate; unavailable usage/version observations are not invented.Diagrams
The new dashed stages connect existing APM source selection to exact-artifact assessment; execution requires explicit native-advisory consent, while offline planning stops before baseline creation.
flowchart LR subgraph Sources["Source selection"] Source["Markdown + frontmatter"] Installed["Existing manifest, lock, installed skill"] Plan["Strict offline plan"] Source --> Plan Installed --> Plan end subgraph Native["Explicit native-advisory run"] Baseline["Captured working bytes"] Copilot["Copilot through structured adapter"] Baseline --> Copilot end subgraph Assessment["Exact artifact assessment"] Artifact["Capture one fresh file"] Checks["Independent copy for each check"] Artifact --> Checks end subgraph Outcome["Local observations"] Record["Atomic record and outcome"] end Plan --> Baseline Copilot --> Artifact Checks --> Record classDef new stroke-dasharray: 5 5; class Source,Plan,Baseline,Copilot,Artifact,Checks,Record new;Trade-offs
run,budget, orsandboxrefuses rather than downgrading.copilot mcp list --jsonsupplies merged User/Workspace/Plugin/Builtin names for repeated per-invocation disable flags. No hardcoded user server list, retained config values, global config writes, or additional tool grants. Unobservable/failed inventory stops before production; extensions remain outside isolation guarantees.Benefits
Validation
Real native acceptance—not mocks
Both disposable fixtures completed through Copilot CLI 1.0.83-5 before publication. No native demos were rerun merely to open this PR.
examples/contracts/first-contractgpt-6-astra/gpt-6-astra0 / 000examples/contracts/reuse-contractgpt-6-astra/gpt-6-astra0 / 000Both produced captured JSON artifacts and ended with naturally terminated original process groups, without signals. The reuse output satisfied the extra
Check first:criterion supplied by the installed skill. Earlier unsuccessful attempts remained HALTED with provisional artifacts; they were not rewritten as success. Private run records/transcripts are retained locally, not uploaded in this PR.Local automated evidence
The publication run used the retained worktree-local environment with existing system-site packages and
--no-sync. This is not frozen-lock reproducibility, the full repository test suite, or a claim of remote CI green.Exact targeted command and observed result
PYTHONPATH="$PWD/src" .venv/bin/python -m pytest -p no:cacheprovider -q \ tests/unit/contracts \ tests/unit/test_copilot_runtime.py tests/unit/test_runtime_factory.py \ tests/unit/commands/test_run_command_surface.py tests/unit/commands/test_run_phase3.py \ tests/unit/test_yaml_io.py tests/unit/utils/test_atomic_io.py \ tests/unit/policy/test_discovery.py tests/unit/policy/test_discovery_policy_resolution.py \ tests/unit/policy/test_chain_discovery_shared.py \ tests/unit/scripts/test_architecture_registry.py \ tests/unit/scripts/test_architecture_registry_conflicts.py tests/test_console.pyPublication quality gates and base integration
Full Ruff check/format, pylint R0801, auth boundary, architecture registry/boundaries, YAML I/O, portable-path, 2100-line, assertion-quality, exact-test-duplicate, and whitespace gates passed locally.
The same static gates were assessed on a disposable merge tree of
f906a748ewithorigin/mainat13386d993; no feature-branch merge or main-checkout changes were made. The archived tree was given private Git/index metadata for the tracked-file ratchets.uv run --no-sync --extra dev ruff check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/:uv run --no-sync --extra dev ruff format --check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/on that merge tree:uv run --no-sync --extra dev python scripts/check_test_assertions.py --root <merged-tree>andscripts/check_exact_test_duplicates.py --root <merged-tree>:The Mermaid block was rendered successfully with
mmdc.Scenario Evidence
.contract.mdnames—still run as scripts without--on.tests/unit/contracts/test_dispatch.py::test_contract_named_script_is_still_a_script_without_on; existingtests/unit/commands/test_run_command_surface.pytests/unit/contracts/test_cli_lifecycle.py::test_plan_neither_executes_native_nor_mutates_project_or_home;test_run_without_consent_refuses_without_native_or_run_directorytests/unit/contracts/test_frontend.py;tests/unit/contracts/test_contract_policy.py;tests/unit/contracts/test_cli_lifecycle.py::test_unsupported_source_cannot_be_waived_with_consenttests/unit/contracts/test_imports.py::test_local_identity_and_snapshot_do_not_claim_locked_content_hash;tests/unit/contracts/test_logger.py::test_reuse_plan_shows_resolved_skill_identity_before_consenttests/unit/contracts/test_reliability.py::test_exact_bytes_and_independent_self_applied_patch;test_effective_tracked_edits_deletions_and_selected_untracked;tests/unit/contracts/test_engine.py::test_stale_output_cannot_satisfy_missing_new_outputtests/unit/contracts/test_engine.py;tests/unit/contracts/test_example_checks.py;tests/unit/contracts/test_stream.py::test_nonzero_result_is_sticky_even_after_idle_and_later_zerotests/unit/contracts/test_contract_runtime.py::test_unobservable_inventory_refuses_without_retaining_values;test_inventory_operational_failure_never_launches_producertests/unit/contracts/test_pty_lifecycle.py::test_pty_interrupt_leaves_halted_record_and_no_success;tests/unit/contracts/test_terminal_pty.py;tests/unit/contracts/test_logger.py::test_full_source_artifact_and_log_paths_stay_copyabletests/unit/contracts/test_reliability.py::test_inventory_never_executes_configured_fsmonitor;test_post_replace_directory_sync_failure_cannot_leave_verified_recordHow to test
Prerequisites: macOS/Linux, Git, Python 3, this APM checkout, and an authenticated native Copilot with the requested model available. The live commands consume model usage; they are separate from automated fake-harness coverage.
uv sync --frozen --extra dev, then retainexport APM_BIN="$PWD/.venv/bin/apm". This is reviewer setup, not a claim that the author's environment reproduced the frozen lock.uv run --frozen --extra dev pytest -p no:cacheprovider -q tests/unit/contracts; expect deterministic contract, process, and PTY coverage without native model inference."$APM_BIN" run ./handoff.contract.md --on copilot --model gpt-6-astra --allow-advisory. Expect observable phases, VERIFIED/0 when the declared check passes, and printed artifact/record/log paths under.apm/runs/; inspect the result rather than relying only on chat output.handoff-style, disclose local-source assurance, and the run to assess both handoff and caution-format criteria:Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com