feat(audit): close audit-blindness gap for local .apm/ content (#887)

[danielmeppiel]

What & Why

CISOs reported an audit-blindness gap: apm install was deploying local .apm/ content into governed directories (.github/, .claude/, etc.), but apm audit --ci never hash-verified those deployed files. The lockfile recorded local_deployed_files and their hashes, yet the audit pipeline only walked package dependencies — so post-install tampering of any locally-sourced file went undetected.

This PR closes the gap with end-to-end hash verification of every managed file plus a new includes: manifest field that gives users (and enterprises) explicit governance consent over what local content gets deployed.

Closes #887.

Solution overview

• Self-entry synthesis at the lockfile read boundary — local-content state is projected as a virtual <self> dependency keyed by _SELF_KEY = ".". The on-disk YAML schema is unchanged; to_yaml() does a round-trip pop+restore so the synthetic entry never persists.
• New includes: manifest field — three modes: auto (deploy all .apm/), an explicit list, or omitted (deploy nothing). Purely additive; existing manifests keep working.
• Hash verification in the content-integrity audit check — now covers the self-entry alongside package deps, detecting post-install tampering on any managed file.
• New baseline check includes-consent — advisory only, never blocks.
• New policy manifest.require_explicit_includes (block / warn / off) — enterprises can mandate explicit include lists.
• apm init -y scaffolds includes: auto — clean first-run UX.
• Bundle stripping — local-content fields are removed before bundle publish; no leakage into the marketplace.

Critical bug fixes discovered during dogfood

These were caught while dogfooding the new audit on the APM repo itself and are essential to the fix:

• cleanup.py — orphan loop was deleting all 26 deployed files post-install because the synthesized self-entry was misclassified as an orphan package. Fixed by treating _SELF_KEY as protected.
• lockfile.py — is_semantically_equivalent skipped local_deployed_file_hashes comparison, so hash-only updates never persisted. Fixed by including the hashes in the equivalence check.
• lockfile_enrichment.py (bundle path) — local-content fields needed stripping after the YAML round-trip. Fixed.

Audit coverage: before / after

flowchart LR
  subgraph BEFORE["BEFORE (audit blind to local .apm/)"]
    I1[apm install] --> S1[.apm/ source files]
    S1 --> D1[.github/ deployed]
    I1 --> L1[apm.lock.yaml<br/>local_deployed_files + hashes<br/>written but never read]
    A1[apm audit --ci] --> L1
    A1 -.never reads.-> D1
    D1 -.HASH DRIFT INVISIBLE.-> X1((CISO blind))
  end

Loading

flowchart LR
  subgraph AFTER["AFTER (every managed file verified)"]
    I2[apm install] --> S2[.apm/ source files]
    S2 --> D2[.github/ deployed]
    I2 --> L2[apm.lock.yaml<br/>local_deployed_files + hashes]
    L2 --> SE[from_yaml synthesizes<br/>self-entry under &lt;self&gt;]
    A2[apm audit --ci] --> L2
    SE --> HV[content-integrity:<br/>hash-verify every<br/>deployed file]
    HV -. detects .-> D2
    HV --> OK([7/7 green])
    HV --> DRIFT([hash-drift: file<br/>dep=&lt;self&gt;])
  end

Loading

Verification

• Unit suite: 5383/5384 pass. The sole failure is the pre-existing flaky test_user_scope_skips_workspace_runtimes, unrelated to this change.
• Live dogfood end-to-end:
  • apm install clean, apm audit --ci returns 7/7 green
  • Introduced drift on a deployed file -> audit detects with dep=<self>
  • Restored file -> audit returns to 7/7 green
• apm init -y confirmed to scaffold includes: auto.
• Pre-existing dogfood drift on auth-expert.agent.md and cicd.instructions.md remains visible — these are real post-install hand-edits from prior PRs and are handled outside this PR.

Panel reviews

All three specialist panels reviewed; CEO signed off SHIP.

• Supply chain security — PASS-WITH-NOTES. Path-validation guard added to the hash-verify loop.
• CLI logging UX — PASS-WITH-NOTES. Status-symbol contradiction fixed, remediation made conditional, hashes truncated for terminal width.
• DevX UX — PASS-WITH-NOTES. init scaffold, doc check counts updated (6+16 -> 7+17), error wording branched, dep=<self> label introduced, cli-commands.md updated.
• APM CEO — SHIP. "Every managed file, verified" — the audit-coverage milestone.

Breaking changes

None. Purely additive. No migration required for any user class — existing manifests, lockfiles, and CI flows continue to work unchanged.

Version

Bumps to 0.10.0 (minor: significant new functionality, no breakage).

Test plan

uv run pytest tests/unit tests/test_console.py
uv run pytest tests/integration/test_local_content_audit.py   # new file: 5 subprocess tests A-E

[Copilot]

Pull request overview

Closes the apm audit --ci integrity gap for locally-deployed .apm/ content by projecting local lockfile state into a virtual self-entry and extending baseline audit to hash-verify all managed files (including local ones), plus adding an includes: manifest consent signal with optional policy enforcement.

Changes:

• Synthesize a virtual . / <self> lockfile dependency in-memory and update key call sites to skip it where appropriate.
• Extend baseline content-integrity to include per-file SHA-256 drift detection; add includes-consent baseline advisory and manifest.require_explicit_includes policy check.
• Add/adjust tests and docs to cover the new audit behavior, policy seam wiring, and bundling safeguards (strip local-content metadata; skip local-source deps in bundles).

Show a summary per file

File	Description
uv.lock	Bumps apm-cli version to 0.10.0.
pyproject.toml	Bumps project version to 0.10.0.
apm.yml	Bumps repo manifest version and adds includes: auto.
apm.lock.yaml	Updates local deployed file hashes (dogfood drift baseline).
CHANGELOG.md	Adds Unreleased entries for self-entry/includes/audit hash drift and related fixes.
docs/src/content/docs/reference/manifest-schema.md	Documents new includes: manifest field and policy interaction.
docs/src/content/docs/reference/lockfile-spec.md	Documents the in-memory self-entry convention and invariants.
docs/src/content/docs/reference/cli-commands.md	Updates apm audit --ci baseline check enumeration and hash drift behavior.
docs/src/content/docs/integrations/ci-cd.md	Updates baseline/policy check counts (7 + 17).
docs/src/content/docs/guides/ci-policy-setup.md	Updates baseline/policy check counts and text references.
docs/src/content/docs/enterprise/governance-guide.md	Updates governance matrix/counts and adds explicit-includes policy documentation.
docs/src/content/docs/enterprise/apm-policy.md	Updates CI-time baseline check count in docs.
packages/apm-guide/.apm/skills/apm-usage/governance.md	Extends governance skill docs with explicit-includes knob + includes: explanation.
src/apm_cli/deps/lockfile.py	Adds _SELF_KEY, synthesizes self-entry in from_yaml(), excludes it in new get_package_dependencies(), updates equivalence and installed-paths guard.
src/apm_cli/policy/ci_checks.py	Extends baseline checks: local-only lockfile relevance, orphan skip for self, hash drift verification, adds includes-consent advisory.
src/apm_cli/policy/policy_checks.py	Adds explicit-includes policy check and a seam param to run it only when manifest context is provided.
src/apm_cli/policy/parser.py	Parses/validates manifest.require_explicit_includes.
src/apm_cli/policy/schema.py	Adds require_explicit_includes to policy schema.
src/apm_cli/install/phases/policy_gate.py	Threads ctx.apm_package.includes into dependency policy checks when available.
src/apm_cli/install/phases/cleanup.py	Protects the synthesized self-entry from orphan cleanup deletion.
src/apm_cli/bundle/packer.py	Skips source=local dependencies when collecting deployed files for apm bundles.
src/apm_cli/bundle/lockfile_enrichment.py	Strips local_deployed_files + local_deployed_file_hashes from bundle lockfile YAML.
src/apm_cli/models/apm_package.py	Adds includes field parsing/validation to APMPackage.
src/apm_cli/integration/skill_integrator.py	Uses get_package_dependencies() to avoid <self> in ownership maps.
src/apm_cli/integration/mcp_integrator.py	Uses get_package_dependencies() to avoid <self> during transitive collection.
src/apm_cli/commands/_helpers.py	apm init scaffolds includes: auto and skips self-entry in install-path expectations.
src/apm_cli/commands/uninstall/engine.py	Uses get_package_dependencies() to avoid self-entry crashes in uninstall flows.
src/apm_cli/commands/deps/cli.py	Uses get_package_dependencies() so self-entry does not show up in apm deps tree.
tests/unit/test_lockfile_self_entry.py	Adds unit coverage for self-entry synthesis + serialization invariants.
tests/unit/test_self_entry_caller_guards.py	Regression tests for representative callers skipping the self-entry.
tests/unit/test_packer.py	Adds tests for excluding local-source deps in bundles and stripping local-content fields.
tests/unit/test_deps_list_tree_info.py	Adds guard test ensuring deps tree hides self-entry.
tests/unit/test_ci_checks.py	Adds baseline audit coverage for hash drift + local-only repo behavior + includes-consent.
tests/unit/test_audit_policy_command.py	Updates baseline check count expectations in CI/policy JSON tests.
tests/unit/test_apm_package.py	Adds unit tests for includes parsing and behavior.
tests/unit/install/test_policy_gate_phase.py	Tests that policy_gate threads manifest_includes correctly.
tests/unit/policy/test_policy_checks.py	Updates expected policy check count to 17 and imports explicit-includes check.
tests/unit/policy/test_run_dependency_policy_checks.py	Adds seam-level tests for explicit-includes check gating.
tests/unit/policy/test_parser.py	Adds parser/validator tests for require_explicit_includes.
tests/unit/install/test_file_scanner.py	Adds coverage tests for compute_deployed_hashes directory/symlink/empty-file behavior.
tests/integration/test_local_content_audit.py	Adds end-to-end subprocess tests for local-content install + audit drift detection + policy enforcement.

Copilot's findings

Comments suppressed due to low confidence (1)

docs/src/content/docs/reference/manifest-schema.md:180

• This section claims apm audit emits an [!] includes-consent advisory, but the current CI renderer only shows pass/fail status symbols and includes-consent returns passed=True with no [!] prefix in the message. Either update the implementation to include [!] in the includes-consent message (including JSON/SARIF), or adjust this doc text to describe the advisory without asserting a specific status symbol.

Declares which local `.apm/` content the project consents to publish when packing or deploying. Three forms are supported:

1. **Undeclared** -- field omitted. Legacy behaviour: all local `.apm/` content is published as if `auto` were set. `apm audit` emits an `[!]` includes-consent advisory whenever local content is deployed under this form.
2. **`includes: auto`** -- explicit consent to publish all local `.apm/` content via the file scanner. No path enumeration required. Default for newly initialised projects.

• Files reviewed: 40/41 changed files
• Comments generated: 5

[danielmeppiel]

Note on follow-up

This PR is the foundation that Epic #898 builds on. The Epic-PR will rebase on top of the work landed here and extend it with:

• per-asset enumeration inside bundle directories (the gap exposed by PR harden(apm-review-panel): one-comment discipline + Hybrid E auth routing + apm-primitives-architect persona #882)
• skill-as-package model (Skills and agents as first-class APM packages: intra-repo local-dep resolution + per-asset lockfile hashes #896)
• apm audit --drift user-facing alias
• dependency-closure policy rule
• self-host CI dogfood on microsoft/apm

No scope change requested here; this PR ships as designed and closes #684 + #887. The Epic owns the next layer.