refactor(ci): parallelize native qualification and promote verified release candidates - #2879
refactor(ci): parallelize native qualification and promote verified release candidates#2879Daniel Meppiel (danielmeppiel) wants to merge 23 commits into
Conversation
…candidates Preserve native coverage, test the Windows candidate, bind promotion to immutable evidence, harden merge gates, and record safe timing snapshots. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It makes broad, security-sensitive changes across release qualification and CI gating that warrant careful human verification in GitHub Actions.
Review tier: Lite
Findings: 1
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
tests/utils/windows_installer_candidate.py — This new test helper imports packaging.version.Version, but packaging is not declared in… |
What changed in this PR
This PR refactors the CI/release qualification pipeline to decouple native unit tests, builds, integration, isolated validation, and installer verification, and to promote only release artifacts that are bound to immutable, verified qualification evidence.
Changes:
- Adds a reusable per-platform workflow (
release-platform.yml) that runs native units/build first, then lets integration/validation/installer consumers depend only on the platform build artifact. - Introduces candidate packaging + promotion authorities (
scripts/package_release.py,scripts/release-candidate.cjs) that package once, verify archive identity/digests, enforce attempt-scoped artifact identities, and fail closed on ambiguity. - Hardens CI/test gates (merge-gate required-check polling, runtime prerequisite strictness, Windows installer candidate fixture + contracts, pytest timing snapshotting) and updates contributor docs accordingly.
| File | Description |
|---|---|
| tests/utils/windows_installer_candidate.py | Adds verified Windows installer candidate fixtures and a loopback server used by installer E2E. |
| tests/unit/test_windows_unit_diagnostics.py | Updates the workflow contract for Windows unit-test diagnostics in the new reusable workflow. |
| tests/unit/test_windows_signing_contract.py | Updates signing-step contract assertions to the new per-platform workflow job layout. |
| tests/unit/test_windows_installer_candidate.py | Adds cross-platform unit proofs for candidate-only Windows installer fixture inputs and integrity. |
| tests/unit/test_unix_installer_checksums.py | Ensures fake tool invocations are stdlib-only (python -S) for determinism and speed. |
| tests/unit/test_runtime_prerequisite_gate.py | Adds unit coverage for failing closed when selected runtime prerequisites are missing in strict CI mode. |
| tests/unit/test_pytest_timing_workflows.py | Adds contract tests for immutable timing snapshots and disjoint shard selection guarantees. |
| tests/unit/test_platform_contract_workflow.py | Reworks hosted workflow contracts for parallel native qualification, artifact identity, and publication gating. |
| tests/unit/test_merge_gate_wait.py | Adds hermetic tests for the merge-gate polling script’s fail-closed semantics and identity checks. |
| tests/unit/test_mcp_registry_collection.py | Verifies MCP registry E2E module import/collection does not attempt network access. |
| tests/unit/test_ado_live_workflow_contract.py | Updates ADO-live selection contracts to the extracted platform workflow and catalog. |
| tests/unit/scripts/test_package_release.py | Adds unit coverage for “package once” archives and verify/extract safety checks. |
| tests/quality/test_ci_topology.py | Updates required-check topology assertions to require Lint + ratchets in both PR and merge-group contexts. |
| tests/integration/test_windows_installer_launchers.py | Switches installer E2E to candidate-archive + loopback fixture model with request auditing. |
| tests/integration/test_mcp_registry_e2e.py | Avoids repeated live health probes by caching and using a fixture for conditional skips. |
| tests/integration/conftest.py | Adds --strict-runtime-prerequisites to fail collection when a selected runtime is missing (instead of skipping). |
| scripts/windows/test-install-script.ps1 | Refactors installer tests to target the verified candidate fixture (no published-release destination), adds tamper rejection + candidate hash assertions. |
| scripts/windows/build-binary.ps1 | Hardens build SHA injection (CRLF-safe) and fails closed when injection doesn’t apply. |
| scripts/test-integration.sh | Makes runtime provisioning selectable (APM_TEST_RUNTIMES) and enforces strict runtime prerequisite collection in CI. |
| scripts/release-platforms.json | Introduces the canonical platform catalog and per-platform integration-marker policy. |
| scripts/release-candidate.cjs | Implements candidate planning/qualification/verification logic with immutable artifact IDs, config hashing, and fail-closed evidence checks. |
| scripts/package_release.py | Implements archive creation, digest sidecars/metadata, safe extraction, and binary identity verification used by validation and publication. |
| scripts/build-binary.sh | Removes UPX provisioning messaging/logic for consistency with the release packaging policy. |
| docs/src/content/docs/contributing/integration-testing.md | Updates integration-testing documentation to reflect the new qualification, runtime selection, strict runtime prerequisites, and artifact provenance rules. |
| docs/src/content/docs/contributing/development-guide.md | Updates contributor guidance to reflect the gate check authority and candidate reuse constraints. |
| .github/workflows/release-platform.yml | Adds the reusable native lane (units/build + consumers + per-platform “Native Candidate Gate”). |
| .github/workflows/merge-gate.yml | Requires Lint + architecture ratchets in both contexts and documents the stronger poll semantics. |
| .github/workflows/ci.yml | Adds workflow_call support and publishes timing/JUnit artifacts for shards; runs release-candidate JS tests. |
| .github/workflows/ci-runtime.yml | Supplies missing E2E/integration opt-ins for runtime smoke. |
| .github/workflows/ci-integration.yml | Freezes one shared integration timing snapshot and publishes shard timing/JUnit artifacts; removes UPX provisioning. |
| .github/scripts/ci/merge_gate_wait.sh | Refactors to single-snapshot polling, enforces GitHub Actions identity and ambiguity fail-closed semantics, and rejects non-success conclusions. |
| .github/instructions/cicd.instructions.md | Synchronizes CI/CD rules with the new workflow topology, candidate provenance, and artifact handling. |
| .github/actions/pytest-timing/action.yml | Adds a composite action for timing restore and immutable snapshot publication (restore-only, no rolling writes). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ployment Register strict runtime options before whole-tree collection and regenerate the CI instruction self-deployment from its canonical source. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve exact candidate and scenario identity, bind rerun artifacts to attempts, avoid AST work in lexical guards, and add opt-in controlled performance probes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Match the native artifact envelope instead of flattening release-assets during upload. Add a regression for the failed hosted probe layout. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Phase-two architecture performance evidencePermanent raw evidence for the lexical optimization in this PR. This is a source-only, direct Six fixed mutation scenarios, ABBA order, two repeats each: 34.490s baseline -> 10.525s optimized (69.5% lower). All 24 scenario assertions passed, with zero report failures. The five duplicate-heavy scenarios drop from 477 parses/indexes each to 1-13; frontmatter remains the broad-parse control. Original source label: The earlier failed 337-case full-family pair (212.677s -> 304.880s) is explicitly excluded. Its slowdown was not reproduced, but its cause is not established. No whole-family speedup is inferred from the later successful matrix. Raw 24 scenario executions, outcomes and counters{"abba_index": 1, "batch_elapsed_seconds": 17.454663875018014, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.8384222500026226, "failure_count": 0, "guard_id": "contracts-tooling-cached-policy-shape", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.7854032089817338, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[contracts-tooling-cached-policy-shape]", "outcome": "passed", "rule_id": "contracts-tooling-cached-policy-shape", "violation_count": 1}
{"abba_index": 1, "batch_elapsed_seconds": 17.454663875018014, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 3.034132166998461, "failure_count": 0, "guard_id": "contracts-tooling-frontmatter-yaml", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.974740584002575, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[contracts-tooling-frontmatter-yaml]", "outcome": "passed", "rule_id": "contracts-tooling-frontmatter-yaml", "violation_count": 1}
{"abba_index": 1, "batch_elapsed_seconds": 17.454663875018014, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.8665437919844408, "failure_count": 0, "guard_id": "install-deployment-frozen-mutation-eligibility", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.813187583000399, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-frozen-mutation-eligibility]", "outcome": "passed", "rule_id": "install-deployment-frozen-mutation-eligibility", "violation_count": 1}
{"abba_index": 1, "batch_elapsed_seconds": 17.454663875018014, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.808129540993832, "failure_count": 0, "guard_id": "install-deployment-mcp-ownership-migration", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.754094999982044, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-mcp-ownership-migration]", "outcome": "passed", "rule_id": "install-deployment-mcp-ownership-migration", "violation_count": 1}
{"abba_index": 1, "batch_elapsed_seconds": 17.454663875018014, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.7469971659884322, "failure_count": 0, "guard_id": "install-deployment-prospective-dry-run-plan", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.6794039589876775, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-prospective-dry-run-plan]", "outcome": "passed", "rule_id": "install-deployment-prospective-dry-run-plan", "violation_count": 1}
{"abba_index": 1, "batch_elapsed_seconds": 17.454663875018014, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.8505892080138437, "failure_count": 0, "guard_id": "install-deployment-source-plan", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.793726540985517, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-source-plan]", "outcome": "passed", "rule_id": "install-deployment-source-plan", "violation_count": 1}
{"abba_index": 2, "batch_elapsed_seconds": 5.5390263329900336, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.6256675420154352, "failure_count": 0, "guard_id": "contracts-tooling-cached-policy-shape", "metrics": {"ast_visits": 8875, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 1, "read_attempts": 484, "total_seconds": 0.6130847919848748, "tree_index_builds": 1, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[contracts-tooling-cached-policy-shape]", "outcome": "passed", "rule_id": "contracts-tooling-cached-policy-shape", "violation_count": 1}
{"abba_index": 2, "batch_elapsed_seconds": 5.5390263329900336, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.45171933301026, "failure_count": 0, "guard_id": "contracts-tooling-frontmatter-yaml", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.402815333014587, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[contracts-tooling-frontmatter-yaml]", "outcome": "passed", "rule_id": "contracts-tooling-frontmatter-yaml", "violation_count": 1}
{"abba_index": 2, "batch_elapsed_seconds": 5.5390263329900336, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.5407385829894338, "failure_count": 0, "guard_id": "install-deployment-frozen-mutation-eligibility", "metrics": {"ast_visits": 8088, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 2, "read_attempts": 484, "total_seconds": 0.5392124580102973, "tree_index_builds": 2, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-frozen-mutation-eligibility]", "outcome": "passed", "rule_id": "install-deployment-frozen-mutation-eligibility", "violation_count": 1}
{"abba_index": 2, "batch_elapsed_seconds": 5.5390263329900336, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.504773833003128, "failure_count": 0, "guard_id": "install-deployment-mcp-ownership-migration", "metrics": {"ast_visits": 15300, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 5, "read_attempts": 484, "total_seconds": 0.5029639170097653, "tree_index_builds": 5, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-mcp-ownership-migration]", "outcome": "passed", "rule_id": "install-deployment-mcp-ownership-migration", "violation_count": 1}
{"abba_index": 2, "batch_elapsed_seconds": 5.5390263329900336, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.5211999999883119, "failure_count": 0, "guard_id": "install-deployment-prospective-dry-run-plan", "metrics": {"ast_visits": 7836, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 3, "read_attempts": 484, "total_seconds": 0.5194111670134589, "tree_index_builds": 3, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-prospective-dry-run-plan]", "outcome": "passed", "rule_id": "install-deployment-prospective-dry-run-plan", "violation_count": 1}
{"abba_index": 2, "batch_elapsed_seconds": 5.5390263329900336, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.6020510410016868, "failure_count": 0, "guard_id": "install-deployment-source-plan", "metrics": {"ast_visits": 43658, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 13, "read_attempts": 484, "total_seconds": 0.595757874980336, "tree_index_builds": 13, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-source-plan]", "outcome": "passed", "rule_id": "install-deployment-source-plan", "violation_count": 1}
{"abba_index": 3, "batch_elapsed_seconds": 5.495421583997086, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.5644712499924935, "failure_count": 0, "guard_id": "contracts-tooling-cached-policy-shape", "metrics": {"ast_visits": 8875, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 1, "read_attempts": 484, "total_seconds": 0.5629522079834715, "tree_index_builds": 1, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[contracts-tooling-cached-policy-shape]", "outcome": "passed", "rule_id": "contracts-tooling-cached-policy-shape", "violation_count": 1}
{"abba_index": 3, "batch_elapsed_seconds": 5.495421583997086, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.5544120000267867, "failure_count": 0, "guard_id": "contracts-tooling-frontmatter-yaml", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.503135792008834, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[contracts-tooling-frontmatter-yaml]", "outcome": "passed", "rule_id": "contracts-tooling-frontmatter-yaml", "violation_count": 1}
{"abba_index": 3, "batch_elapsed_seconds": 5.495421583997086, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.5248760420072358, "failure_count": 0, "guard_id": "install-deployment-frozen-mutation-eligibility", "metrics": {"ast_visits": 8088, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 2, "read_attempts": 484, "total_seconds": 0.5232549580105115, "tree_index_builds": 2, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-frozen-mutation-eligibility]", "outcome": "passed", "rule_id": "install-deployment-frozen-mutation-eligibility", "violation_count": 1}
{"abba_index": 3, "batch_elapsed_seconds": 5.495421583997086, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.5034217910142615, "failure_count": 0, "guard_id": "install-deployment-mcp-ownership-migration", "metrics": {"ast_visits": 15300, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 5, "read_attempts": 484, "total_seconds": 0.5014852079912089, "tree_index_builds": 5, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-mcp-ownership-migration]", "outcome": "passed", "rule_id": "install-deployment-mcp-ownership-migration", "violation_count": 1}
{"abba_index": 3, "batch_elapsed_seconds": 5.495421583997086, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.4908202909864485, "failure_count": 0, "guard_id": "install-deployment-prospective-dry-run-plan", "metrics": {"ast_visits": 7836, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 3, "read_attempts": 484, "total_seconds": 0.4890739999827929, "tree_index_builds": 3, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-prospective-dry-run-plan]", "outcome": "passed", "rule_id": "install-deployment-prospective-dry-run-plan", "violation_count": 1}
{"abba_index": 3, "batch_elapsed_seconds": 5.495421583997086, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 0.6410540840006433, "failure_count": 0, "guard_id": "install-deployment-source-plan", "metrics": {"ast_visits": 43658, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 13, "read_attempts": 484, "total_seconds": 0.6248712920059916, "tree_index_builds": 13, "tree_index_cache_hits": 0}, "mode": "optimized_lexical_lines", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-source-plan]", "outcome": "passed", "rule_id": "install-deployment-source-plan", "violation_count": 1}
{"abba_index": 4, "batch_elapsed_seconds": 17.514485708001303, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.7862215409986675, "failure_count": 0, "guard_id": "contracts-tooling-cached-policy-shape", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.736373082996579, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[contracts-tooling-cached-policy-shape]", "outcome": "passed", "rule_id": "contracts-tooling-cached-policy-shape", "violation_count": 1}
{"abba_index": 4, "batch_elapsed_seconds": 17.514485708001303, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 3.0431924579897895, "failure_count": 0, "guard_id": "contracts-tooling-frontmatter-yaml", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.9902510830143, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[contracts-tooling-frontmatter-yaml]", "outcome": "passed", "rule_id": "contracts-tooling-frontmatter-yaml", "violation_count": 1}
{"abba_index": 4, "batch_elapsed_seconds": 17.514485708001303, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.9645252500195056, "failure_count": 0, "guard_id": "install-deployment-frozen-mutation-eligibility", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.9145757909864187, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-frozen-mutation-eligibility]", "outcome": "passed", "rule_id": "install-deployment-frozen-mutation-eligibility", "violation_count": 1}
{"abba_index": 4, "batch_elapsed_seconds": 17.514485708001303, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.908012875006534, "failure_count": 0, "guard_id": "install-deployment-mcp-ownership-migration", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.8573549590073526, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-mcp-ownership-migration]", "outcome": "passed", "rule_id": "install-deployment-mcp-ownership-migration", "violation_count": 1}
{"abba_index": 4, "batch_elapsed_seconds": 17.514485708001303, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.74130629200954, "failure_count": 0, "guard_id": "install-deployment-prospective-dry-run-plan", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.6776912079949398, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-prospective-dry-run-plan]", "outcome": "passed", "rule_id": "install-deployment-prospective-dry-run-plan", "violation_count": 1}
{"abba_index": 4, "batch_elapsed_seconds": 17.514485708001303, "blamed_count": 1, "broken_input_stages": [], "elapsed_seconds": 2.901462291018106, "failure_count": 0, "guard_id": "install-deployment-source-plan", "metrics": {"ast_visits": 649730, "max_parses_per_file": 1, "max_reads_per_file": 1, "max_tree_index_builds_per_file": 1, "parse_attempts": 477, "read_attempts": 484, "total_seconds": 2.84993866700097, "tree_index_builds": 477, "tree_index_cache_hits": 0}, "mode": "baseline_file_facts", "nodeid": "tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[install-deployment-source-plan]", "outcome": "passed", "rule_id": "install-deployment-source-plan", "violation_count": 1}Portable reproduction harnessOnly the output-directory assignment below differs from the recorded inline harness. The original saved harness SHA-256 is PERF_OUTPUT="$(mktemp -d)" PYTHONHASHSEED=0 uv run --frozen --no-sync python -c "$(cat /path/to/lexical-abba-harness.py)"import importlib, json, os, pathlib, subprocess, sys, time
art=pathlib.Path(os.environ["PERF_OUTPUT"])
art.mkdir(parents=True, exist_ok=True)
jsonl=art/"lexical-abba-duplicate-subset-c026361e2.jsonl"
summary=art/"lexical-abba-duplicate-subset-c026361e2-summary.json"
child_code = r"""
import json, sys, time
mode=sys.argv[1]
case_ids=sys.argv[2].split(",")
if mode == "baseline_file_facts":
import scripts.architecture_linter.checks.lexical_shared as ls
def old_duplicate_definition_lines(provider, *, rule_id, prefix, pattern, owner, message, respect_exempt):
findings=[]
for path in ls.python_paths(provider, under=prefix):
if path == owner:
continue
facts = provider.file_facts(path)
if facts.read_error is not None:
continue
for number, line in enumerate(facts.lines, start=1):
if respect_exempt and ls.EXEMPT_MARKER in line:
continue
match = pattern.search(line)
if match is not None:
findings.append(ls.violation(rule_id, path, message, line=number, column=match.start()+1))
return findings
ls.duplicate_definition_lines = old_duplicate_definition_lines
import tests.integration.test_architecture_owner_rule_mutations as tm
from scripts.architecture_linter.runner import run_selected_rules
cases_by_guard={case.guard_id: case for case in tm.MUTATIONS}
rows=[]
for guard_id in case_ids:
case=cases_by_guard[guard_id]
node=f"tests/integration/test_architecture_owner_rule_mutations.py::test_owner_rule_catches_its_guard_mutation[{case.guard_id}]"
started=time.perf_counter()
report=run_selected_rules(tm.ROOT, (case.rule_id,), source_overrides={case.path: tm._mutate(case)})
elapsed=time.perf_counter()-started
blamed=[v for v in report.violations if v.rule_id == case.rule_id]
broken=[f.stage for f in report.failures if f.stage in (f"read:{case.path}", f"parse:{case.path}")]
rows.append({
"mode": mode,
"guard_id": case.guard_id,
"rule_id": case.rule_id,
"nodeid": node,
"elapsed_seconds": elapsed,
"outcome": "passed" if (blamed and not broken) else "failed",
"violation_count": len(report.violations),
"blamed_count": len(blamed),
"failure_count": len(report.failures),
"broken_input_stages": broken,
"metrics": {
"read_attempts": report.metrics.read_attempts,
"parse_attempts": report.metrics.parse_attempts,
"ast_visits": report.metrics.ast_visits,
"tree_index_builds": report.metrics.tree_index_builds,
"tree_index_cache_hits": report.metrics.tree_index_cache_hits,
"max_reads_per_file": report.metrics.max_reads_per_file,
"max_parses_per_file": report.metrics.max_parses_per_file,
"max_tree_index_builds_per_file": report.metrics.max_tree_index_builds_per_file,
"total_seconds": report.metrics.total_seconds,
},
})
print(json.dumps(rows, sort_keys=True))
"""
case_ids="contracts-tooling-cached-policy-shape,contracts-tooling-frontmatter-yaml,install-deployment-frozen-mutation-eligibility,install-deployment-mcp-ownership-migration,install-deployment-prospective-dry-run-plan,install-deployment-source-plan"
order=["baseline_file_facts","optimized_lexical_lines","optimized_lexical_lines","baseline_file_facts"]
all_rows=[]
jsonl.write_text("", encoding="utf-8")
for idx, mode in enumerate(order, start=1):
started=time.perf_counter()
proc=subprocess.run([sys.executable,"-c",child_code,mode,case_ids], text=True, capture_output=True, env=dict(os.environ, PYTHONHASHSEED="0"))
elapsed=time.perf_counter()-started
if proc.returncode != 0:
rec={"abba_index":idx,"mode":mode,"returncode":proc.returncode,"elapsed_seconds":elapsed,"stdout":proc.stdout[-4000:],"stderr":proc.stderr[-4000:]}
jsonl.open("a",encoding="utf-8").write(json.dumps(rec,sort_keys=True)+"\n")
print(json.dumps(rec,indent=2,sort_keys=True))
sys.exit(proc.returncode)
rows=json.loads(proc.stdout)
for row in rows:
row["abba_index"]=idx
row["batch_elapsed_seconds"]=elapsed
jsonl.open("a",encoding="utf-8").write(json.dumps(row,sort_keys=True)+"\n")
all_rows.append(row)
by_mode={}
for row in all_rows:
by_mode.setdefault(row["mode"],[]).append(row)
summary_data={"artifact":str(jsonl),"order":order,"case_ids":case_ids.split(","),"rows":len(all_rows),"all_passed":all(r["outcome"]=="passed" for r in all_rows),"by_mode":{}}
for mode, rows in by_mode.items():
summary_data["by_mode"][mode]={
"case_runs":len(rows),
"total_case_seconds":sum(r["elapsed_seconds"] for r in rows),
"mean_case_seconds":sum(r["elapsed_seconds"] for r in rows)/len(rows),
"mean_parse_attempts":sum(r["metrics"]["parse_attempts"] for r in rows)/len(rows),
"mean_ast_visits":sum(r["metrics"]["ast_visits"] for r in rows)/len(rows),
"mean_tree_index_builds":sum(r["metrics"]["tree_index_builds"] for r in rows)/len(rows),
"mean_read_attempts":sum(r["metrics"]["read_attempts"] for r in rows)/len(rows),
}
summary.write_text(json.dumps(summary_data,indent=2,sort_keys=True),encoding="utf-8")
print(json.dumps(summary_data,indent=2,sort_keys=True)) |
Two-worker cold sharding was not accepted: outcome parity held but hosted images differed and raw timing was below target. Retry three workers per shard without relaxing proof checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Both bounded hosted comparisons preserved every case but failed strict runner-image identity. Retain one four-worker release shard and the read-only two-shard experiment; do not claim unaccepted ARM savings. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Phase two: measured results and expensive-test evidencePhase two is folded into this PR, not a separate PR. Performance acceptance Accepted measurements
The Windows jobs, including setup/upload but excluding initial allocation, Hosted evidence: first Windows, Read-only documentation/wheel builders also completed successfully ARM decisionDeferred from production at Raw pytest times were 1167.573s baseline versus 458.564s/562.505s proposed; Candidate archive SHA256 for that final experiment: Whole-release accountingHistorical release: 39m16s. Retain the conservative phase-one fresh Experiments rejected, not hidden
The different ARM/Windows lock hashes are exactly LF versus CRLF checkout Expensive tests: actual hosted costsSource: ARM run 34152045985, These are summed JUnit case elapsed times, including fixtures, not CPU
The slowest individual baseline case is the generated lifecycle Cold two-shard assignment balanced counts, not work: 4811 cases each, Three Artifactory cases each cost about 35s on the hosted runner. All three formerly Copilot-gated configuration cases passed in both ARM Reproduce and inspectDownload the exact run's To reproduce the family ranking from each downloaded JUnit: import collections
import xml.etree.ElementTree as ET
families = collections.defaultdict(lambda: [0, 0.0])
for case in ET.parse("test-results/integration-shard-1.xml").getroot().iter("testcase"):
prop = case.find("./properties/property[@name='apm_performance_nodeid']")
node = prop.get("value")
row = families[node.split("::")[0]]
row[0] += 1
row[1] += float(case.get("time", "0"))
for name, (count, seconds) in sorted(families.items(), key=lambda item: -item[1][1]):
print(f"{seconds:9.3f}s {count:5d} {name}")Raw hosted artifacts have 30-day retention. This comment preserves the |
Compare ARM and Linux integration plus Intel units against complete unchanged native/source authorities. Match actual runner identities before pytest and retain exact scenario outcomes and observed native gate delays in the DAG model. Production sharding remains unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Whole-pipeline sharding model and hosted proofFinal result: three full attempts completed; required failures prevent Decision boundaryRun a read-only, five-platform release-qualification rehearsal at one PR The experiment tests three independent latency levers together, because
Integration pairs consume exactly the same once-built native archive and Why these three surfacesHistorical run 34104956270 provides actual stage durations:
These are historical observations, not predictions for today's source or Critical-path modelFor each variant, measure completion of every required source/native flowchart LR
O["one rehearsal SHA / origin"] --> S["source CI"]
O --> U["five native unit lanes"]
O --> B["five native builds / exact archives"]
B --> V["five isolated validations / Windows installer"]
B --> I["native integration"]
U --> UP["Intel: baseline versus two shards"]
I --> IP["ARM and Linux x64: baseline versus two shards"]
O --> D["read-only docs / wheels"]
S --> R["compare evidence-ready critical paths"]
V --> R
UP --> R
IP --> R
D --> R
Let
For a sharded surface, test completion is the latest shard completion. Proposed native-gate timestamps are therefore modeled with observed delays, Both variants coexist in one rehearsal. The workflow itself waits for all Environmental control before expensive executionEach paired surface has three actual test runners: baseline, proposed-1, All three must agree on source SHA, selection, candidate bytes (integration), The existing post-run exact-case/outcome checker remains unchanged in its Empirical protocol and acceptance
At the earlier modeled 32m12s-34m12s baseline, 10% means 3m13s-3m25s; Implementation ownership and current state
Implementation is pushed to PR #2879 at Independent review caught an omitted native-gate floor before any experiment The real GitHub attempt-specific APIs were checked: completed jobs expose Attempt 1: 34162882041Executed at merge SHA
The opt-in label has been removed, preventing accidental extra runs. Attempt Corrective commit before attempt 2
The exact Windows archive contains 50 DLLs. Under the original harness root, The public API prerequisite has real PowerShell negative tests proving an Attempt 2: 34165233237Executed at merge SHA
ARM preserved all 9622 outcomes (9468 passed, 152 skipped, 2 xfailed) on The Intel JUnit producer correction worked: completed pytest reports now Windows public isolated validation passed with the API-only token. The All available proofs and the complete job inventory are preserved in Attempt 3: 34167328643The final full attempt completed with failed required controls after Both integration pairs passed exact original-case/outcome comparison again:
ARM retained all 9622 outcomes; Linux retained all 9620. In actual Why the full result is rejectedThe registry-cache defect from attempt 2 did not recur. Intel proposed The shortened Windows fixture now reaches candidate download, checksum Candidate artifact Final empirical decisionThe preregistered three-full-attempt budget is exhausted. All three full The independent integration evidence is repeatable:
Do not average unrelated job clocks into a release total or discard the Production sharding stays unchanged. The next safety prerequisite is a The whole-DAG implementation and rejecting counterexamples remain available |
Emit concrete unittest subtest cases in the xdist controller without changing worker scheduling or relaxing JUnit parity. Accept read-only public API credentials without promoting them to PAT/Models aliases. Reserve native DLL path headroom in the installer fixture while retaining adversarial quoting and all original checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the Windows installer harness short while providing an uncreated isolation root, and clear temporary MCP registry configuration from the process cache on fixture teardown. Preserve native assertions and strict benchmark controls. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Repair the import-state leak exposed by the last Intel sharding rehearsal. Preserve both sys.modules and parent-package identity, with regressions for rich and colorama fallbacks. Do not relax performance acceptance or production gates. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add read-only, non-promotable full-DAG proof with strict archive, job, source identity, and actual GitHub clock validation. Preserve production job authorities and default test scheduling. Use the empirically proven short Windows installer temp root and reject empty or skipped installer evidence. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Create a fresh push event after the failed mixed-attempt rerun so the proof workflow can run as attempt 1 with matching artifact names. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Download exact native observation artifact IDs before terminal verification and use the canonical record validator at proof production. Preserve the measured history rather than accepting green-but-incomplete proof files. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Actual release wall-clock resultObserved improvement: 34m41s to 24m53s, saving 9m48s (28.26%). Both selected GitHub workflows succeeded. This is a measured read-only release-preparation comparison, not an estimate or a timed signing/publication run.
Runner elapsed time is not billable cost: OS multipliers and pricing are not applied. More parallelism and stronger source qualification buy lower latency at higher total runner time. Exact executions and clock
The primary clock is
Included: five native platforms, each side's unit/integration selections, binary builds, isolated validation, installer coverage, native archive packaging/verification, docs, wheel/sdist, and the proposed source qualification. Excluded: signing/notarization, publishing, deployment, private-service acceptance, and post-publication checks. Production integration sharding remains 1x4. Evidence recovery and fail-closed repairThe original terminal proofs verified five native archives/sidecars and the docs/wheel/sdist artifacts, but incorrectly contained zero runner records: the terminal jobs did not download the separately retained observation artifacts. The strict comparator correctly rejected those originals. The parent independently downloaded the original observation artifacts by immutable artifact ID, verified each downloaded ZIP against the GitHub API's SHA-256 digest, and checked every recovered runner record against the actual run, attempt, source, controller, and API-head identities. This recovered 9 baseline records (8 native plus the terminal record) and 15 proposed native records. The original hosted proof files were preserved byte-for-byte. Separate, explicitly labelled post-run reconciled proofs retain the original proof digest and per-artifact provenance. The unchanged strict comparator then passed, using the predeclared 14/43 required-job name sets and unmodified attempt-specific GitHub timestamps. No GitHub environment was spoofed; no missing runner record was invented. Original hosted proof artifacts: baseline The subsequent PR repair selects/downloads exact native observation artifact IDs before verification and rejects absent or malformed records at proof production, not only at comparison. This reporting-only repair follows the measured commit; it is not represented as having run in the historical 24m53s execution. No additional full run was launched to conceal the gap. That follow-up is pushed on #2879 at Scenario coverage
All 39 replaced unit identities are in Windows integration uses the existing PowerShell wrapper without JUnit passthrough; its successful job/log and recorded runner are retained, but this report does not claim exact JUnit scenario parity for that lane. Legacy baseline smoke reports contain 9 skips on each Linux/Windows build, whereas the proposed built-candidate core reports contain 7 passes per native platform; those are different selections, not falsely equivalent cases. The focused Windows root proof 34210978528 used the same built archive: the long-root installer case failed; the short-root case passed with zero skips. The earlier green-but-skipped run 34210113252 remains rejected. Expensive tests and the remaining critical pathThe proposed macOS ARM integration job takes 20m11s and finishes last. Its native gate completes at The baseline tail includes the combined macOS Intel job, then final Windows isolated validation, packaging, queued late docs/wheel builds, and the terminal verifier. The benefit is observed at the whole-workflow boundary, not calculated by adding independent lane percentages. From proposed macOS ARM JUnit, the largest families by summed testcase elapsed time are:
The clean architecture shell-wrapper case alone reports 91.378s; the generated lifecycle reference-model case reports 81.033s. These are genuine optimization targets, not permission to delete assertions, reduce examples, or share mutable fixture state. Summed case seconds overlap across workers and must not be added to wall-clock savings. Intel unit hotspots include Unix install ownership (178.595 case-seconds), checksum fixtures (157.246s), and public release metadata (138.046s), but Intel units are not the final bottleneck in the selected proposed run. Rejected executions and protocol deviations
The orchestration exceeded the declared four-run cap: six unique full-workflow run IDs and eight attempt executions occurred. It also launched an early proposed run alongside the baseline retry and an extra label run alongside the final push run. These are coordination failures, not compliant execution of the original experimental plan; no more runs are authorized. Runner images were not uniformly matched: for example, Linux ARM integration moved from |
Resolve CodeQL alert 208 by reporting in-process ordinal comparison booleans instead of SHA256 fingerprints of synthetic tokens. Keep absence, preservation, and non-disclosure assertions unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Docs sync advisoryVerdict: no_change * Pages affected: 0 * LLM calls: 1/15 No additional user-facing corpus changes are required: this candidate changes release orchestration and test tooling, not CLI/API or dependency-parser contracts. The existing contributing-guide updates are included in the expanded-scope release panel. This is documentation-impact classification, not native qualification or publication evidence. |
APM Review Panel:
|
| Persona | B | R | N | Takeaway |
|---|---|---|---|---|
| Python Architect | 0 | 0 | 0 | Final fold preserves centralized promotion, independent native qualification and separate publisher authority. No remaining architecture concern; final native evidence is pending. |
| CLI Logging Expert | 0 | 0 | 0 | Both prior logging findings are resolved: rejected JUnit retains raw diagnostics, and automatic candidate recovery no longer prescribes manual reruns. |
| DevX UX Expert | 0 | 0 | 0 | Recovery guidance and fresh/reuse planning messages resolve both prior UX findings; publication retries remain deliberately fail-closed and operator-owned. |
| Supply Chain Security | 0 | 0 | 0 | Cross-run verifier coverage is resolved; no remaining high-confidence concern in the reviewed delta. Native qualification remains pending. |
| OSS Growth Hacker | 0 | 0 | 0 | Stale anchors are resolved. Historical timing remains qualified, and the current release-authority narrative does not claim new-head production speed or approval. |
| Auth Expert | 0 | 0 | 0 | API-only caller/helper separation resolves the prior finding; default credential behavior and public dependency checks remain intact. Native qualification remains pending. |
| Doc Writer | 0 | 0 | 0 | Both original documentation findings are resolved. Recovery guidance preserves publication boundaries; final-head native qualification remains pending. |
| Test Coverage | 0 | 0 | 0 | All three prior findings are resolved; new helper and comparability regression traps are present. Parent logs support local coverage, not final-head native signing or publication. |
| Performance Expert | 0 | 0 | 0 | Both prior performance findings are resolved at 7227d25: observation assessment is linear and explicit, toy auto-workers are bounded, and historical timings remain historical. |
B = blocking-severity findings, R = recommended, N = nits.
Counts are signal strength, not gates. The maintainer ships.
Architecture
classDiagram
direction LR
class Candidate["scripts/release-candidate.cjs"]
Candidate : plan() qualify() verify()
class Native[".github/workflows/release-platform.yml"]
class Archive["scripts/package_release.py"]
Archive : package() verify_extract()
class Evidence["release-candidate-evidence.json"]
Evidence : run_id run_attempt head_sha config_hashes required_jobs platforms
class Release[".github/workflows/build-release.yml"]
class Builders["docs-build.yml / pypi-distributions.yml"]
Release ..> Candidate : promotion authority
Release ..> Native : native qualification
Release ..> Builders : read-only prebuilds
Native ..> Archive : package and checked extraction
Candidate ..> Evidence : writes and verifies JSON
note for Candidate "Functional core / imperative shell; module, not runtime class"
note for Native "Fan-out/fan-in: independent units/build; applicable results join at gate"
class Candidate:::touched
classDef touched fill:#fff3b0,stroke:#d47600
flowchart TD
E["Tag / schedule / dispatch: .github/workflows/build-release.yml"] --> P["[I/O][NET] scripts/release-candidate.cjs plan()"]
P --> R{"Reusable candidate_run_id?"}
R -->|no| U["[EXEC] release-platform.yml unit-tests / release-unit.yml"]
R -->|no| B["[EXEC] release-platform.yml build: build-binary; optional sign-binary.ps1; test_core_smoke.py"]
B --> A["[FS][NET] package_release.py package(); upload candidate-attempt-binary"]
A --> C["[EXEC] verify_extract(); full-validation integration/validation; Windows installer when applicable"]
U --> G["release-platform.yml gate: require applicable successes"]
C --> G
G --> Q["[I/O][NET] qualify(): source CI plus native evidence"]
Q -->|tag| V["[I/O][NET] verify(): source run/attempt/jobs, fingerprints, artifact IDs/digests"]
Q -->|schedule or dispatch| S["[NET] Retain attempt-scoped evidence; no publication"]
R -->|yes, tag| V
V --> K{"All checks and verifyLocalArchiveBytes() succeed?"}
K -->|no| F["Throw; failed Actions job; no publication"]
K -->|yes| W["[FS][NET] Exact assets upload; create-release"]
P -->|stable tag; PyPI also public| D["[EXEC][NET] docs-build.yml / pypi-distributions.yml read-only builds"]
W --> J{"Release and corresponding build succeeded?"}
D --> J
J -->|yes| PUB["[NET] deploy-docs / publish-pypi"]; J -->|no| N["No corresponding publication"]
Recommendation
Code advisory only: no further bounded code changes are recommended. Native run 34337498500 was pending/in progress at briefing; its success is not presumed. Prior f4 qualification 34331009260, equal-tree merge 0bb0/c862 and old CodeQL results are historical, not new-head proof. Human codeowner/last-push approval, security approval and production signing/publication authority are not conferred. The parent can append execution evidence to existing comment 5599634294.
Full per-persona findings
Python Architect
No findings.
CLI Logging Expert
No findings.
DevX UX Expert
No findings.
Supply Chain Security
No findings.
OSS Growth Hacker
No findings.
Auth Expert
No findings.
Doc Writer
No findings.
Test Coverage
No findings.
Performance Expert
No findings.
This panel is advisory. It does not block merge. Re-apply the
panel-review label after addressing feedback to re-run.
Preserve API-only Windows dependency validation and failure diagnostics; strengthen immutable promotion/signing regressions and runner observation comparability without changing native scheduling or publication authority. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

refactor(ci): parallelize native qualification and promote verified release candidates
TL;DR
Fold both release-engineering phases into this PR: parallelize qualification, test and publish the exact built archives, remove redundant source-test work, and prebuild docs/wheels without publishing privileges. Two successful read-only GitHub pipelines now show 34m41s -> 24m53s: 9m48s / 28.26% shorter, with 22.77% higher unweighted runner time. The Windows installer blocker is resolved, but production sharding remains unchanged; the earlier three sharding rehearsals remain rejected.
Important
Measured proposed source:
1687380d74cc255519b2ab93ced5ac8744fdd54b. This is an observed successful pair, not a controlled population estimate: retries exceeded the declared budget, an extra run overlapped, and images varied. Runner records were transparently reconciled from original artifacts; a subsequent fail-closed reporting repair is not included in the historical timing. Signing/publishing were excluded. The earlier 22-29m planning envelope remains withdrawn.Problem (WHY)
Motivation: the maintainer requested lower latency without weakening safeguards in an agent-developed repository with limited routine human review. Ten bounded test/CI investigations informed the changes; estimates were not treated as measurements.
Historical baseline: run 34104956270, SHA
8c2e0d9c352e2ed0e8c56b40063a63e1dd4a1937.Approach (WHAT)
Implementation (HOW)
.github/workflows/release-platform.yml,.github/workflows/release-integration.yml.github/workflows/build-release.yml,scripts/release-platforms.json.github/workflows/docs.yml,.github/workflows/docs-build.yml,.github/workflows/pypi-distributions.ymlscripts/package_release.pyscripts/release-candidate.cjs,scripts/release-candidate.test.cjs.github/workflows/ci-source-performance.yml,.github/workflows/ci-release-performance.yml.github/workflows/ci-release-rehearsal.yml,.github/workflows/release-unit.yml.github/workflows/ci-release-wallclock.yml,scripts/release_wallclock.py, corresponding unit contracts.github/workflows/ci-windows-installer-proof.yml,scripts/windows/run-installer-proof.ps1, corresponding unit contracts.github/actions/performance-cohort/action.yml,scripts/performance_cohort.py,scripts/performance-cohort.cjsscripts/release_rehearsal.py,tests/unit/scripts/test_release_rehearsal.py,tests/unit/test_release_rehearsal_workflow.pytests/unit/scripts/test_performance_cohort.py,scripts/performance-cohort.test.cjsscripts/compare_test_runs.py,scripts/pytest_performance_evidence.pyscripts/architecture_linter/checks/lexical_shared.pytests/unit/scripts/test_architecture_runner.py,tests/integration/test_architecture_owner_rule_mutations.pyscripts/windows/test-install-script.ps1,tests/utils/windows_installer_candidate.pytests/integration/test_windows_installer_launchers.py,tests/unit/test_windows_installer_candidate.pytests/unit/test_windows_installer_launchers.py,tests/unit/install/test_mcp_registry_config_layer.pytests/unit/test_console_utils.pysys.modulesafter fallback probes; Rich/Colorama identity regressions catch the shard-exposed leak.scripts/windows/test-release-validation.ps1,scripts/windows/test-dependency-integration.ps1,tests/unit/test_release_validation_prerequisites.pypyproject.toml,uv.lockpackagingdev dependency for candidate version comparison; no package-version changes.scripts/build-binary.sh,scripts/windows/build-binary.ps1scripts/test-integration.shtests/conftest.py,tests/integration/conftest.py,tests/unit/test_runtime_prerequisite_gate.pytests/integration/test_mcp_env_var_copilot_e2e.py,tests/unit/test_mcp_runtime_prerequisite.py.github/workflows/ci.yml,.github/workflows/ci-integration.yml,.github/workflows/ci-runtime.yml.github/actions/pytest-timing/action.yml,tests/unit/test_pytest_timing_workflows.py.github/workflows/merge-gate.yml,.github/scripts/ci/merge_gate_wait.sh,tests/unit/test_merge_gate_wait.py,tests/quality/test_ci_topology.pytests/integration/test_mcp_registry_e2e.py,tests/unit/test_mcp_registry_collection.pytests/unit/test_unix_installer_checksums.pypython -Sfor stdlib-only fake tools; all 102 scenarios retained.tests/unit/scripts/test_package_release.py,tests/unit/scripts/test_compare_test_runs.pytests/unit/test_platform_contract_workflow.py,tests/unit/test_release_performance_workflow.py,tests/unit/test_release_publication_overlap.pytests/unit/test_ado_live_workflow_contract.py,tests/unit/test_docs_contract_workflow.py,tests/unit/test_security_workflow_contracts.py,tests/unit/test_windows_compat_gate_workflow.pytests/unit/test_windows_signing_contract.py,tests/unit/test_windows_unit_diagnostics.py.apm/instructions/cicd.instructions.md,.github/instructions/cicd.instructions.md,apm.lock.yamldocs/src/content/docs/contributing/development-guide.md,docs/src/content/docs/contributing/integration-testing.mdReview anchors: native executor, publication, proof checker.
No application dependency or production installer interface changes. No new CHANGELOG entry: internal release/test orchestration; upstream release notes remain unchanged.
Diagrams
The artifact/authority pipeline shows independent evidence converging before publication; full/Windows-only consumers remain conditional, and prebuilding never grants publishing permission.
flowchart LR P["plan"] -->|"fresh full"| S["candidate-checks"] P -->|"fresh"| U["native unit-tests"] P -->|"fresh"| B["native build and package"] subgraph Native["Per-platform evidence"] B --> I["integration-tests-shard"] I --> F["Integration Tests fan-in"] B --> V["release-validation"] B --> W["windows-installer"] U --> G["Native Candidate Gate"] F --> G V --> G W --> G end S --> Q["candidate-ready"] G --> Q Q --> E["verify-candidate"] P -->|"trusted exact-SHA candidate"| E E --> A["10 verified assets"] A --> R["create-release"] P -->|"stable tag"| DB["build-docs-artifact"] P -->|"stable public tag"| WB["build-pypi-distributions"] DB --> DP["deploy-docs"] R --> DP WB --> WP["publish-pypi"] R --> WP classDef new stroke-dasharray: 5 5; class P,B,I,F,G,Q,E,A,DB,WB new;Trade-offs
Benefits
Validation
Current review candidate
7227d255170f29f01119651b0f17fdc4af45fd0afolds the actual panel's signing-order/cross-run regression traps, explicit API-only Windows dependency path, diagnostic retention, and observation comparability checks. Local results: 61 Node tests, 267 targeted Python tests, 88 observation tests after formatting, 24 killed/restored guard mutations, and the current seven-part lint plus architecture boundary check. Nonpublishing qualification 34337498500 succeeded: 38 required jobs, five platform archive/executable bindings, and original proof/index ZIP digests and JSON contents verified. Tested mergec5563f9ad159a06235c0de6024bb66397a7ae186and source head7227d255170f29f01119651b0f17fdc4af45fd0ahave exactly equal tree697913c4b45e72bb498fc4dc9dc82d3fddc52d68. The required gate and CodeQL succeeded. Final actual panel resolves all 15 prior items; itsship_nowstance is code advice only. Human codeowner/last-push review remains outstanding. Original timing comparability is inconclusive: six Windows observation fingerprints exactly match CRLF-rendered canonical Git bytes; no proof was normalized and no timing benefit is inferred. Prior all-five run 34331009260 remains history forf4a4a02b3and equal-tree merge0bb0f7c1c217a8b56eb1abf92e8317520c4c138d(treec862763ab944c0119688d845f13194b3ad30ae7b). Neither run proves production signing/publication or grants tagging/merge approval. Adapted #2882 owns lifecycle identity/receipt integration and the full MQ source-evidence gate; this root fold does not change that interface.Actual wall-clock comparison
Full reconciled report: clocks, artifact recovery, scenario inventories, expensive tests, rejected attempts, and protocol deviations.
f8df1b7516b76f97e511:36:14 -> 12:10:55on 2026-09-081687380d712:54:25 -> 13:19:18on 2026-09-08All 14 baseline / 43 proposed required jobs plus their terminal jobs succeeded. Both original proofs omitted separately uploaded runner records. Immutable artifact ZIP digests and recorded run/source identities were independently checked; original proofs remain unchanged, and the unchanged strict comparator passes explicitly derived reconciled proofs containing 9 baseline / 15 proposed records. Production of an empty-record proof now fails at the producer, with immutable observation downloads wired into the terminal.
The reporting-only follow-up at
aa850fabcpassed 62 targeted contracts; the candidate Node suite passed 39 cases. Canonical local lint/format/duplication and boundary guards passed. This is local evidence, not a claim that the follow-up was timed in the historical full run.All four Unix integration lanes retain every original JUnit ID with no passed-to-skipped transition and add one case each. Native unit lanes grow 22,450 -> 22,841 cases each: 39 old topology-contract IDs are replaced and 430 added; no original product-unit ID is missing. Windows integration retains job/log evidence rather than claiming JUnit parity. Proposed installer: one passed case, zero skips.
The declared four-run budget was exceeded: six full-workflow IDs and eight attempt executions occurred; mixed-attempt retries are rejected. The selected proposed run overlaps an unselected PR-label run, and Linux ARM integration images differ. No further runs were authorized in that measurement campaign. The current review qualification above was separately authorized and does not reset or retroactively accept that campaign. These limitations qualify the observed 28.26%, not the arithmetic or the fact that both selected workflows succeeded.
Earlier bounded experiments (retained, not retroactively accepted)
Hosted Windows proofs: 34151563754, 34152045787, and 34153648403. Same-image pairing, exact scenarios/outcomes and the unchanged 5% threshold passed in all three. Latest executed merge SHA:
5730579ab67fd0bf92c1a034c9940097d7ba2c1b.Source CI passed at
05c3aa563,e93483c26, and3aa6eb958. The first ARM probe failed before pytest because the benchmark upload flattenedrelease-assets/; it is excluded.e93483c26repaired the envelope and added a rejecting mutation.Both earlier ARM probes (2x2, 2x3) retained 9622 identical outcomes but mixed macOS images. They correctly failed comparison. Production was restored to 1x4; that revision's source CI passed.
The predeclared whole-rehearsal plan and live results compare ARM integration 1x4/2x3, Linux x64 integration 1x4/2x4 and Intel units 1xauto/2x4. Per-lane threshold: 15%; combined modeled DAG: 10%, target 30%; maximum three full attempts.
Attempt 1 has valid Linux evidence, but ARM image mismatch, Intel JUnit capture and Windows controls invalidated the whole result. Attempt 2 has valid ARM/Linux pairs; installer root reuse and registry cache leakage were then repaired at
de21774b4. Final attempt 34167328643, merge76f4e903604961741c34e87b8e7ff3fe21223270, independently repeats both integration wins but fails Intel's console import-state probe and the Windows candidate installer. Three attempts are exhausted; no combined DAG is accepted.The console-helper repair at
1f9a5295fhas passing source CI 34168861498, not a replacement Intel performance proof. The later same-archive Windows proof 34210978528 reproduced the long-root failure and passed the full short-root installer harness with zero skips; production now uses that short pytest-owned root without warning suppression. The earlier green-but-skipped diagnostic 34210113252 remains rejected.Hosted JUnit identifies the expensive families: owner-rule mutations 322.080 case-seconds, config lifecycle 185.565s, required lifecycle state machine 122.510s, and frontmatter/BOM 114.296s. Architecture totals 27.78% of baseline summed case elapsed, not release wall time. No cases/examples were cut. Permanent empirical report preserves the comparisons, rejections and profiling method; architecture proof includes the exact ABBA harness and raw rows.
Local commands and measured source evidence
Canonical Ruff over
src/,tests/, the architecture linter and the two new proof scripts:Canonical pylint R0801 command:
bash scripts/lint-auth-signals.sh:Focused final native, timing, auth, publication, Windows and probe contracts:
Final repair set, including both fixture-isolation leaks, actual Windows-root construction and proof-plugin tests under outer xdist:
Full corrected owner mutation matrix:
Node candidate/cohort trust: 47 passed, zero failures. YAML/2100-line/path guards, auth/architecture boundaries and actionlint passed. Source CI 34162508726 passed at
8b752a2febefore the full rehearsal.The earlier architecture full-family pair (337 cases, both failed: 212.677s -> 304.880s) is not a speed proof. The successful 343-case run proves correctness, not same-case whole-family speedup. Its JUnit attributes 97.914s of case time to transport guards, still roughly 3.0-3.4s each.
The six-case ABBA comparison invokes
run_selected_rules(...)with the same mutations and equivalent failure assertions as the integration cases. All 24 direct executions passed; the exact harness and JSONL are retained. No parser-cache redesign was added.Scenario Evidence
tests/unit/test_merge_gate_wait.py;tests/unit/test_platform_contract_workflow.py;tests/quality/test_ci_topology.pytests/integration/test_windows_installer_launchers.py;tests/unit/test_windows_installer_candidate.py;tests/unit/test_windows_installer_proof_workflow.pytests/unit/scripts/test_package_release.py;scripts/release-candidate.test.cjstests/unit/test_pytest_timing_workflows.py;tests/unit/scripts/test_compare_test_runs.pytests/unit/scripts/test_compare_test_runs.py;tests/unit/test_release_performance_workflow.pytests/unit/test_release_publication_overlap.pytests/unit/test_runtime_prerequisite_gate.py;tests/unit/test_mcp_runtime_prerequisite.pytests/unit/test_mcp_registry_collection.py;tests/unit/test_unix_installer_checksums.pytests/integration/test_architecture_owner_rule_mutations.py;tests/unit/scripts/test_architecture_runner.pytests/unit/scripts/test_performance_cohort.py;tests/unit/scripts/test_release_rehearsal.py::test_late_required_native_authorities_cannot_be_dropped_to_manufacture_a_wintests/unit/test_release_validation_prerequisites.pytests/unit/install/test_mcp_registry_config_layer.py::test_project_fixture_clears_registry_cache_when_its_config_scope_ends; original resolver default teststests/unit/test_console_utils.py::TestImportFallbacks::test_fallback_probe_restores_both_import_paths; original double-check-lock testtests/unit/scripts/test_release_wallclock.py::test_verify_rejects_missing_observations_before_emitting_a_proof;tests/unit/test_release_wallclock_workflow.pyHow to test
node --test scripts/release-candidate.test.cjs; expect fresh/warm, identity, tampering and retry negatives to pass.ci-performancefor small pairs orci-release-rehearsalfor the full DAG; require exact outcomes, matched actual runners and the documented thresholds. Remove labels before unrelated pushes.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com