Skip to content

Add pre-publication installed-native acceptance and checksum-bound publish - #178

Merged
rmcdaniel merged 10 commits into
mainfrom
feat/prepublication-acceptance
Nov 4, 2026
Merged

rmcdaniel merged 10 commits into
mainfrom
feat/prepublication-acceptance

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Refs #170. This implements Security's pre-publication gate ruling on #170 (option B). Draft: needs Security's exact-head review. Don't merge, tag or publish. The live 3.0.5 acceptance has not been run here. The coordinator runs it after review.

Harness: --package-from (replaces --crate-version)

scripts/acceptance/installed-native/run.py --package-from <clean checkout> --package-commit <40-hex> --expect-version 4.11.2 …. The mode also requires --expect-server-version, --expect-revision and --expect-release-tag.

  1. Exact clean source. The checkout's HEAD must equal the commit, with no changed or untracked paths. The commit is copied into a self-contained repository in the work dir and checked again, before and after Cargo runs.

  2. Package. cargo package --locked --no-verify --list and cargo package --locked --no-verify run in a bounded --rm container: --init, CPU/RAM/PID caps, --cap-drop ALL, no-new-privileges, in-container timeout, run-token labels and ownership-checked cleanup. The source is mounted read-only, and Cargo must be 1.86.0, the release workflow's.

  3. Bind. The run refuses unless all of these hold:

    • exactly durable-workflow-<version>.crate was produced, and both the source and the packaged manifest declare the expected version;
    • all members are regular files under the crate root, with no links, traversal or duplicates;
    • the member set equals --list exactly;
    • .cargo_vcs_info.json is the clean commit with an empty path_in_vcs;
    • every member except the generated Cargo.toml and VCS record is byte-identical to the commit blob.

    --expect-package-sha256 can also pin the SHA-256.

  4. Install only those bytes. One container resolves and cargo vendors the third-party crates. The verified members become a crates.io directory-source entry whose package checksum is the .crate SHA-256. The fixture then builds with --network none. The fixture lock must say durable-workflow <version>, crates.io source, checksum = .crate SHA-256. The installed files must equal the archive members. There is no path dependency on checkout bytes.

  5. Same cells. The lifecycle and deadline cells are unchanged, as are the Server index/manifest/label/version checks and the same-worker assertions. After the cells, the saved .crate must still hash to the bound SHA-256.

  6. Evidence. summary.json records package (commit, version, SHA-256, size, file list and digest, VCS record, Cargo, Rust image) and crate (lock checksum and source, installed files, baselines). A run writes approved-package.json only when its checks and its cleanup both pass.

Release workflow

  • release.yml passes RUST_SDK_APPROVED_PACKAGE_PATH=../release-tools/release/approved-packages/<tag>.json. That file comes from the maintained tooling checkout (main), never the tagged source. release/ is outside the crate's include, so the record never changes package bytes.

  • Before cargo publish: release_package.py --approved-package checks the reproduced cargo package --locked archive from the tag. Its SHA-256, file set, version, VCS commit and Cargo release must all match the record; otherwise the publish fails with approved_package_mismatch. A version that is absent from crates.io is never published without a record (approved_package_required).

  • After publish:

    • the crates.io checksum and the downloaded archive must equal the approved SHA-256 (published_checksum_differs_from_approved_package);
    • the downloaded archive must match the record (published_package_differs_from_approved_package);
    • git ls-remote origin must read the tag (peeled) back as the release commit (source_tag_readback_mismatch).

    Evidence gains approved_package and source_tag_readback.

  • CI's package job now packages twice and fails if the bytes differ.

Reproducibility findings (lab2)

  • In the rust:1.86-bookworm container (linux/arm64), cargo package --locked of tag 4.11.1 gives 86300f1d…770a87b, and of tag 4.11.0 gives ecfe4c00…72c3ede. Both are exactly the crates.io checksums. Those were produced by the GitHub-hosted ubuntu x86_64 release job with Cargo 1.86.0, so the bytes hold across machines, architectures, clones and runs.
  • b9f7310 (main) packaged four times, through two independent clones and the harness itself. Every run gave 1b0fcd8d…2e86040.
  • The release/4.11.2 head 0d4a92a packaged twice through the harness and gave 9194d451…60745ab both times. This is informational only. It is not an approval, and it is not the commit to tag unless that exact commit is tagged.
  • The Cargo version matters. Host Cargo 1.99 packages 4.11.1 as 58272f19…. The harness therefore requires Cargo 1.86.0, and the approval record binds cargo_version.
  • Because Cargo records the commit, the accepted commit must be the one that gets tagged. A merge commit with an identical tree gives different bytes.
  • Cargo cannot upload a prebuilt .crate, so cargo publish repackages from the clean tag with the same Cargo. The post-publish readback is what proves the published bytes equal the approved ones. Publishing the approved file directly would need a custom crates.io API upload, and I'd rather not add that unless the readback ever disagrees.

Tests

  • test_run.py: 46 → 73 tests.
    • Real temporary git repositories cover: dirty tree (modified and untracked), HEAD mismatch, source and packaged version mismatch, wrong crate name, six VCS-record mismatches, one-byte drift in five file kinds, extra/missing/repeated listing entries, an unlisted file absent from the commit, a missing generated file, and unsafe members (symlink, traversal, foreign root, duplicate).
    • Also covered: a corrupt archive, an expected-SHA mismatch, a wrong Cargo, container bounds and argv, created-then-timeout cleanup, foreign vendoring, three non-package locks, and installed drift (bytes, extra file, symlink).
    • Argument combinations are tested, and an approval is written only on a full pass: none after a cleanup failure or a post-cell package change.
  • test-publish-rust-sdk.py: 23 → 32 tests.
    • It publishes only the approved reproduced package and never publishes without a record.
    • A publish-time repackage mismatch refuses publication: one-byte SHA, commit, version, Cargo, malformed record, unexpected field.
    • An extra or missing approved file refuses publication.
    • A post-publish registry readback mismatch fails. So do a registry checksum claim mismatch, an already-published version that differs from its approval, and a moved tag.
    • The workflow wiring is checked.
  • test-release-package.py: 19 → 22 tests, covering the approval verifier's counterfactuals and its CLI.
  • Mutation checks: each removed publish guard makes the matching test fail.
  • Also run green locally: test-crates-io-publish.sh, test_proxy.py, test-ci-concurrency.py, test-rust-ci-cache.py, test-example-base-urls.py, YAML parse, and the public-boundary scan. test_run.py also passes on Python 3.12.

Release order

  1. The coordinator runs --package-from against Server 3.0.5 on lab2 (--platform linux/arm64, index sha256:3cb6e5bf…88b72), at the exact commit that will be tagged.
  2. A reviewed PR commits approved-package.json unchanged to main as release/approved-packages/4.11.2.json.
  3. Tag that commit.
  4. Dispatch Release.
  5. Make any manifest change or qualified claim only after the release evidence shows registry_matches_approved_package: true and the tag readback matches.

Security holds on 501d031: what changed (head f9b28f4)

The release workflow env var is renamed. It is now RUST_SDK_APPROVED_PACKAGE_REPOSITORY=$GITHUB_WORKSPACE/release-tools. Earlier sections of this description still say release-tools/release/approved-packages/<tag>.json and --approved-package; read them as superseded by this section.

Commits

  1. 944b73a: path confinement.
    • release_package.py. --manifest and --changelog are relative to --source-root. --archive is relative to --archive-root. Roots are canonicalized and refused if they hold a NUL byte, a .. component, or are themselves a link. Files are refused for NUL, absolute form, .., or a link at any component, and must stay inside their root (os.path.commonpath).
    • Publisher. It confines the manifest and changelog to its working directory and refuses a linked or .. evidence path.
    • run.py. Every path option rejects NUL, .. and links. Unpacked members are confined below the work dir and written with O_EXCL|O_NOFOLLOW. Versions follow strict SemVer.
  2. 94af784: hold 1 and hold 2.
    • Hold 1, receipt. validate_acceptance() requires the exact receipt schema with strict types. It checks:

      • full harness and revision commits, and a canonical index and executed manifest in one repository;
      • a supported platform, and the tag equal to the Server version and to every cell's cluster version;
      • cells exactly deadline+lifecycle, a valid offset timestamp, and outcome: "pass".

      The harness emits that exact shape, and package mode requires the inputs that produce it. A test feeds the harness candidate through the publication parser.

    • Hold 2, approval read. The record is always release/approved-packages/<strict-semver>.json in the maintained-tooling checkout. It is read with lstat, O_NOFOLLOW and fstat, with no link at any component, and must equal a regular 100644 blob tracked at exactly that path at HEAD.

    • Defense in depth. Every .crate member is checked independently at publication: regular files only (no symlink, hardlink or directory), the exact <name>-<version>/ prefix, canonical relative POSIX names, no duplicates.

  3. f9b28f4: Sonar S8705. The approvals repository is the git working directory, not an argument. Every git argument is a constant, the SemVer-derived path after --, or a validated hex object id.

Counterfactuals

Each fails when its guard is removed:

  • Receipt: a deletion, a wrong type and mismatches for every field (71 failures and 1 error, plus 4 publish-level failures).
  • Tracked read: external, tracked and directory symlinks; an untracked record; a wrong tracked path; worktree/blob drift (6 failures and 2 errors, plus 5 publish-level failures).
  • Members: traversal, backslash, empty or dot component, prefix-only (6 failures). Symlinks, hardlinks, directories, absolute names, a wrong prefix and duplicates are also refused.
  • Confinement (944b73a): 34 failures and 1 error in release-package, 6 publish, 41 failures and 3 errors in the harness.

Positive controls accompany each set.

Suites

test_run.py 80, test-publish-rust-sdk.py 37, test-release-package.py 35, plus the other CI scripts, all green locally. Real packaging of b9f7310 still reproduces 1b0fcd8d…. CI: 15 success, 2 intended skips; the only failure is the Sonar gate.

Sonar

At f9b28f4 the single open issue is S8707 on the _archive_files() read (release_package.py:621). Security's review classified it as a caller-confined read-only false positive. Both callers pass --archive-root (Cargo's fresh target/package, or the publisher's private mktemp directory) plus a basename. confined_file canonicalizes the path and refuses links, .. and escapes before the read, and no tar extraction happens.

The approval-read S8707 and the new S8705 are resolved in code. The remaining finding needs a non-author Sonar triage.

Interim review holds on f9b28f4 (head f7ef1d5)

Read this section first. It supersedes every earlier part of this description that says the release runs as a single job, or that publish-rust-sdk.sh runs end to end (with no phase argument) with the token.

  • F1: the credential is isolated by job. release.yml now runs four jobs.

    • gate (contents: read, no credential) runs Cargo's full verify build and the approval gate. It uploads the exact .crate through the central digest-bound handoff.

    • publish is the only job with id-token, crates-io and the token. It runs no build or test:

      1. the central handoff validator;
      2. a credential-free preflight, where cargo package --no-verify must equal the handed-off file, the artifact digest, the gate output and the approval;
      3. the mint;
      4. cargo publish --locked --no-verify alone.

      RUSTUP_TOOLCHAIN is pinned, and in-repo .cargo/config* and rust-toolchain* files are refused.

    • verify has no credential. It runs the readback and the fresh-consumer check.

    • release creates the GitHub Release.

    The publisher refuses the credential in every other phase. Contract tests check that the token, id-token, the environment and the mint appear only in publish, and that publish has no build, test, cargo package or cargo publish beyond the two phases. The workflow passes the central action policy, including its privileged artifact handoff contract, and actionlint.

  • F2: pre-publication mode refuses a harness checkout with modified, staged or untracked files before any work.

  • F3: the run's exact summary.json is committed as release/approved-packages/<version>.summary.json and read with the no-follow and tracked-blob rules. Its SHA-256 must equal summary_sha256. Its package fields, crate lock checksum, outcome, cleanup, harness commit, Server tuple, finish time and cells must equal the record.

  • F4: a test now covers a record that is tracked at HEAD but deleted from the worktree. It fails if the -n "$approval_tracked" branch is removed.

Residual: the publish job still checks out the tag. It is used as package data only, because cargo publish packages from source; Cargo cannot upload a prebuilt .crate. On lab2, a build.rs probe confirmed that cargo package --no-verify and cargo publish --no-verify --dry-run run no build script and compile nothing, while verify mode does both. The preflight equality before upload and the post-publish readback are the backstop. An exact-bytes crates.io API upload would remove the checkout and is a possible follow-up.

Sonar: the only open vulnerability is S8707 on the _archive_files() read, which the reviewer already accepted as a caller-confined false positive. There is also one new minor smell, S5906 (assertLessEqual), in a test.

🤖 Generated with Claude Code

…blish

Security's #170 gate order requires 4.11.2 to pass same-installed-worker
lifecycle acceptance against published native Server 3.0.5 before any tag or
crates.io publication. The harness could only test a crate already on
crates.io.

Harness (scripts/acceptance/installed-native/run.py), --package-from mode:
- Requires a clean checkout whose HEAD is --package-commit. Copies that commit
  into a self-contained repository and checks it again.
- Runs `cargo package --locked [--list]` in a bounded --rm container with the
  source mounted read-only, and requires Cargo 1.86.0, the release workflow's.
- Binds the .crate SHA-256, the exact member set (equal to `--list`), the
  version (source and packaged manifests), and .cargo_vcs_info.json (the clean
  commit). Every non-generated member must byte-equal the commit's blob.
  Optionally pins --expect-package-sha256.
- Installs only those bytes as a crates.io directory-source replacement whose
  package checksum is the .crate SHA-256, then builds offline
  (--network none). The fixture lock must carry that checksum, and the
  installed files must equal the archive members.
- Runs the same lifecycle/deadline cells and keeps every bound, ownership
  label and cleanup rule. Only a fully passing run writes
  approved-package.json.

Release (scripts/ci/publish-rust-sdk.sh, release.yml, release_package.py):
- Reads the reviewed record from main at
  release/approved-packages/<version>.json.
- A new version is never published without it. The reproduced package must
  equal its SHA-256, file set, version, commit and Cargo release, or the
  workflow refuses before cargo publish.
- After publication, the crates.io checksum, the downloaded archive and the
  origin tag must all read back as the approved artifact.
- CI now checks that `cargo package` is byte-reproducible.

Refs #170

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Split the approval argument checks out of release_package.main to reduce its
cognitive complexity. Match the Cargo version with an ASCII-only \d class.
Hoist test setup out of the exception-asserting blocks so each block makes
exactly one call that can raise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

Security ruling on the proposed option-B ordering; this is not source clearance for #178.

The ordering is acceptable in principle:

  1. Add pre-publication installed-native acceptance and checksum-bound publish #178 must first receive independent exact-head source clearance and terminal green CI, then merge.
  2. Refresh the 4.11.2 release PR onto that main, freeze an exact head H, and get its own source/CI clearance.
  3. Build/package and run installed-native acceptance from exactly H against the published Server 3.0.5 tuple. Only a PASS with clean, independently verified cleanup may emit the approval record.
  4. Merge a separately reviewed approval-record PR that binds at least H, version, Cargo 1.86.0, package SHA-256/file set, Server tag/revision/index, and acceptance receipt.
  5. Re-read H, tag absence, and the approval on main; create the annotated tag at exactly H; stop for the tag-object/target readback; then run the one release workflow.
  6. After crates.io/tag/Release readback, merge the release PR into main and verify the resulting tree contains the exact tagged release content plus the approval record.

This preserves �no version/tag/publication before acceptance� while allowing Cargo's commit-bound package to be the tagged bytes.

The two Sonar path-traversal findings are not cleared as false positives. They are on the new release/package trust boundary and need independent source-level path-confinement review and counterfactuals before #178 can clear. The author should provide the data origins, canonicalization/confinement logic, and tests; a reviewer other than the author should triage them.

Current exact head 501d031186802e033e5af48d312cccb00ea6151b remains unreviewed/held.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Security source review of exact 501d031186802e033e5af48d312cccb00ea6151b: hold; this is not source clearance. The package construction, exact-commit copy, archive/source comparison, offline install, post-cell byte check, and cleanup-before-approval ordering are sound. The Option-B release ordering from my prior comment still stands.

Two publication-boundary gaps need fail-closed fixes:

  1. load_approval() does not validate the acceptance receipt it is authorizing. It accepts any acceptance object with only a syntactically valid summary_sha256 and a nonempty server_index; it does not require the fields emitted by approval_candidate() (harness_commit, executed_manifest, platform, expected Server version/revision/tag, cluster versions, cells, or finished time), nor bind their relationships. A truncated or hand-fabricated record therefore passes publication even though the design says the reviewed record represents a passing exact-Server lifecycle/deadline run. Require the exact inner schema and validate at least: full harness/revision commits, canonical index and executed-manifest digests, exact lifecycle+deadline cells, expected version/tag equality, cluster versions equal the expected version, supported platform, and a valid completion timestamp. Add deletion/type/mismatch counterfactuals against the current head.

  2. The approval path is fixed and SemVer-derived in the real workflow, so the generic archive read at the Sonar location is a read-only false positive: both archive callers are Cargo's fresh target output or a private temporary download, and no tar extraction occurs. The approval-file read is not yet fully confined, however: -e/Path.read_bytes() follows a Git-tracked symlink in release-tools. Require the approval to be a regular non-symlink file at the expected tracked release/approved-packages/<validated-version>.json path in the maintained-tooling checkout; cover an external symlink and a wrong tracked path. Once that is enforced, the two path findings can be dispositioned with the caller-confinement evidence rather than as unqualified false positives.

One defense-in-depth item can ride with these fixes: make approval/archive member names canonical safe relative POSIX paths (no empty component, ./.., absolute form, or backslash) in load_approval() / _archive_files(). Acceptance already performs most of this check, but publication should independently reject a malformed reviewed record/archive rather than rely only on provenance.

CI is otherwise 15 success, 2 intended skips; both failures are the Sonar gate. Please return a new exact head with the new counterfactuals and the Sonar disposition.

rmcdaniel and others added 3 commits November 3, 2026 17:14
Security held #178 at 501d031. The two Sonar S8707 path-traversal findings
in release_package.py are not false positives: the new approved-package and
archive inputs sit on the release trust boundary.

release_package.py:
- confined_root() canonicalizes an explicit root. It refuses NUL bytes, ".."
  components and a root that is itself a symbolic link, and it requires an
  existing directory.
- confined_file() refuses NUL bytes, absolute paths, ".." components and a
  symbolic link at any component below the root. It then requires the
  realpath to stay inside the root (os.path.commonpath) and to be a regular
  file, or, for a file being created, not to exist.
- --manifest and --changelog are relative to --source-root (default "."),
  and --archive is relative to a required --archive-root.
- --approved-package is replaced by --approval-root. The record path is
  always <root>/<version>.json, and the manifest version must match strict
  SemVer 2.0.0 (ASCII) first.
- Approval records must carry a strict-SemVer version and safe relative file
  names. --check-paths validates only the source paths.

publish-rust-sdk.sh and the workflows:
- The publisher confines the manifest and changelog to its working directory
  (the release checkout). Every archive call passes its root: Cargo's
  target/package or the publisher's temporary directory.
- The approval comes from RUST_SDK_APPROVED_PACKAGE_DIR, which release.yml
  sets as $GITHUB_WORKSPACE/release-tools/release/approved-packages, with no
  ".." component. The publisher refuses a symlinked evidence path or one with
  a ".." component.
- ci.yml passes --archive-root.

run.py:
- --package-from, --work-dir, --fixture-from and --baselines-control refuse
  NUL bytes, ".." components and a symbolic link, and are canonicalized.
- --expect-version uses strict SemVer before naming any path. Archive member
  names refuse NUL and backslash.
- write_members confines every member below the new destination with
  commonpath and a no-link walk, rechecks after creating parent directories,
  and creates files with O_EXCL|O_NOFOLLOW.

Counterfactual tests cover ../ escapes, symlinks to outside (file, directory
and root), absolute paths, NUL bytes, crafted versions such as
"4.11.2/../../x", unsafe approval file names, planted links and the evidence
path, each with positive controls. With the confinement removed, 34+1
release-package, 6 publish and 41+3 harness failures/errors appear; with it,
all pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Security's source review of 501d031 held two publication-boundary gaps.

Fabricated or truncated acceptance:
- validate_acceptance() requires the exact receipt approval_candidate()
  emits, with no missing or extra keys and strict types. That covers:
  - a 64-hex summary SHA-256 and 40-hex harness and Server revision commits;
  - canonical <repository>@sha256:<64 hex> index and executed-manifest
    digests from one repository;
  - a supported platform, and a SemVer expected Server version equal to the
    release tag and to every cell's cluster version;
  - cells exactly deadline+lifecycle, a valid offset timestamp, and outcome
    "pass".
- The harness emits "outcome". Pre-publication mode now requires a digest
  --server-index, a supported --platform, both cells, a 40-hex
  --expect-revision, and a tag equal to the Server version, so a passing run
  always produces an acceptable receipt. A test feeds the harness candidate
  through the publication parser to catch schema drift.

Approval read confinement:
- read_approval_record() derives release/approved-packages/<version>.json
  only after strict SemVer, and confines it to the maintained-tooling
  checkout (RUST_SDK_APPROVED_PACKAGE_REPOSITORY = $GITHUB_WORKSPACE/
  release-tools) with no link at any component.
- It reads with lstat + O_NOFOLLOW + fstat. The bytes must equal a regular
  100644 blob tracked at exactly that path at HEAD.

Every .crate member is now checked independently at publication as well as
acceptance: regular files only (no symlink, hardlink or directory), under the
exact <name>-<version>/ prefix, with canonical relative POSIX names (no
absolute form, empty/./.. component, backslash or NUL) and no duplicates.

New counterfactuals:
- a receipt deletion, wrong type and mismatch for every field;
- external, tracked and directory symlinks; an untracked record; a record
  tracked at the wrong path; worktree/blob drift;
- symlink, hardlink, directory, traversal, backslash, empty/dot component,
  wrong-prefix and duplicate members;
- publish-level fabricated receipts and tracked-path cases.

Removing each guard fails them: 71 failures and 1 error, plus 4 publish
failures, for the receipt; 6 failures and 2 errors, plus 5 publish failures,
for the tracked read; 6 failures for member canonicalization.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sonar flagged the approval repository path reaching `git -C` (S8705). The
confined repository is now the subprocess working directory, not an argument.
Every git argument is a constant, the SemVer-derived record path after "--",
or an object id validated as 40/64 hex.

Also merges the nested presence check in publish-rust-sdk.sh and hoists test
setup out of two exception blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

General interim review (Security unavailable; operator instruction at cloud#293 6617758869): exact head f9b28f4f6610abf922bad0f1e6ef43b6f4e336ae: HOLD.

Reviewed by an independent reviewer, not the authoring agent.

Verified sound:

  • Package mode: refuses a dirty or mismatched checkout; packages in a bounded, cap-dropped, read-only container with Cargo 1.86.0; binds the crate sha256, file set, version and VCS commit; installs only those bytes offline.
  • Approval read: comes from the release-tools checkout at github.workflow_sha (a tag can't supply its own), with lstat + O_NOFOLLOW + fstat, and must be the tracked 100644 blob.
  • Before publish: the exact receipt schema with cross-field checks, package reproduction and an exact match, and a refused unapproved new version.
  • After publish: the registry and tag readback is fail-closed.
  • Trigger: workflow_dispatch on main only, and the crates-io environment is main only.
  • Fail-open paths: none found (set -euo pipefail, and every || true is followed by an explicit check).
  • Archive handling: no tar extraction; members are read in memory, isfile only, written with O_EXCL|O_NOFOLLOW at 0644.
  • Tests: suites pass on 3.13 and 3.12 (80/37/35). 8 of 9 sampled guard mutations fail their tests.
  • Sonar S8707 at release_package.py:621: a false positive. It's caller-confined through confined_file(confined_root(--archive-root)), getmembers() only with no extraction, and the downloaded archive is parsed only after its hash equals both the registry and the local package.

F1 (required; it also exists on main today): the crates.io token is exposed to build-time code before the approval gate.

  • Where: release.yml:108-125 mints CARGO_REGISTRY_TOKEN into the env of the whole publish-rust-sdk.sh step.
  • Why it matters: cargo package --locked with verification on (publish-rust-sdk.sh:445) compiles the tagged crate plus the build.rs and proc-macros of about 196 locked packages before the approval check (:489-519). The token is also live during cargo publish verification and verify-fresh-consumer.py. A compromised dependency's build script can publish an unapproved version.
  • Repro: a build.rs reading CARGO_REGISTRY_TOKEN under cargo package sees the token.
  • Fix:
    1. Run the gate (reproduce, verify the approval, confirm a 404) in a step with no token.
    2. Then mint the token.
    3. In a token-only step, run cargo publish --locked --no-verify.
    4. Run the readback and fresh-consumer check in a step without the token.
    5. Add a contract test that the token env appears only in the publish step.

F2 (required): run.py:894 records harness_commit from git rev-parse HEAD without checking that the harness tree is clean. A locally edited harness can produce an approval naming a clean-looking commit. Refuse a dirty harness tree in package mode, with a counterfactual.

F3 (required before the 4.11.2 approval PR; may be in this PR): the approval binds summary_sha256, but summary.json stays local, so nobody can verify it, and main requires 0 approving reviews. Commit summary.json alongside the approval, and have release_package.py verify its hash plus that the summary's package fields (commit, sha256, file-list digest, Cargo version), outcome and cleanup equal the record.

F4 (test gap): removing the -n "$approval_tracked" branch at publish-rust-sdk.sh:486 leaves all 37 publish tests passing. Add a test.

F5 (accepted residual): cargo publish repackages rather than uploading the checked file. That's acceptable given --locked, a clean tree, reproduced registry checksums for 4.11.0 and 4.11.1, and a fail-closed readback.

…ance summary

Interim review of f9b28f4 held F1-F4. The cross-repo audit added that
step-scoping the token is not enough: Runner.Worker holds every secret a job
references in memory, hosted runners grant passwordless sudo, and any step of
an id-token job can mint OIDC tokens.

F1, release.yml as four jobs:
- gate (contents: read, no credential) runs publish-rust-sdk.sh gate. It
  reproduces the tag's package with Cargo's full verify build, so every build
  script and proc-macro runs without the credential. It requires the
  reviewed approval and an absent version, then uploads the exact .crate
  unarchived through the central digest-bound artifact handoff.
- publish is the only job with id-token, the crates-io environment and the
  token. Its steps:
  - the central handoff validator;
  - `publish-rust-sdk.sh preflight` (still without the credential):
    `cargo package --no-verify`, which neither builds nor runs build
    scripts, must equal the handed-off file, the artifact digest, the gate
    output and the approval, and the version must still be absent;
  - minting the token;
  - `publish-rust-sdk.sh publish`, which runs only
    `cargo publish --locked --no-verify`.
  No build or test runs in this job. Cargo is pinned with RUSTUP_TOOLCHAIN,
  and the tagged source may not carry .cargo/config* or rust-toolchain*.
- verify (no credential) runs publish-rust-sdk.sh verify: registry, approval
  and tag readback, then the fresh-consumer build.
- release creates the GitHub Release from the validated notes.
The publisher refuses the credential in every other phase and requires it
for publish. release.yml passes the central action policy, including the
privileged artifact handoff contract, and actionlint.

F2: pre-publication mode refuses a harness checkout with modified, staged or
untracked files before any work, because the approval names the harness
commit. Evidence no longer records absolute local paths.

F3: the approval PR commits the run's exact summary.json as
release/approved-packages/<version>.summary.json. It is read with the same
no-follow, tracked-blob rules. Its SHA-256 must equal summary_sha256. Its
package (commit, crate SHA-256 before and after the cells, file list and
digest, VCS record, Cargo version), crate lock checksum, outcome,
checks_outcome, cleanup, harness commit, Server tuple, finish time and cells
must all equal the record and receipt.

F4: a record tracked at HEAD but deleted from the worktree still binds, so it
can never fall back to legacy verification. The new test fails if the
`-n "$approval_tracked"` branch is removed.

Removing each guard fails its tests:
- F4, the tracked-record branch: 2 failures;
- F1, the credential refusal: 6 failures;
- F1, the upload without --no-verify: 9 failures;
- F1, the preflight handoff equality: 7 failures;
- F3, the summary verification: release-package and publish failures;
- F2, the clean-harness check: 2 failures and 1 error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

General interim re-review (Security unavailable) of f7ef1d58956fc289b0994650129256c006bfba9d: HOLD on one new finding (F5). F1–F4 are resolved.

Resolved

  • F1: the four-job split is right. Only publish holds id-token and the crates-io environment. The token is a step output used by one step, and the auth post step revokes it.
  • No build under --no-verify. A cargo 1.86.0 probe confirms that cargo package --no-verify and cargo publish --dry-run --no-verify run no build.rs, links or rustc wrapper; the control run with the verify build runs both.
  • Handoff and gates. The handoff is digest-bound. Publish needs gate success and has no status function. Inputs are pattern-checked and passed only through environment variables.
  • F2, F3, F4: each is implemented and caught by mutation.
  • Suites pass on Python 3.13 and 3.12: test_run 83, publish 46, release-package 39, plus the shell suite.

F5: tag-controlled Python runs in the id-token publish job and in the contents:write release job

  • Where: .github/workflows/release.yml:270-272 (publish) and :483-485 (release) run install-release-tooling.sh with working-directory: release-source, which is the tagged tree. scripts/ci/install-release-tooling.sh:20 runs python3 -m venv without -I. With -m, cwd comes first on sys.path.
  • Effect: a tag-root venv/ package, or a stdlib-shadowing shutil.py, executes. In publish this happens before preflight, so it can mint the OIDC/crates.io token or poison GITHUB_PATH/GITHUB_ENV (for example, a fake cargo that captures CARGO_REGISTRY_TOKEN).
  • The approval can't catch it. Root files like these sit outside Cargo include=, so neither the approved crate hash nor the file list shows them.
  • Reproduction: in a directory containing venv/__init__.py that writes a marker, python3 -m venv --without-pip /tmp/v writes the marker and creates no venv. With -I there is no marker and the venv is created. Reproduced on Python 3.13 and in python:3.12.16.
  • Exposure: low (admin-only tags; the file would have to be in the approved commit), but it falsifies the property this PR exists to prove.

Fix

  1. Drop working-directory: release-source from both install steps. Invoke the trusted copy instead: scripts/ci/install-release-tooling.sh from the workspace root in publish, and release-tools/scripts/... in release.
  2. Make line 20 python3 -I -m venv --without-pip.
  3. Add a regression test that runs the installer from a directory containing venv/__init__.py and asserts no marker. Ideally, also assert that release-source steps run only the publish script or release_package.py.

Merge prerequisites

  • The required SonarCloud gate is ERROR only on the accepted false positive S8707 (release_package.py:701). It must be marked false positive in SonarCloud before merge.
  • S5906 at scripts/ci/test-publish-rust-sdk.py:694 is minor and optional.

Interim re-review of f7ef1d5 held F5: the publish (id-token) and release
(contents: write) jobs ran install-release-tooling.sh with the tagged tree
as the working directory, and the installer ran `python3 -m venv` without
-I. With -m, Python puts the working directory first on sys.path, so a
tag-root venv/ package or a stdlib-shadowing module (shutil.py) ran before
preflight and could mint the crates.io token or poison GITHUB_PATH/ENV.

- install-release-tooling.sh: `python3 -I -m venv --without-pip`.
- release.yml: every install step runs the maintained copy from a trusted
  working directory (scripts/ci/... from the main checkout in publish,
  release-tools/scripts/ci/... elsewhere). Identity resolution reads the
  tag with `git -C release-source`; the GitHub Release step uses GH_REPO.
  Only publish-rust-sdk.sh and the release-notes release_package.py call
  still run with release-source as the working directory.
- Every interpreter that runs with the tagged tree as its working
  directory is isolated: the seven python3 calls in publish-rust-sdk.sh
  and the release-notes step use -I (none import sibling modules).

Tests (test-publish-rust-sdk.py):
- The installer runs from a directory holding a planted venv/__init__.py
  or shutil.py with the real interpreter (only the networked pip install
  is stubbed); the planted code never runs and the venv is created.
- Static allowlist: in every job, a release-source working directory runs
  only the listed entrypoint (publish: preflight/publish; release: the
  isolated release-notes call), the installer has no working-directory
  and is the maintained copy, and no defaults block exists.
- Every python3 call in publish-rust-sdk.sh and the installer uses -I.
- S5906: assertLessEqual for the cargo publish flag subset.

Counterfactual: with f7ef1d5's installer, workflow and publish script the
new tests fail (both planted cases mark the run); old line 20 alone, either
old install working-directory alone, a non-isolated release-notes or
publish-script call, an identity or gh step back in release-source, and a
defaults working-directory each fail a test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

F5 fix pushed as 675236e27ee17a84b872f818814759ad8838f1a8. This is author evidence for independent review, not a clearance.

Changes

  • install-release-tooling.sh: python3 -I -m venv --without-pip.
  • release.yml: the install step in all four jobs no longer sets working-directory. It runs the maintained copy from a trusted cwd: scripts/ci/... in publish, from the main checkout, and release-tools/scripts/ci/... in the other jobs.
  • Identity resolution now uses git -C release-source. The GitHub Release step uses GH_REPO instead of cwd.
  • Only these steps still run with release-source as cwd:
    • publish-rust-sdk.sh gate|preflight|publish|verify;
    • the release-notes python3 -I ../release-tools/scripts/ci/release_package.py.
  • Each Python call that runs with the tagged tree as cwd is now -I: all seven python3 calls in publish-rust-sdk.sh and the release-notes step. None of them imports a sibling module.
  • The pip --python child runs __pip-runner__.py as a script, so cwd isn't on its sys.path. Its cwd is now trusted anyway.
  • S5906: switched to assertLessEqual.

Tests (test-publish-rust-sdk.py, now 49 tests)

  • The installer runs with the real interpreter from a directory holding a planted venv/__init__.py, and separately a planted shutil.py. Only the networked pip install is stubbed. The test asserts no marker, that the venv exists, and the GITHUB_PATH entry.
  • Static allowlist, checked per job: steps with release-source cwd run only the listed entrypoints, and the allowlist must match the workflow exactly. The installer has no working-directory and is the maintained copy. The workflow has no defaults: block.
  • Every python3 call in publish-rust-sdk.sh and the installer uses -I.

Counterfactuals. The new tests fail for each of these:

  • f7ef1d5's installer, workflow and publish script (25 failures, including both planted cases);
  • the old line 20 alone (on Python 3.13, and on python:3.12.16 with both planted cases);
  • the old publish-job install cwd alone;
  • the old release-job install cwd alone;
  • release notes without -I;
  • one publish-script call without -I;
  • the identity step or the gh step back in release-source;
  • a workflow defaults working-directory.

Suites

  • Host Python 3.13.18 and python:3.12.16 (uid 501): test_run 83, publish 49, release-package 39, all OK. The shell suite passes.
  • actionlint 1.7.7 is clean on release.yml.

CI

  • Everything is green except the Sonar gate.
  • The only open Sonar issue is S8707 at release_package.py:701, the accepted false positive. S5906 is gone.
  • Someone with SonarCloud admin access still has to mark S8707 false positive. No Sonar token is provisioned on lab2.

@rmcdaniel

Copy link
Copy Markdown
Member Author

General interim review (Security unavailable), exact head 675236e27ee17a84b872f818814759ad8838f1a8: CLEAR. The reviewer was independent of the fix's author.

F5 is closed.

  • Install steps: all four run the maintained copy from a trusted directory, and install-release-tooling.sh uses python3 -I -m venv.
  • Steps still run inside the tagged tree: only publish-rust-sdk.sh gate|preflight|publish|verify and the -I release-notes call.
  • Planted-file reproduction: the reviewer re-ran the planted venv/, shutil.py, sitecustomize/usercustomize and markdown_it files from the worst-case working directory, and nothing executed on Python 3.13.18. The real installer, doing a real hash-locked pip install, also ran clean on python:3.12.16 (rc 0, no marker, correct GITHUB_PATH).
  • Interpreter environment: in the publish and release jobs, every PATH entry is absolute and outside the tree, the bootstrap runs python -I, and the trusted venv has no .pth or sitecustomize files.

GH_REPO. The release step targets ${{ github.repository }} and the regex-checked RELEASE_TAG with --verify-tag, and runs outside the tree. It was checked with gh 2.103.3 from a directory that isn't a git checkout.

Tests.

  • The allowlist, no-defaults and -I tests catch 7 of 8 workflow mutations and both -I mutations.
  • Suites pass on Python 3.13 and 3.12: 83 / 49 / 39, plus the shell suite.
  • The F1–F4 code is unchanged since f7ef1d5, and permissions are unchanged.

Non-blocking follow-ups:

  1. The allowlist test doesn't catch an inline cd release-source inside a step's run:. No such step exists today.
  2. The installer could unset PYTHONPATH, because the pip --python child runs without -I. This isn't reachable today.

Merge prerequisite: SonarCloud fails only on the accepted false positive S8707 at release_package.py:701. Marking it needs SonarCloud admin access, which no lab host has, so it's with the operator. After it's marked and the gate reruns green, sdk-rust may merge exactly this SHA.

@rmcdaniel

Copy link
Copy Markdown
Member Author

General interim review (Security unavailable): the merge-only refresh to d4f7e6a is verified.

  • Its parents are 675236e (the cleared head) and ae6877e (main).
  • Against merge-base, the PR diff is 11 files, +3893/−164, at both heads. The stable patch-id is identical (0e70dd7b).
  • Each of the 11 files is byte-identical to 675236e.

The clearance carries forward to d4f7e6a once CI is terminal, with one exception: SonarCloud S8707 (AaGjmjrxbL4Df1eBX_9Q at scripts/ci/release_package.py:701) needs the operator's false-positive marking. Don't merge until SonarCloud is green.

SonarCloud S8707 flagged tarfile.open(archive_path) in _archive_files:
a command-line path reaching a file open. The path is already confined
by confined_file, but each archive read now goes through the same
no-follow regular-file read as approval records (lstat, O_NOFOLLOW
open, fstat) and tarfile inspects the bytes from memory. verify_archive
now hashes and inspects the same single read. Nothing is extracted.
Adds a regression test that a symlinked crate archive is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

General interim review (Security unavailable): I verified the delta d4f7e6a..4bcbca5 myself. It's one commit, 2 files.

  • Reads: all three crate reads (_archive_files, verify_archive, the Cargo.lock restore) now go through _read_regular_no_follow(path, "crate archive") (lstat + O_NOFOLLOW + fstat) and tarfile.open(fileobj=io.BytesIO(...)). No path-based tarfile.open remains, and nothing is extracted.
  • Messages: _read_regular_no_follow's new what label defaults to the old wording, so the approval messages are unchanged.
  • Test: test_archive_reached_through_a_symlink_is_refused was added, and it fails on the old code.
  • Nit, non-blocking: verify_archive reads the crate twice (inside _archive_files, then for the checksum and members), both no-follow.

This resolves SonarCloud S8707 (AaGjmjrxbL4Df1eBX_9Q) by hardening rather than suppression. Clearance carries to 4bcbca5 once CI is terminal and the SonarCloud quality gate is green. Merge at that SHA and post a readback.

S8707 traces the CLI path into _read_regular_no_follow's lstat and open.
Every caller confines it under an explicit root first (--archive-root or
the approvals repository via confined_root/confined_file), and these are
two different roots, so mark the two reads NOSONAR with the reason next
to them. Name the repeated "crate archive" label CRATE_ARCHIVE (S1192).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

General interim review (Security unavailable): I verified the delta 4bcbca5..8325ac7 myself. It's one commit, one file, with no behaviour change.

  • S1192: the CRATE_ARCHIVE constant is used at all 3 sites.
  • S8707: # NOSONAR on the os.lstat and os.open lines of _read_regular_no_follow, with a justification comment and docstring. I checked the justification against the code: _run confines every CLI path before it's opened. Archives go through confined_file(confined_root(--archive-root), --archive), manifest and changelog under --source-root, and approvals through read_approval_record(confined_root(--approval-repository)). load_approval is test-only. Inside the helper, lstat + O_NOFOLLOW + fstat refuse a final link and any non-regular file. The suppression is justified.

Clearance carries to 8325ac7 once CI is terminal and the SonarCloud gate is green. Merge at that SHA and post a readback.

@sonarqubecloud

sonarqubecloud Bot commented Nov 4, 2026

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel marked this pull request as ready for review November 4, 2026 06:42
@rmcdaniel
rmcdaniel merged commit 21c62ec into main Nov 4, 2026
19 checks passed
@rmcdaniel

Copy link
Copy Markdown
Member Author

Merged through GitHub at 21c62ecb6e515629b6eee69fa83c832dc2322a2f, using --match-head-commit 8325ac7869282fa40dd97f85abe806be5c5de3eb, after General's exact-head clearance, 17 passing checks, two intended skips and the passing SonarCloud gate.

Readback: the merge and cleared head both have tree 6622e2e433322443c37203e7bf6891a8bf1d749d. No release tag or publication has occurred. Next is refreshing #177 onto this main and freezing its release source for Security review before the installed-native run.

@rmcdaniel
rmcdaniel deleted the feat/prepublication-acceptance branch November 4, 2026 06:43
@rmcdaniel rmcdaniel mentioned this pull request Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant