schema-2 sharded receipts: per-shard fingerprints + policy-gated carry-forward

CHANGELOG.md

@@ -4,6 +4,21 @@ All notable changes to pytest-gpu-proof are documented here. The format is
 based on [Keep a Changelog](https://keepachangelog.com/); versions follow
 [SemVer](https://semver.org/) (pre-1.0: minor bumps may break).
 
+## [0.3.0] — 2026-08-08
+
+### Added
+- Schema `"2"` sharded receipts: `--gpu-proof-shard NAME` +
+  `--gpu-proof-shard-fingerprint-paths` emit a per-shard narrow fingerprint;
+  `gpu-proof merge` unions schema-2 shards (unique names enforced).
+- Verifiable carry-forward: `gpu-proof merge --carry-from OLD` grafts shards
+  absent from the fresh inputs iff the old commit is an ancestor AND the
+  shard's narrow fingerprint recomputes clean; grafted shards are marked
+  `carried` and the verifier rejects them unless the policy sets
+  `allow_carried: true` (bounded by `carried_max_age_days`, default 30).
+- Verifier: schema-2 checks (per-shard fingerprint recompute, membership
+  partition of `tests[]`, carried-shard policy gate). Schema-1 receipts are
+  unchanged and a schema-1 receipt carrying a `shards` block is rejected.
+
 ## [0.2.0] — 2026-08-07
 
 ### Added

docs/security_model.md

@@ -73,3 +73,15 @@ that a specific GPU executed a specific workload in a verified environment.
 This is out of scope for v1 but the receipt format is designed to be extendable —
 a `hardware_attestation` block could be added to the `environment` section in a
 future version without breaking existing receipts.
+
+## Carried shards (schema 2)
+
+A carried shard is an attestation about a **prior** run: the tests passed at an
+ancestor commit, and the shard's declared input paths are byte-identical at the
+verified commit (the verifier recomputes this; it is not taken on faith). What
+is NOT re-established: that the prior run's environment still exists, or that
+paths *outside* the shard's declared fingerprint didn't change its behavior —
+declaring too-narrow shard paths weakens the claim, exactly like declaring
+too-narrow global fingerprint paths. That is why `allow_carried` defaults to
+**false**: accepting carried shards is an explicit policy decision, bounded by
+`carried_max_age_days`.

docs/sharding.md

@@ -60,3 +60,50 @@ an **attestation by the merger**. Shard signatures are recorded as provenance
 but not re-verified at merge time (merging is offline); the merged signature
 is what CI verifies. If shards were signed by someone else, verification of
 the merged receipt attests that *you* vouch for the union.
+
+## Per-shard fingerprints & carry-forward (schema 2)
+
+Declare each invocation as a **shard** and the receipt becomes schema `"2"`,
+carrying that shard's own *narrow* fingerprint over the paths you declare:
+
+```bash
+pytest tests/gpu/test_a.py --gpu-proof-enable \
+    --gpu-proof-shard=test_a \
+    --gpu-proof-shard-fingerprint-paths=tests/gpu/test_a.py,src/kernels_a \
+    --gpu-proof-out=receipts/a.json
+```
+
+`gpu-proof merge` unions schema-2 shards exactly like schema-1 receipts (shard
+names must be unique). The new capability is **carry-forward**:
+
+```bash
+gpu-proof merge --out gpu-proof.json --carry-from last-green/gpu-proof.json \
+    --repo . receipts/*.json
+```
+
+Shards present in the old receipt but absent from the fresh inputs are grafted
+in, **marked `carried`**, iff:
+
+1. the old receipt's commit is an **ancestor** of the fresh one (same history), and
+2. the shard's narrow fingerprint **recomputes identical** against the current
+   tree — the inputs that shard proved are unchanged.
+
+A shard whose inputs changed refuses to carry (re-run it). Freshly re-run
+shards always win over old ones.
+
+### Verification of schema-2 receipts
+
+`gpu-proof verify` additionally checks, for every shard: the narrow
+fingerprint recomputes clean at the verifying tree, and shard membership
+exactly partitions `tests[]`. **Carried shards are rejected by default** — the
+policy must opt in:
+
+```yaml
+allow_carried: true          # default false — the trust boundary
+carried_max_age_days: 30     # carried shard's ORIGINAL run must be fresher
+```
+
+A receipt with carried shards verified under `allow_carried: true` means:
+*every test either ran at this commit, or ran at an ancestor commit on inputs
+that are provably byte-identical today, within the age window* — and the
+merger signed for that claim.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pytest-gpu-proof"
-version = "0.2.0"
+version = "0.3.0"
 description = "pytest plugin for GPU equivalence testing with signed receipts verified via GitHub SSH keys"
 readme = "README.md"
 requires-python = ">=3.11"

src/pytest_gpu_proof/__init__.py

@@ -1 +1 @@
-__version__ = "0.2.0"
+__version__ = "0.3.0"

src/pytest_gpu_proof/cli.py

@@ -97,6 +97,15 @@ def main():
     mp.add_argument("--unsigned", action="store_true", default=False,
                     help="Write signature: null — the merged receipt then "
                     "verifies only with --allow-unsigned, loudly")
+    mp.add_argument("--carry-from", default=None, metavar="RECEIPT",
+                    help="Graft still-valid shards from an older schema-2 "
+                    "receipt: each absent-from-fresh shard is carried iff the "
+                    "old commit is an ancestor of the new one AND its narrow "
+                    "fingerprint recomputes clean at the current tree. Carried "
+                    "shards are marked and gated at verify time by policy "
+                    "allow_carried (default: rejected).")
+    mp.add_argument("--repo", default=".", metavar="PATH",
+                    help="Repo root for carry-from fingerprint/ancestry checks")
 
     args = parser.parse_args()
 
@@ -109,6 +118,8 @@ def main():
                 github_user=args.github_user,
                 key_path=args.key,
                 unsigned=args.unsigned,
+                carry_from=args.carry_from,
+                repo_root=args.repo,
             )
         except MergeError as e:
             print(f"gpu-proof merge: {e}", file=sys.stderr)

src/pytest_gpu_proof/config.py

@@ -18,6 +18,10 @@ class GpuProofConfig:
     github_username: Optional[str] = None
     max_age_days: int = 30
     require_gpu: bool = False
+    # Sharded emission (schema "2"): a declared shard name + its narrow
+    # fingerprint paths (None -> the global fingerprint_paths).
+    shard_name: Optional[str] = None
+    shard_fingerprint_paths: Optional[List[str]] = None
 
 
 def load_toml_defaults(root: Union[str, Path]) -> Dict[str, Any]:
@@ -66,6 +70,15 @@ def resolve(opt_name, toml_key, default):
     else:
         paths = [str(p) for p in raw_paths]
 
+    raw_shard_paths = resolve("--gpu-proof-shard-fingerprint-paths",
+                              "shard_fingerprint_paths", None)
+    if isinstance(raw_shard_paths, str):
+        shard_paths = [p.strip() for p in raw_shard_paths.split(",") if p.strip()]
+    elif raw_shard_paths is not None:
+        shard_paths = [str(p) for p in raw_shard_paths]
+    else:
+        shard_paths = None
+
     max_age = toml_cfg.get("max_age_days")
     max_age_days = int(max_age) if max_age is not None else 30
 
@@ -85,4 +98,6 @@ def resolve(opt_name, toml_key, default):
         github_username=resolve("--gpu-proof-github-user", "github_username", None),
         max_age_days=max_age_days,
         require_gpu=bool(toml_cfg.get("require_gpu", False)),
+        shard_name=resolve("--gpu-proof-shard", "shard_name", None),
+        shard_fingerprint_paths=shard_paths,
     )

src/pytest_gpu_proof/merge.py

@@ -60,10 +60,11 @@ def merge_payloads(receipts: List[dict], sources: List[str]) -> dict:
         raise MergeError("nothing to merge")
 
     _require_identical(receipts, sources, lambda r: r.get("schema_version"), "schema_version")
-    if receipts[0].get("schema_version") != "1":
+    schema = receipts[0].get("schema_version")
+    if schema not in ("1", "2"):
         raise MergeError(
-            f"unsupported schema_version {receipts[0].get('schema_version')!r} "
-            f"(this version merges schema '1' receipts)"
+            f"unsupported schema_version {schema!r} (this version merges "
+            f"schema '1' and schema '2' receipts, not mixed)"
         )
     _require_identical(receipts, sources, lambda r: r.get("repo", {}).get("commit_sha"),
                        "repo.commit_sha")
@@ -90,6 +91,24 @@ def merge_payloads(receipts: List[dict], sources: List[str]) -> dict:
     if not tests:
         raise MergeError("merged receipt would contain zero tests")
 
+    # schema 2: union the shards lists (shard names must be unique across
+    # inputs; a shard's node_ids stay attached to it, so the merged receipt
+    # still partitions tests[] by shard for the verifier's membership check).
+    merged_shards = None
+    if schema == "2":
+        merged_shards = []
+        shard_names: dict = {}
+        for src, r in zip(sources, receipts):
+            for shard in r.get("shards") or []:
+                nm = shard.get("name")
+                if nm in shard_names:
+                    raise MergeError(
+                        f"duplicate shard name {nm!r} in {src} (already from "
+                        f"{shard_names[nm]}) — shard names must be unique."
+                    )
+                shard_names[nm] = src
+                merged_shards.append(shard)
+
     sessions = [r.get("session", {}) for r in receipts]
     started = min(s.get("started_at") for s in sessions)
     ended = max(s.get("ended_at") for s in sessions)
@@ -126,26 +145,117 @@ def merge_payloads(receipts: List[dict], sources: List[str]) -> dict:
             for src, r, s in zip(sources, receipts, sessions)
         ],
     }
+    if merged_shards is not None:
+        merged["shards"] = merged_shards
     return merged
 
 
+def _git_is_ancestor(repo_root: str, ancestor: str, descendant: str) -> bool:
+    import subprocess
+    try:
+        subprocess.run(
+            ["git", "-C", repo_root, "merge-base", "--is-ancestor", ancestor, descendant],
+            capture_output=True, check=True)
+        return True
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return False
+
+
+def carry_forward(payload: dict, old_receipt: dict, old_source: str,
+                  repo_root: str = ".") -> dict:
+    """Graft shards from ``old_receipt`` that are ABSENT from the fresh merged
+    ``payload``, marking each ``carried``. Soundness conditions per shard:
+
+      1. the old receipt is schema '2' and its commit is an ANCESTOR of the
+         fresh payload's commit (same history, older point);
+      2. the shard's narrow fingerprint recomputes IDENTICAL against the
+         current tree — the inputs that shard proved are unchanged at HEAD.
+
+    Shard signatures are provenance (merge is offline); the re-signed merged
+    receipt is the attestation, and the VERIFIER re-checks every carried
+    shard's fingerprint and gates them on policy ``allow_carried``."""
+    from .fingerprint import compute_fingerprint
+
+    if old_receipt.get("schema_version") != "2":
+        raise MergeError(
+            f"--carry-from {old_source}: not a schema '2' (sharded) receipt")
+    old_sha = old_receipt.get("repo", {}).get("commit_sha")
+    new_sha = payload.get("repo", {}).get("commit_sha")
+    if old_sha and new_sha and old_sha != new_sha and not _git_is_ancestor(
+            repo_root, old_sha, new_sha):
+        raise MergeError(
+            f"--carry-from {old_source}: its commit {old_sha[:12]} is not an "
+            f"ancestor of {new_sha[:12]} — different history, cannot carry.")
+
+    fresh_names = {s.get("name") for s in payload.get("shards") or []}
+    fresh_ids = {t.get("node_id") for t in payload.get("tests", [])}
+    old_tests = {t.get("node_id"): t for t in old_receipt.get("tests", [])}
+    carried_count = 0
+    for shard in old_receipt.get("shards") or []:
+        nm = shard.get("name")
+        if nm in fresh_names:
+            continue  # freshly re-run — the new result wins
+        sfp = shard.get("fingerprint", {})
+        snow = compute_fingerprint(sfp.get("included_paths", []), root=repo_root)
+        if snow["digest"] != sfp.get("digest"):
+            raise MergeError(
+                f"--carry-from {old_source}: shard {nm!r} fingerprint no longer "
+                f"matches the current tree — its inputs changed; re-run it "
+                f"instead of carrying.")
+        ids = shard.get("node_ids", [])
+        dup = fresh_ids.intersection(ids)
+        if dup:
+            raise MergeError(
+                f"--carry-from {old_source}: shard {nm!r} would re-introduce "
+                f"node id(s) already present (e.g. {sorted(dup)[0]!r}).")
+        grafted = dict(shard)
+        grafted["carried"] = {
+            "from": old_source.rsplit("/", 1)[-1],
+            "original_commit_sha": old_sha,
+            "original_ended_at": old_receipt.get("session", {}).get("ended_at"),
+            "original_signer": (old_receipt.get("signature") or {}).get("signer"),
+        }
+        payload.setdefault("shards", []).append(grafted)
+        for nid in ids:
+            if nid not in old_tests:
+                raise MergeError(
+                    f"--carry-from {old_source}: shard {nm!r} claims {nid!r} "
+                    f"which is not in its receipt's tests[].")
+            payload["tests"].append(old_tests[nid])
+            fresh_ids.add(nid)
+        carried_count += 1
+    payload["session"]["node_ids"] = [t["node_id"] for t in payload["tests"]]
+    payload["schema_version"] = "2"
+    if not carried_count:
+        print(f"[gpu-proof] merge: nothing to carry from {old_source} "
+              f"(all its shards were freshly re-run)")
+    return payload
+
+
 def merge_receipts(
     paths: List[str],
     out: str,
     *,
     github_user: Optional[str] = None,
     key_path: Optional[str] = None,
     unsigned: bool = False,
+    carry_from: Optional[str] = None,
+    repo_root: str = ".",
 ) -> dict:
     """Merge receipts at ``paths`` and write the re-signed result to ``out``.
 
     ``github_user`` overrides the recorded signer identity (default: the first
     shard's ``repo.github_username`` — correct when the merger is also the shard
     runner). ``unsigned=True`` writes ``signature: null`` (verifies only with
     ``--allow-unsigned``, loudly, same as the plugin's 'none' backend).
+    ``carry_from`` grafts still-valid shards from an older schema-2 receipt —
+    see :py:func:`carry_forward` for the soundness conditions.
     """
     receipts = [load_receipt(p) for p in paths]
     payload = merge_payloads(receipts, list(paths))
+    if carry_from:
+        payload = carry_forward(payload, load_receipt(carry_from), carry_from,
+                                repo_root=repo_root)
     if github_user:
         payload["repo"] = dict(payload.get("repo", {}))
         payload["repo"]["github_username"] = github_user

src/pytest_gpu_proof/plugin.py

@@ -246,6 +246,21 @@ def pytest_addoption(parser):
         metavar="PATHS",
         help="Comma-separated paths to fingerprint (default: src,tests)",
     )
+    group.addoption(
+        "--gpu-proof-shard",
+        default=None,
+        metavar="NAME",
+        help="Declare this run as one SHARD of a larger suite: the receipt is "
+        "emitted as schema '2' with a per-shard fingerprint, enabling "
+        "verifiable carry-forward via `gpu-proof merge --carry-from`",
+    )
+    group.addoption(
+        "--gpu-proof-shard-fingerprint-paths",
+        default=None,
+        metavar="PATHS",
+        help="Comma-separated paths for THIS shard's narrow fingerprint "
+        "(default: the global fingerprint paths)",
+    )
     group.addoption(
         "--gpu-proof-github-user",
         default=None,

src/pytest_gpu_proof/receipt.py

@@ -101,8 +101,28 @@ def build_receipt_payload(
             )
     fingerprint = compute_fingerprint(config.fingerprint_paths)
 
-    return {
-        "schema_version": "1",
+    # Sharded emission (schema "2", additive): when the run declares a shard
+    # name, the receipt carries a `shards` list whose single entry pins THIS
+    # shard's own narrow fingerprint (its declared paths, defaulting to the
+    # global fingerprint paths) and its test membership by node id. The flat
+    # `tests` list remains authoritative for outcomes; the global fingerprint
+    # keeps its schema-1 meaning. This is what makes per-shard carry-forward
+    # verifiable later: a shard whose narrow fingerprint still recomputes clean
+    # provably ran on identical inputs.
+    shard_name = getattr(config, "shard_name", None)
+    schema_version = "2" if shard_name else "1"
+    shards = None
+    if shard_name:
+        shard_paths = getattr(config, "shard_fingerprint_paths", None) or config.fingerprint_paths
+        shards = [{
+            "name": shard_name,
+            "fingerprint": compute_fingerprint(shard_paths),
+            "node_ids": [t["node_id"] for t in test_results],
+            "carried": None,
+        }]
+
+    payload = {
+        "schema_version": schema_version,
         "mode": config.mode,
         "repo": {
             "remote_url": remote_url,
@@ -120,6 +140,9 @@ def build_receipt_payload(
         "tests": test_results,
         "environment": _env_info(),
     }
+    if shards is not None:
+        payload["shards"] = shards
+    return payload
 
 
 def finalize_receipt(payload: dict, signer) -> dict: