diff --git a/README.md b/README.md index 65ea6de6..5a95805f 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ Every number below comes from a file on `main` (the one exception, a reviewer fi | Signed model manifest checked next to cp_verify, stored small example (PR #59) | measured, unreplicated | 137 of 137 registered tampers refused, 10 of 10 untampered verifies accepted (cp_verify YES), verify about 1.41 s; 14 of 37 fields are the trusted signer's word: 11 always (10 as a one-field change, plus commitment.kind, which a matching seed file passes even with --public-weights-seed) and 3 (upstream, committed files, seed digest) unless the caller passes the rehash flag; small-order ed25519 keys and R refused; symlinks in rehashed dirs, deep nesting and non-true none refused | stored synthetic example, fixture tokenizer/template/system prompt, no real model; client-side detached manifest, pure-Python ed25519 (RFC 8032 3 of 3, 64 of 64 vs cryptography); CPU only, re-measured after PREREG amendment 3 | [experiments/20261104-58-signed-manifest/RESULTS.md](experiments/20261104-58-signed-manifest/RESULTS.md) | | Still open: integer-vs-BF16/FP8 fidelity (#8), KDA model-scale sweep (PR #42), cost teardown and larger-memory GPU test (PR #44) | in progress | none yet | open PRs, issues and lanes, not merged | [docs/research-roadmap.md](docs/research-roadmap.md) | -_Last regenerated from `research/results.yaml` on 2026-11-05 (UTC), against main `b6186e6c`._ +_Last regenerated from `research/results.yaml` on 2026-11-05 (UTC), against main `05d0ad8e`._ What this table does not show: no full-size (Maverick) run, no real model weights on GPU, no serving, no TEE, no production use, and no comparison with other verifiable-inference systems. The GPU rows are upstream's small example on made-up weights. diff --git a/research/results.yaml b/research/results.yaml index 226aa8de..ec2805ae 100644 --- a/research/results.yaml +++ b/research/results.yaml @@ -384,12 +384,12 @@ results: conditions: "stored synthetic example, fixture tokenizer/template/system prompt, no real model; client-side detached manifest, pure-Python ed25519 (RFC 8032 3 of 3, 64 of 64 vs cryptography); CPU only, re-measured after PREREG amendment 3" evidence: [experiments/20261104-58-signed-manifest/RESULTS.md] metrics: - - {name: registered_tampers_refused, stat: count, value: 137, total: 137, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "enumerated tamper set, each deterministic; registered threshold 100%"} - - {name: untampered_accepted, stat: count, value: 10, total: 10, unit: count, n: 10, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "repeats of one deterministic check, not independent trials"} - - {name: trusted_resign_accepted_fields, stat: count, value: 10, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "one-field change re-signed by the trusted key, accepted; no threshold"} - - {name: trusted_resign_accepted_always_fields, stat: count, value: 11, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "accepted when re-signed by the trusted key with every flag the caller can pass on CPU: the 10 one-field fields plus commitment.kind (matching seed file, amendment 3); no threshold"} - - {name: trusted_resign_accepted_unless_flag_fields, stat: count, value: 3, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "schema-valid coordinated change re-signed by the trusted key, accepted without the matching rehash flag, refused with it (upstream.kind, committed_files, seed digest; commitment.kind moved to always in amendment 3); no threshold"} - - {name: verify_wall, stat: median, value: 1.407, unit: s, n: 10, spread: {sd: 0.027, min: 1.380, max: 1.471}, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "whole verify call incl. one cp_verify on the stored small example; shared VM; no threshold"} + - {name: registered_tampers_refused, stat: count, value: 137, total: 137, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "enumerated tamper set, each deterministic; registered threshold 100%"} + - {name: untampered_accepted, stat: count, value: 10, total: 10, unit: count, n: 10, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "repeats of one deterministic check, not independent trials"} + - {name: trusted_resign_accepted_fields, stat: count, value: 10, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "one-field change re-signed by the trusted key, accepted; no threshold"} + - {name: trusted_resign_accepted_always_fields, stat: count, value: 11, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "accepted when re-signed by the trusted key with every flag the caller can pass on CPU: the 10 one-field fields plus commitment.kind (matching seed file, amendment 3); no threshold"} + - {name: trusted_resign_accepted_unless_flag_fields, stat: count, value: 3, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "schema-valid coordinated change re-signed by the trusted key, accepted without the matching rehash flag, refused with it (upstream.kind, committed_files, seed digest; commitment.kind moved to always in amendment 3); no threshold"} + - {name: verify_wall, stat: median, value: 1.407, unit: s, n: 10, spread: {sd: 0.027, min: 1.380, max: 1.471}, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "whole verify call incl. one cp_verify on the stored small example; shared VM; no threshold"} - id: open-studies title: "Still open: integer-vs-BF16/FP8 fidelity (#8), KDA model-scale sweep (PR #42), cost teardown and larger-memory GPU test (PR #44)"