Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Every number below comes from a file on `main` (the one exception, a reviewer fi
| Signed model manifest checked next to cp_verify, stored small example (PR #59) | measured, unreplicated | 137 of 137 registered tampers refused, 10 of 10 untampered verifies accepted (cp_verify YES), verify about 1.41 s; 14 of 37 fields are the trusted signer's word: 11 always (10 as a one-field change, plus commitment.kind, which a matching seed file passes even with --public-weights-seed) and 3 (upstream, committed files, seed digest) unless the caller passes the rehash flag; small-order ed25519 keys and R refused; symlinks in rehashed dirs, deep nesting and non-true none refused | stored synthetic example, fixture tokenizer/template/system prompt, no real model; client-side detached manifest, pure-Python ed25519 (RFC 8032 3 of 3, 64 of 64 vs cryptography); CPU only, re-measured after PREREG amendment 3 | [experiments/20261104-58-signed-manifest/RESULTS.md](experiments/20261104-58-signed-manifest/RESULTS.md) |
| Still open: integer-vs-BF16/FP8 fidelity (#8), KDA model-scale sweep (PR #42), cost teardown and larger-memory GPU test (PR #44) | in progress | none yet | open PRs, issues and lanes, not merged | [docs/research-roadmap.md](docs/research-roadmap.md) |

_Last regenerated from `research/results.yaml` on 2026-11-05 (UTC), against main `b6186e6c`._
_Last regenerated from `research/results.yaml` on 2026-11-05 (UTC), against main `05d0ad8e`._
<!-- results:end -->

What this table does not show: no full-size (Maverick) run, no real model weights on GPU, no serving, no TEE, no production use, and no comparison with other verifiable-inference systems. The GPU rows are upstream's small example on made-up weights.
Expand Down
12 changes: 6 additions & 6 deletions research/results.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -384,12 +384,12 @@ results:
conditions: "stored synthetic example, fixture tokenizer/template/system prompt, no real model; client-side detached manifest, pure-Python ed25519 (RFC 8032 3 of 3, 64 of 64 vs cryptography); CPU only, re-measured after PREREG amendment 3"
evidence: [experiments/20261104-58-signed-manifest/RESULTS.md]
metrics:
- {name: registered_tampers_refused, stat: count, value: 137, total: 137, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "enumerated tamper set, each deterministic; registered threshold 100%"}
- {name: untampered_accepted, stat: count, value: 10, total: 10, unit: count, n: 10, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "repeats of one deterministic check, not independent trials"}
- {name: trusted_resign_accepted_fields, stat: count, value: 10, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "one-field change re-signed by the trusted key, accepted; no threshold"}
- {name: trusted_resign_accepted_always_fields, stat: count, value: 11, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "accepted when re-signed by the trusted key with every flag the caller can pass on CPU: the 10 one-field fields plus commitment.kind (matching seed file, amendment 3); no threshold"}
- {name: trusted_resign_accepted_unless_flag_fields, stat: count, value: 3, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "schema-valid coordinated change re-signed by the trusted key, accepted without the matching rehash flag, refused with it (upstream.kind, committed_files, seed digest; commitment.kind moved to always in amendment 3); no threshold"}
- {name: verify_wall, stat: median, value: 1.407, unit: s, n: 10, spread: {sd: 0.027, min: 1.380, max: 1.471}, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 0ac5fca15f69c3620dbcd3d3116c5d81d12b2c92, source: experiments/20261104-58-signed-manifest/numbers.json, note: "whole verify call incl. one cp_verify on the stored small example; shared VM; no threshold"}
- {name: registered_tampers_refused, stat: count, value: 137, total: 137, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "enumerated tamper set, each deterministic; registered threshold 100%"}
- {name: untampered_accepted, stat: count, value: 10, total: 10, unit: count, n: 10, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "repeats of one deterministic check, not independent trials"}
- {name: trusted_resign_accepted_fields, stat: count, value: 10, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "one-field change re-signed by the trusted key, accepted; no threshold"}
- {name: trusted_resign_accepted_always_fields, stat: count, value: 11, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "accepted when re-signed by the trusted key with every flag the caller can pass on CPU: the 10 one-field fields plus commitment.kind (matching seed file, amendment 3); no threshold"}
- {name: trusted_resign_accepted_unless_flag_fields, stat: count, value: 3, total: 37, unit: count, n: 1, spread: null, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "schema-valid coordinated change re-signed by the trusted key, accepted without the matching rehash flag, refused with it (upstream.kind, committed_files, seed digest; commitment.kind moved to always in amendment 3); no threshold"}
- {name: verify_wall, stat: median, value: 1.407, unit: s, n: 10, spread: {sd: 0.027, min: 1.380, max: 1.471}, hardware: cpu-epyc-rome-kvm-16, date: 2026-11-05, commit: 05d0ad8e75ae814a514d0106f2bf3c2cb08ec2c0, source: experiments/20261104-58-signed-manifest/numbers.json, note: "whole verify call incl. one cp_verify on the stored small example; shared VM; no threshold"}

- id: open-studies
title: "Still open: integer-vs-BF16/FP8 fidelity (#8), KDA model-scale sweep (PR #42), cost teardown and larger-memory GPU test (PR #44)"
Expand Down