Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
- name: Install rivet
run: |
cargo install --git https://github.com/pulseengine/rivet \
--tag v0.13.0 \
--tag v0.41.1 \
rivet-cli --bin rivet --locked
rivet --version

Expand All @@ -51,6 +51,9 @@ jobs:
- name: bazel build (AADL → WIT → wit-bindgen → wasm component)
run: bazel build //...

- name: verify.py bucketing self-test (rivet and bazel stubbed)
run: python3 -m unittest discover -s tools -p 'test_verify.py' -v

- name: artifact-driven verification gate (real bazel test per artifact)
# verify.py shells out to `bazel test --test_arg=--exact …` per
# entry in each comp-req's verified-by list. Honest signal: red
Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ wit:

# ── Artifact-driven verification gate (always runs) ─────────────────
# Same shape as the LS-N gate in pulseengine/meld. Walks every
# comp-req with status:approved, finds tests by name convention,
# comp-req with status:valid, reads its verified-by tests through rivet,
# reports PASSED / FAILED / MISSING per artifact. Exits 1 if any
# FAILED or MISSING.
#
Expand All @@ -64,7 +64,7 @@ wit:
# scopes in or out.
VARIANT ?= dev
verify:
@$(PYTHON) tools/verify.py --artifacts artifacts --variant $(VARIANT)
@$(PYTHON) tools/verify.py --rivet $(RIVET) --schemas $(SCHEMAS) --variant $(VARIANT)

# ── sigil-signed release manifest (optional) ────────────────────────
attest:
Expand Down
22 changes: 12 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,29 +62,31 @@ the methodology is the lesson.

**The gate reports this as KNOWN-GAP.** Until 2026-11 the gate went
red on this finding, which hid every other regression behind the same
signal. The gap is now waived by `DR-KVS-KEY-VALIDATION-GAP`, an approved
signal. The gap is now waived by `DR-KVS-KEY-VALIDATION-GAP`, a valid
decision record tagged `known-gap`. The four surface tests still run on
every `make verify` and are listed by name. Current output (dev variant):

```
BUCKET ID EVIDENCE
─────────────────────────────────────────────────────────────────────
PASSED COMP-REQ-KVS-ATOMIC-STORE 5 test(s) all green
PASSED COMP-REQ-KVS-KEY-ENCODING 1 test(s) all green
PASSED COMP-REQ-KVS-VALUE-CHECKSUM 5 test(s) all green
PASSED COMP-REQ-KVS-ATOMIC-STORE 5 test(s) all green
KNOWN-GAP COMP-REQ-KVS-KEY-LENGTH 1 test(s) failed, waived by DR-KVS-KEY-VALIDATION-GAP:
- test_..._32_byte_cap_enforced
KNOWN-GAP COMP-REQ-KVS-KEY-NAMING 3 test(s) failed, waived by DR-KVS-KEY-VALIDATION-GAP:
- test_..._space_rejected
- test_..._dot_rejected
- test_..._slash_rejected
KNOWN-GAP COMP-REQ-KVS-KEY-LENGTH 1 test(s) failed, waived by DR-KVS-KEY-VALIDATION-GAP:
- test_..._32_byte_cap_enforced
─────────────────────────────────────────────────────────────────────
3 PASSED, 0 FAILED, 0 MISSING, 2 KNOWN-GAP, 0 STALE-GAP
```

The waiver only covers FAILED. A waived comp-req whose tests all pass
is reported as STALE-GAP and fails the gate until the decision record
is removed. MISSING evidence is never waivable. `make verify VARIANT=prod`
is removed. MISSING evidence, a malformed `verified-by:` entry and a
build or discovery error are never waivable: only a test that ran and
failed is. `tools/test_verify.py` checks these rules in CI. `make verify VARIANT=prod`
additionally lists COMP-REQ-KVS-INLINE-STORAGE as MISSING and exits 1.

INLINE-STORAGE is MISSING (not PASSED) on purpose: the spec asks for
Expand All @@ -104,7 +106,7 @@ they belong to. Pulseengine's `verified-by:` mechanism forces
test-to-requirement traceability at the level of named test functions,
and the gate then asserts each named test PASSED. Requirements with no
verifying test stay loudly red; a failing test is only accepted with a
named, approved decision record.
named, valid decision record.

## What this gate measures

Expand All @@ -116,7 +118,7 @@ Each comp-req artifact carries a `verified-by:` list of
2. **Runs** — invokes `bazel test --test_arg=--exact --test_arg=<name>`
per entry. Any non-zero exit → bucket `FAILED`. All green → bucket
`PASSED`.
3. **Waives** — a `FAILED` comp-req that an approved `known-gap`
3. **Waives** — a `FAILED` comp-req that a valid `known-gap`
decision record links with `satisfies` → bucket `KNOWN-GAP`
(exit 0). If such a comp-req passes → bucket `STALE-GAP` (exit 1).

Expand All @@ -126,8 +128,8 @@ falsifications, the response options are explicit:
- Add runtime validation to the impl → tests go green.
- Edit the upstream RST to drop those requirements → artifact updates,
tests can be deleted.
- Downgrade the comp-req's `status` from `approved` to `draft` (the
gate only enforces approved artifacts).
- Downgrade the comp-req's `status` from `valid` to `draft` (the
gate only enforces valid artifacts).

All three are legitimate engineering responses to evidence — the gate
makes the choice explicit, where a coverage-wedge visualisation makes
Expand All @@ -142,7 +144,7 @@ it implicit.
| `tests/surface/surface_tests.rs` | One `#[test]` per comp-req, asserting requirement-as-test against the vendored upstream code (verbatim spec text, no embellishment) | None — eclipse tests are organized by module, not requirement |
| `arch/kvs.aadl` (spar AADL) | Typed feature group + subprogram signatures + ARP4761 safety properties | None |
| `arch/kvs.wit` (binary contract) | WIT interface that wit-bindgen turns into a Rust trait. `src/lib.rs` implements that trait by **delegating to the vendored eclipse-score `rust_kvs`**; the .wasm component therefore links the *real* upstream code, not a toy stub. If the upstream API drifts or the WIT signature drifts, the Rust compile fails — binary-contract enforcement runs over real code. | None — interface stops at the rendered diagram |
| `tools/verify.py` (artifact-driven gate) | Walks every approved comp-req, reads `verified-by:`, runs `bazel test` per entry, exits red on any gap | None — eclipse renders a coverage pie chart |
| `tools/verify.py` (artifact-driven gate) | Walks every valid comp-req (read through `rivet list --format json --full`), reads `verified-by:`, runs `bazel test` per entry, exits red on any gap | None — eclipse renders a coverage pie chart |
| `attestation/release-manifest.yaml` (sigil) | Signed in-toto-style attestation tying artifact hashes + WIT hash + evidence hashes | Green CI badge |

## What's *real* infrastructure vs *example skeleton*
Expand Down
28 changes: 14 additions & 14 deletions artifacts/architecture.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ artifacts:
- id: FEAT-PERSISTENCY
type: feat
title: Persistency
status: approved
status: valid
description: >
User-visible persistency feature exposed through a managed API.
Eclipse: feat__persistency.
Expand All @@ -33,7 +33,7 @@ artifacts:
- id: COMP-KVS
type: comp
title: persistency::kvs
status: approved
status: valid
description: >
Key-value store implementation of the persistency feature.
Eclipse: comp__persistency_kvs.
Expand All @@ -56,7 +56,7 @@ artifacts:
- id: LOGIC-ARC-INT-KVS
type: logic-arc-int
title: KVS Interface
status: approved
status: valid
description: >
Logical interface of the key-value store: store / load / delete /
snapshot operations over (key, value) pairs.
Expand All @@ -69,7 +69,7 @@ artifacts:
- id: LOGIC-ARC-INT-OP-KVS-STORE
type: logic-arc-int
title: Store
status: approved
status: valid
description: Persist a (key, value) pair durably.
fields:
safety-level: ASIL_B
Expand All @@ -82,7 +82,7 @@ artifacts:
- id: LOGIC-ARC-INT-OP-KVS-LOAD
type: logic-arc-int
title: Load
status: approved
status: valid
description: Retrieve a previously-stored value by key, with integrity check.
fields:
safety-level: ASIL_B
Expand All @@ -95,7 +95,7 @@ artifacts:
- id: LOGIC-ARC-INT-OP-KVS-DELETE
type: logic-arc-int
title: Delete
status: approved
status: valid
description: Remove a key-value pair durably.
fields:
safety-level: ASIL_B
Expand All @@ -108,7 +108,7 @@ artifacts:
- id: LOGIC-ARC-INT-OP-KVS-SNAPSHOT-CREATE
type: logic-arc-int
title: Create Snapshot
status: approved
status: valid
description: Atomically capture the current state of the KVS.
fields:
safety-level: ASIL_B
Expand All @@ -121,7 +121,7 @@ artifacts:
- id: LOGIC-ARC-INT-OP-KVS-SNAPSHOT-RESTORE
type: logic-arc-int
title: Restore Snapshot
status: approved
status: valid
description: Atomically restore the KVS state from a prior snapshot.
fields:
safety-level: ASIL_B
Expand All @@ -136,7 +136,7 @@ artifacts:
- id: DD-STA-KVS-KEY-VALIDATOR
type: dd-sta
title: Key validator
status: approved
status: valid
description: >
Static design of the syntactic key validator. Validates the
alphabet, length boundaries, and leading-dot rejection.
Expand All @@ -151,7 +151,7 @@ artifacts:
- id: DD-STA-KVS-BACKEND-STRATEGY
type: dd-sta
title: Storage backend strategy
status: approved
status: valid
description: >
Strategy pattern: `KvsBackend` trait with two implementations
(RocksDb for production, InMemory for tests).
Expand All @@ -166,7 +166,7 @@ artifacts:
- id: DD-DYN-KVS-ATOMIC-STORE
type: dd-dyn
title: Atomic store sequence
status: approved
status: valid
description: >
Sequence design for the atomic store path: WAL append → fsync
→ live-table update → log rotation. Power-loss between any
Expand All @@ -184,7 +184,7 @@ artifacts:
- id: SW-UNIT-KVS-VALIDATOR
type: sw-unit
title: KVS key validator implementation
status: approved
status: valid
description: Rust source implementing the syntactic key validator.
fields:
safety-level: ASIL_B
Expand All @@ -200,7 +200,7 @@ artifacts:
- id: SW-UNIT-KVS-BACKEND
type: sw-unit
title: KVS RocksDB backend implementation
status: approved
status: valid
description: Rust source implementing the RocksDB-backed `KvsBackend`.
fields:
safety-level: ASIL_B
Expand All @@ -216,7 +216,7 @@ artifacts:
- id: SW-UNIT-KVS-ATOMIC-STORE
type: sw-unit
title: KVS atomic-store implementation
status: approved
status: valid
description: Rust source implementing the WAL-based atomic store.
fields:
safety-level: ASIL_B
Expand Down
24 changes: 12 additions & 12 deletions artifacts/requirements.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ artifacts:
- id: STKH-REQ-PERSIST-STORAGE
type: stkh-req
title: Persistent storage capability
status: approved
status: valid
description: >
The SW-platform shall provide persistent storage for application
data that survives power-cycles and software updates.
Expand All @@ -31,7 +31,7 @@ artifacts:
- id: STKH-REQ-PERSIST-INTEGRITY
type: stkh-req
title: Stored-data integrity
status: approved
status: valid
description: >
The SW-platform shall detect corruption of persisted data
caused by power-loss-during-write or media degradation.
Expand All @@ -47,7 +47,7 @@ artifacts:
- id: FEAT-REQ-PERSIST-STORE
type: feat-req
title: Store data
status: approved
status: valid
description: >
The persistency feature shall accept (key, value) pairs and
persist them durably across power cycles.
Expand All @@ -67,7 +67,7 @@ artifacts:
- id: FEAT-REQ-PERSIST-LOAD
type: feat-req
title: Load data
status: approved
status: valid
description: >
The persistency feature shall retrieve a previously-stored
value by key, or report not-found.
Expand All @@ -84,7 +84,7 @@ artifacts:
- id: FEAT-REQ-PERSIST-INTEGRITY-CHECK
type: feat-req
title: Integrity check on load
status: approved
status: valid
description: >
The persistency feature shall compute and verify a checksum
on every load, returning an error rather than corrupted data
Expand All @@ -102,7 +102,7 @@ artifacts:
- id: FEAT-REQ-PERSIST-NO-RUNTIME-ALLOC
type: feat-req
title: No dynamic allocation post-init
status: approved
status: valid
description: >
The persistency feature shall not perform heap allocation
after initialization (no malloc/free on the hot path).
Expand All @@ -125,7 +125,7 @@ artifacts:
- id: COMP-REQ-KVS-KEY-NAMING
type: comp-req
title: Key Naming
status: approved
status: valid
description: >
The component shall accept keys that consist solely of alphanumeric
characters, underscores, or dashes.
Expand Down Expand Up @@ -156,7 +156,7 @@ artifacts:
- id: COMP-REQ-KVS-KEY-LENGTH
type: comp-req
title: Key Length
status: approved
status: valid
description: >
The component shall limit the maximum length of a key to 32 bytes.
Eclipse: comp_req__kvs__key_length (verbatim).
Expand All @@ -179,7 +179,7 @@ artifacts:
- id: COMP-REQ-KVS-KEY-ENCODING
type: comp-req
title: Key Encoding
status: approved
status: valid
description: >
The component shall encode each key as valid UTF-8.
Eclipse: comp_req__kvs__key_encoding (verbatim).
Expand All @@ -204,7 +204,7 @@ artifacts:
- id: COMP-REQ-KVS-VALUE-CHECKSUM
type: comp-req
title: Per-value checksum
status: approved
status: valid
description: >
Each stored value shall carry a CRC32C checksum computed at
store time and verified at load time.
Expand Down Expand Up @@ -232,7 +232,7 @@ artifacts:
- id: COMP-REQ-KVS-ATOMIC-STORE
type: comp-req
title: Atomic store under power loss
status: approved
status: valid
description: >
A store operation shall either complete fully and durably
or leave the previous value intact. Partial writes must
Expand All @@ -258,7 +258,7 @@ artifacts:
- id: COMP-REQ-KVS-INLINE-STORAGE
type: comp-req
title: Inline storage, no runtime allocation
status: approved
status: valid
description: >
KVS shall use pre-allocated inline buffers sized at component
initialization; no heap allocation on the hot path.
Expand Down
Loading
Loading