Publish the realm's signed line index, so the omission check can fire - #25
Merged
Merged
Conversation
varve has read `line-index-<line>` since 2026-09-06 and no realm has ever written one. Re-measured 2026-10-01: ghcr.io/pulseengine/layers serves 23 layer tags and zero `line-index-*`. REQ-INDEXAUTH-001's own notes name the gap — "a fetch path with no publisher is a check that cannot fire" — so every consumer-side guarantee resting on the index has been reasoning about a document nobody had written, and `varve outdated` could only ever answer "cannot answer". THE LIST IS COMMITTED HERE AND NEVER READ FROM THE REGISTRY (varve DD-027). The index exists to catch a registry HIDING a layer, and every artifact a hiding registry does serve still verifies. Derive the list from `oras repo tags` and the check is vacuous: the hiding registry omits the layer from the listing and therefore from the index, the two agree, and varve reports nothing wrong. It would pass its own tests, publish real signatures, and detect nothing. So `line-index/<line>.json` is a reviewed file, and each deposit appends the layer it just pushed. AFTER the push, never before. An index naming a layer the registry does not serve is precisely what omission detection screams about, so publishing one before the push succeeded would manufacture the alarm this mechanism exists to raise honestly. `tools/append-index.py` is idempotent, because a deposit is re-runnable: the same layer twice is a no-op that does NOT bump the document counter, since a counter that climbed on a re-dispatch would make the next legitimate publish look stale to a consumer that refuses an older index. One layer id arriving with DIFFERENT bytes is a refusal naming both digests, not an update — quietly rewriting the entry would hide exactly what varve's immutability rule exists to catch. Layer ids are ordered NUMERICALLY. A string sort puts 2026.09.12 before 2026.09.2, and that mistake was made against this very realm on 2026-10-01 and reported a staleness figure wrong by eight layers. It is the negative control for the ordering test: swapping the key for the raw string turns two tests red. THE STEP GATES ITSELF by reading the index back the way a CONSUMER does — fetching the manifest, fetching the blob, decoding the DSSE payload — and requiring this layer to be named in it. A signature over a document nobody can fetch is the failure this step exists to end, and "oras said ok" is not "a consumer can read it". THE 23 EXISTING LAYERS ARE DELIBERATELY NOT BACKFILLED, and line-index/README.md says why. The only source for their payload digests is the registry, and a registry-derived entry protects nothing by the same argument as above: a registry that had hidden one of them would have hidden it from the backfill. A document that looks complete and protects nothing is worse than a short one that is honest about its reach. varve treats the index as a FLOOR — extra layers a source serves are never an error — so detection covers every layer from here on. `signed-index = true` is NOT set. A realm declaring it with no index published fails closed on every install; the declaration is the last step, and it lives in varve's varve-realms.toml rather than here. Closes #6. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #6.
The check that could not fire
varve has read
line-index-<line>since 2026-09-06 and no realm has everwritten one. Re-measured today:
ghcr.io/pulseengine/layersserves 23 layertags and zero
line-index-*. REQ-INDEXAUTH-001's own notes name the gap —"a fetch path with no publisher is a check that cannot fire" — so every
consumer-side guarantee resting on the index has been reasoning about a
document nobody had written, and
varve outdatedcould only ever answer"cannot answer".
The list is committed here, never read from the registry
Per varve DD-027, and it is the whole point. The index exists to catch a
registry hiding a layer, and every artifact a hiding registry does serve
still verifies. Derive the list from
oras repo tagsand the check is vacuous:the hiding registry omits the layer from the listing and therefore from the
index, the two agree perfectly, and varve reports nothing wrong. It would pass
its own tests, publish real signatures, and detect nothing.
So
line-index/<line>.jsonis a reviewed file, and each deposit appends thelayer it just pushed.
What the deposit now does, after the push
After, never before — an index naming a layer the registry does not serve
is exactly what omission detection screams about, so publishing one before the
push succeeded would manufacture the alarm this mechanism exists to raise
honestly.
cannot be handed a digest belonging to another layer);
tools/append-index.pyappends{layer, digest, channel, counter};git pull --rebasees, because the scanner commitslayer.tomlon its own schedule;varve sign-indexwith the realm root, key via file descriptor not afile (REQ-NOKEYDISK-001);
oras blob push+oras manifest pushunderline-index-<line>, with themanifest shape copied from
varve docs attach-indexrather thanre-derived — a role annotation spelled differently reads as no index at all.
The step gates itself
It reads the index back the way a consumer does — fetch the manifest, fetch
the blob, decode the DSSE payload — and requires this layer to be named in it.
A signature over a document nobody can fetch is the failure this step exists to
end, and "oras said ok" is not "a consumer can read it".
The appender
Idempotent, because a deposit is re-runnable: the same layer twice is a
no-op that does not bump the document counter. A counter that climbed on a
re-dispatch would make the next legitimate publish look stale to a consumer
that refuses an older index.
One layer id with different bytes is a refusal, naming both digests — not
an update. Quietly rewriting the entry would hide exactly what varve's
immutability rule exists to catch.
Layer ids order numerically. A string sort puts
2026.09.12before2026.09.2— a mistake made against this realm on 2026-10-01 that reported astaleness figure wrong by eight layers. Swapping the sort key for the raw
string turns two tests red (verified).
9 new tests, 33 total green via the existing
check.ymlunittest discovery.The 23 existing layers are deliberately not backfilled
line-index/README.mdsays why. The only source for their payload digests isthe registry, and a registry-derived entry protects nothing by the same
argument as above: a registry that had hidden one of them would have hidden it
from the backfill too. A document that looks complete and protects nothing is
worse than a short one honest about its reach.
varve treats the index as a floor — extra layers a source serves are never
an error — so detection covers every layer from here on. If you would rather
have the backfill anyway, say so and I will add it, labelled as
registry-provenance bootstrap.
signed-index = trueis NOT setA realm declaring it with no index published fails closed on every install. The
declaration is the last step, and it lives in varve's
varve-realms.toml, nothere — so the order is: merge this, let one deposit run, confirm the
line-index-2026.10tag exists, then declare it.What I could not test here
The workflow step itself has not run — it needs the realm's signing secret and
a dispatch. What is verified locally: the appender's behaviour (9 tests, one
negative-controlled), that
varve sign-indexaccepts exactly the document theappender writes, and that the read-back
jqassertion in the step worksagainst a real signed envelope. The same publish recipe is also now exercised
end to end against a live registry by varve's own
tools/systest/line-index.sh(pulseengine/varve#233), including controls for an index signed by the wrong
key and one naming a layer the registry will not serve.
🤖 Generated with Claude Code
https://claude.ai/code/session_019TNtfRjLNhEz82G2ggeeNu