Skip to content

feat(verify): the guard, the table and the vectors for obligations 2 and 3 of #116 - #347

Open
lywinged wants to merge 12 commits into
agentrust-io:mainfrom
lywinged:work/116-pr
Open

lywinged wants to merge 12 commits into
agentrust-io:mainfrom
lywinged:work/116-pr

Conversation

@lywinged

@lywinged lywinged commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Part of #116, and deliberately not closing it.

This lands the half the ruling released. The other half stays with that issue: "Held: the
normative verification.md text for obligations 2 and 3 ...
I carry it." Closing on merge would leave the held text, the deferral of obligations 1 and 4, and
the policy: take-internal label sitting on a closed issue with nothing tracking them, which is
the same shape as the reason row 4 was taken out of this set: coverage that becomes unowned when
the issue closes. So there is no closing keyword here, and the issue stays open for the half its
author carries.

What this changes

Verifier behaviour and conformance vectors for obligations 2 and 3 of #116, quoted from that
issue so this page stands on its own:

  1. A conformant verifier declares the set of versions it supports and MUST refuse versions
    outside that set — refuse, not best-effort.

  2. The verification statement includes the version under which verification ran.

These are the two the issue thread settled as landable without a sponsor. Obligations 1 and 4
are deferred there; the ruling says only that they stay deferred, and the reason developed
earlier in the thread was that no vector written against the current schema can separate them
from the eat_profile const.

No normative text is proposed here. The draft docs/verification.md section for these
obligations is deliberately not in this branch. It is on lywinged/trace-spec
work/116-proposal-text, one file off this repository's main, for the maintainer who said he carries it, to take
or discard. The enumeration of three grounds the ruling said it would keep in whatever text lands
is in that file rather than here, under "Declared support" and "Statement content", because that
is the text it was asked to land in. Nothing in it is proposed upstream by this pull request.

The diff

Twelve commits, nine of them mine. The first three were the branch as opened, rebased onto main
each of the three times it moved while it was being prepared. Then main merged in rather than
rebased onto, twice, because it moved again while the review was in hand, so nothing of mine has
moved underneath what has already been read. Then the review's own changes: 67c3ef1 is what it
asked for, and 6f63406, e8b1c45, 97232b7, 591dee4, 412dfed and 963b392 are passes over
prose that still described the behaviour before it, with the guards that came out of them. The changelog conflict is
resolved in the first merge, 7803701, by keeping all three entries under Unreleased. Every figure
below is recomputed on the current base rather than carried over.
23 files, +3110 −52.

The ruling released three things: the guard, the table and the vectors. All three are here.
The guard is tests/test_verifier_compatibility_separation.py. The vectors are the eight fixtures.
The table is the one in examples/verifier-compatibility/README.md, one row per fixture giving
the outcome and the rule it pins, which is the five-row table from the issue rewritten
against what was committed rather than against what was proposed, minus the row the ruling took
out. It runs to eight rows because it lists every fixture in the directory rather than only the
rows that were proposed; the row the ruling took out was vector 10, and 03 and 08 have since
followed it to the cutover's own coverage.

New examples/verifier-compatibility/ (eight fixtures, README with the table, generator), tests/test_verifier_compatibility_fixtures.py, tests/test_verifier_compatibility_separation.py, tests/test_verifier_compatibility_diagnostics.py
Changed, this issue's behaviour src/agentrust_trace/sign.py, validate.py, revocation.py, __init__.py
Changed, consequences of the above tests/test_sign.py, test_adequacy_all_sets.py, test_public_functions_raise_what_they_document.py, test_safe_integer_range.py, CHANGELOG.md

What it adds

DEFAULT_ACCEPTED_PROFILES is exported rather than left as a private default because
obligation 2 requires a verifier to declare the set it supports, and a caller cannot declare
what it cannot name. It makes addressable a rule spec/trace-v0.2.md already states under
"Changes from v0.1", quoted because the whole non-breaking claim rests on it:

Cutover, not coexistence. A v0.2 verifier MUST require tag:agentrust-io.com,2026:trace-v0.2
and MUST reject the v0.1 identifier. It MUST NOT accept both.

verify_record() takes an accepted_profiles set, refuses a record whose eat_profile is
outside it, and refuses the configuration itself where the declared set names the superseded
identifier or a profile this build carries no schema for. validate.profiles_with_schema()
reads the packaged schemas for their eat_profile const and is that second ceiling.
VerificationResult gains profile and accepted_profiles, both defaulted: a declaration
that lives only in the verifier's documentation cannot be recovered from a result read years
later, which is the condition the issue opens with.

One process note

require-maintainer-approval.yml returns early when the author is in its MAINTAINERS list, and
this account is in it, so the gate check passes on this pull request by skipping rather than by
anyone approving. It can merge on CI alone. I am not merging it: it is mine, it carries a
changes-requested review, and the order the next section states, that verifier behaviour and
vectors land while the normative text follows with a maintainer carrying it, is a maintainer's to
agree rather than mine to assume.

Type of change

  • Editorial (typo, link fix, clarification: no normative effect)
  • Non-breaking spec change (new optional field, new platform profile, informative addition)
  • Breaking spec change (requires 14-day comment period and Project Lead sign-off)
  • Schema change
  • Example addition

Neither box fits, and the template has none for code and tests, so both are ticked and the
position is stated instead.

Non-breaking is exact: no signature, no schema, no wire format, and no caller has to
change. verify_record() gains a parameter and it is defaulted. Every call site in this repository keeps its behaviour, because
verify_record() already accepted the v0.2 profile alone; a caller outside it that passes
nothing sees no change for the same reason.

"Non-normative" needs qualifying rather than claiming. No RFC 2119 keyword is added to any
document under spec/, and none is added to docs/. But a conformance vector set is what implementers get
measured against, and this one still encodes the stronger reading of obligation 2: a declared
set naming an unusable profile is refused, which the draft text does not say in those words. It
was stricter in a second way until the review, in asserting which rule refused, and that half is
retracted: failure is informative and nothing portable reads it. It is normative in effect and
informative in status, which is the order the ruling set deliberately: verifier behaviour and vectors land, the text follows
with a maintainer carrying it. Merging this is agreeing to that order, not only to the code.

Spec section

None modified. The behaviour implements the cutover paragraph quoted above, which is at line 24
of spec/trace-v0.2.md under "Changes from v0.1", not inside a numbered section. The gap the
vectors point at is in docs/verification.md, which this change does not edit.

The separation guard

The part the thread asked for ahead of more vectors. Eight wrong implementations run against the
whole vector set and the module records what each one is caught by, so a figure that stops
being true fails loudly rather than expiring quietly.

The failure it exists to catch already happened once here: the previous version of this set
stopped separating when #156 made verify_record() validate against the packaged v0.2 schema,
a second gate over the same inputs, and nothing inside the set could see it because its own
tests run the real verifier and compare verdicts, and the verdicts did not move. That is a
different failure from the one the issue opens with, verifiers accepting artifacts under
semantics they do not implement; this one is the measurement of that failure going stale.

Measured on this branch:

Vectors 8
Separated from a verifier implementing none of the obligations 4: 01, 04, 06, 09
Inert 4: 02, 05, 07, 11
Wrong implementations in the panel 8
Wrong implementations no vector catches 1
Refusal rules carried by a single vector 1

The effective set is four. The other four carry records whose profile is not v0.2, so the schema
refuses them before any profile rule runs, and they demonstrate nothing about these obligations
today. They are kept because they become live the day both gates come off, and deleting them
now would mean rediscovering them then; they are not counted as evidence that the set tests
anything.

What no vector catches, recorded with its expiry rather than left implicit: a verifier that
satisfies the statement obligation by writing the profile in as a literal. The argument that no
vector written against this build can catch it is a claim about the vectors this build admits,
not about vectors in general: the build packages a schema for two profiles and a conformant
declared set must exclude the superseded one, so exactly one profile can appear in a conformant
statement, and a literal and an observation print the same string. Varying the declared set does
not help, because the set that would distinguish them is the non-conformant one. The entry
expires when a second admissible profile schema ships, and a test asserts that condition
directly.

One refusal rule is carried by a single vector, against #124's standard of at least two
independent vectors per rule: no_accepted_profiles, by 06. It was three until the review;
retiring 03 and 08 into the cutover's coverage took the other two with them, which is a better
record than the one this page carried, and it is measured rather than claimed: KNOWN_THIN
records the one that is left, and the adequacy test fails if the measured set of thin rules stops
matching it in either direction. Obligation 3's statement rule is carried by one vector too, 01,
and will not appear in that table: the framework counts refusal boundaries and 01 is the set's
only positive vector.

The panel is not independent of the vectors. It was written after them by the same hand, so
it agrees with them by construction, which is exactly the caveat #124 makes about a vector set
and the checker that runs it. A reader who stops at the table above should not take the guard
for an outside instrument. The cross-run that issue proposes, a second set through a verifier
its author did not write, is what would fix it, and nothing here substitutes for it.

A defect introduced here and fixed here

accepted_profiles was materialised with tuple(). A bare profile URI passed instead of a
sequence iterated as characters and was refused for naming each of them as a profile this build
carries no schema for: the right verdict reached for the wrong reason. An int or None left
verify_record() raising TypeError, which its docstring does not list. Same shape as the
trusted_key_identifiers case closed in revocation.py by #325, and found the same way, by
that sweep once this parameter was swept.

The contract now: anything that is not an iterable of str, and anything iterable whose members
are not all str, raises ValueError naming the type it got, which is what the docstring
already promises for every other rejection. A bare str is refused as a type rather than
iterated. Six cases pin it: a bare string, an int, None, a dict, bytes, and a list
whose members are not strings, plus a separate control that a real tuple gets past the shape
check and is refused for what it contains.

The question this diff asked, and the answer it was given

The ruling on the v0.1 row was that a vector for the cutover tests an existing rule rather than
an obligation this issue proposes, and that when the issue closes the coverage becomes unowned.
That vector went out, its id retired rather than reused, and its rule got a test in
tests/test_sign.py beside the cutover's own.

This page asked whether the same reasoning reaches 03, which refuses a record carrying the
superseded identifier, and 08, which refuses a verifier declaring both. It does, and the review
said so. Both are retired the same way, ids retired rather than reused, with the explicit
configuration each one stated pinned in tests/test_sign.py. Coverage preserved is measured
rather than asserted: removing the record-side guard fails four tests there, removing the
set-side guard fails three, and the test that carries both retired configurations fails under
either mutation.

Both were inert, so the set still separates four of what are now eight vectors, and two of the
three single-vector rules left with them.

Two changes in files this issue did not ask for

The superseded schema's exemption in tests/test_safe_integer_range.py is rewritten. It
exempted trace-v0.1.json from the integer-bound rule on the ground that nothing loads it, and
checked that by asserting no Python file mentions the filename. profiles_with_schema() reads
every packaged schema's eat_profile const, so that proxy became false.

The third option is worth naming because it is the one that would have avoided touching the
file: have profiles_with_schema() skip the superseded schema. It does not work. The ceiling is
what makes a declared set naming an unusable profile refusable, and if the superseded identifier
were outside the ceiling, a set naming it would be refused as a profile this build cannot check
rather than as the cutover violation it is. The right verdict for the wrong reason, which is the
defect class this set exists to separate.

So the exemption's reason is rewritten to the property it actually rests on, that no record is
ever validated against the file, and three assertions in the test now measure it rather than
proxy it: the profile is in the ceiling profiles_with_schema() returns, the live validator's
eat_profile const is the current profile and not this one, and that reader returns strings
rather than schema objects. The alternative was to stop naming the file in a docstring, which
would have dodged the grep and left a reason standing that is no longer true.

An assertion was removed from that same test. A fourth leg asserted that a record carrying
the superseded profile is rejected on the eat_profile path. Three mutations were run against
it: dropping the live schema's const, setting it to the superseded identifier, and setting the
superseded file's const to the live one. All three fail an earlier assertion in the same test
first, so no positive control reaches the fourth, and it reported nothing. It is recorded in a
comment where it stood rather than kept. This is a coverage deletion in a file this issue did
not ask us to touch, decided on the above and open to being reversed.

_TRACE_PROFILE_V0_1 stays private. Its comment says no caller should be able to spell it
without reading the file. This change does not export it: the tests import the private name
in-repo. The fixtures do print the identifier as a JSON literal, which is spelling it, and there
is no way to publish a vector for a rule about an identifier without naming the identifier. The
tension is real and resolved in favour of not widening the public API.

What the review settled

expected.failure is informative. The portable adapter asserts the verdict and the statement and
nothing about which rule refused, because the draft text asks for a coarser distinction and asks
for it as a SHOULD, so asserting a rule name was this set asking more of a foreign implementation
than the text it encodes. Two controls hold it open, test_a_generic_refusal_passes_this_set in
the adapter and test_a_generic_refusal_is_separated_by_nothing in the separation module, and both
were checked by putting an assertion on the cause back and watching them fail. This library's own
refusal messages are pinned in tests/test_verifier_compatibility_diagnostics.py, whose docstring
closes by saying that nothing in that file may be read as a conformance requirement.

Still open

  • The panel's dependence on the vectors, above.
  • agentrust-io/trace-tests at 660cf84 contains no reference to these vectors or their
    failure names, so removing one breaks nothing there. That is the one consumer measured; no
    claim is made about any other.

Checklist

  • DCO sign-off on all commits (git commit -s)
  • CHANGELOG.md updated (for any normative change)
  • Breaking changes marked with <!-- CHANGED: #NNN: description --> in spec text
  • Backward compatibility statement included (for breaking changes)

The changelog entry is under Added, written because the public API grows even though no
normative text changes. The last two do not apply: nothing breaks.

1720 passed and 6 skipped: test_version declining to compare an installed distribution
that is not this source tree, and five diagnostics cases whose refusal is not a complaint
about a member of the declared set. ruff check src tests scripts,
tools/check_dashes.py and mypy src/agentrust_trace, which are the scopes CI runs, are clean.

…eclarable profile set (agentrust-io#116)

Verifier behaviour and conformance vectors for the two obligations agentrust-io#116's thread
settled as landable without a sponsor: a verifier declares the set of profile URIs
it supports and refuses anything outside it, and the verification statement names
the profile it ran under. The normative `docs/verification.md` text for those
obligations is not here and is not proposed; a maintainer carries it.

What a verifier can now declare. `DEFAULT_ACCEPTED_PROFILES` is the rule
`spec/trace-v0.2.md` already states, made addressable: accept v0.2, refuse the
superseded identifier, never both. `validate.profiles_with_schema()` reads the
packaged schemas for their `eat_profile` const and is the ceiling on any declared
set, so a verifier cannot declare support for a shape it has no way to check. The
superseded schema is read for that const alone and is still never validated
against, which is why the exemption in `tests/test_safe_integer_range.py` is
rewritten rather than dropped: it rested on a grep for the filename, that proxy
became false the moment anything read the directory, and the property it stands
for is now measured instead.

What the statement carries. `VerificationResult` gains `profile` and
`accepted_profiles`. The declared set is there because a declaration that lives
only in the verifier's documentation cannot be recovered from a result read years
later, which is the condition the issue opens with.

The guard, which is the part asked for ahead of more vectors. Nine wrong
implementations run against the whole set and the module records what each is
caught by, so the figure fails loudly instead of expiring quietly the next time a
gate lands over the same inputs. Measured here: the set separates 4 of its 10
vectors against a verifier implementing none of the obligations. One recorded
shortfall stands, a verifier satisfying the statement obligation with a literal,
uncatchable while exactly one profile is admissible in this build.

Found while writing this and fixed here: `accepted_profiles` was materialised with
`tuple()`, so a bare profile URI iterated as characters and was refused for naming
each of them as an unschemaed profile, the right verdict for the wrong reason, and
an `int` or `None` left `verify_record` raising an undocumented `TypeError`. Same
shape as the `trusted_key_identifiers` case closed in `revocation.py`, and found
the same way, by the keyword-argument sweep once the parameter was swept.

1638 passed, 1 skipped; ruff and mypy clean.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
@lywinged
lywinged requested a review from a team as a code owner September 14, 2026 17:22
@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Contributor Check: UNKNOWN

Check Result
Profile UNKNOWN
Credential LOW
Overall UNKNOWN

Automated check by AgenTrust Contributor Check.

@github-actions github-actions Bot added the needs-review:UNKNOWN Contributor check flagged UNKNOWN risk label Sep 14, 2026
`tools/check_dashes.py` is a CI step and this branch did not run it, so three dashes
carried in from the fork went out with the pull request: one in `verify_record`'s
docstring and two in `test_sign.py`. Replaced in the house style, a full stop in two
cases and a comma in the third. No behaviour changes and no test changes.

All five CI steps now run clean locally, with the same arguments the workflow uses:
ruff, the dash check, mypy on `src/agentrust_trace`, and pytest with coverage.
1638 passed, 1 skipped.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
The documentation moved its example model from a real provider and model id to
`example-provider` / `example-model-1`, and these fixtures still carried the old
pair because they were written against the adapter, which that change did not touch.
Regenerated rather than edited: the records are signed, so a hand edit to a signed
field would have left ten invalid signatures behind. The generator carries the new
values and re-signs; `test_every_fixture_signature_is_genuine` passes on the result.

Fixture data only. Every figure recomputed and none moved: ten vectors, four
separating (01, 04, 06, 09), six inert, nine panel rows, one wrong implementation no
vector catches. All five CI steps clean, 1638 passed and 1 skipped.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>

@imran-siddique imran-siddique left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the two open questions:

  • Make failure informative for portable conformance. Keep tests for this library’s diagnostic messages separately. A verifier applying the agreed rules and returning a generic refusal should pass the portable suite. Add that control and remeasure the separation figures; the current panel records three affected vectors while the README says four.
  • Move vectors 03 and 08 into the cutover’s own coverage, following the same ownership decision as vector 10. Preserve their coverage.

Please also resolve the changelog conflict against current main, regenerate the fixtures, and rerun the checks. The current head passes 200 focused tests and hosted CI.

Could you update this by September 18?

Resolves the CHANGELOG conflict under Unreleased/Added by keeping all three entries:
this branch's, and the two that landed on main while it was in review.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
… keeps its own coverage

Both changes were asked for in review on the pull request.

`expected.failure` is informative. The portable adapter asserts the verdict and the
statement and says nothing about which rule refused, because the draft text for agentrust-io#116
asks a verifier to report refusal-for-an-unimplemented-profile distinguishably from a
verification failure, which is coarser than a rule name and is a SHOULD. Asserting the
label was this set asking more of a foreign implementation than the text it encodes.
Two controls hold it open: test_a_generic_refusal_passes_this_set runs every vector
through a verifier that applies every rule and names none of them, and
test_a_generic_refusal_is_separated_by_nothing does the same at the panel. Both fail if
an assertion on the cause comes back, measured by putting one back.

This library's refusal messages are still worth pinning and are now pinned separately,
in tests/test_verifier_compatibility_diagnostics.py, which says in its first paragraph
that nothing in it is a conformance requirement.

The panel loses the near miss that only existed because the comparison looked at the
cause, and gains it back as a control. One recorded figure moves with that revert: the
membership reading no longer fails vector 06, which it refused for the right verdict
under a different rule, and a different rule is not a wrong answer any more.

Vectors 03 and 08 are retired into the cutover's own coverage, the same ownership
decision that moved vector 10: the rule they pin is merged normative text, not an
obligation this issue proposes, and coverage inside a proposal's set becomes unowned
when the proposal closes. Their ids are retired rather than reused, and the explicit
configurations they stated are pinned in tests/test_sign.py beside the cutover's other
tests. Coverage preserved and measured rather than asserted: removing the record-side
guard fails four tests there and removing the set-side guard fails three, and the new
test fails under both.

The set is eight vectors and still separates four, so the two that left were inert. The
rules carried by a single vector drop from three to one.

Also merges current main, resolving the changelog conflict by keeping all three
Unreleased entries, and regenerates the vectors from the generator. ruff, the dash
check and mypy clean; 1715 passed and 6 skipped.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
…sserts the label

A cold read of the reply this branch is about to be handed back with found that the
change announces one thing and the files say another, in the module a reviewer opens
to check it. Everything here is prose and comments except one control and one guard,
and it is all the same defect: text that was true before the review.

`_v_wrong_reason`'s docstring still said the vector set is stricter than the text it
encodes, that the adapter asserts `expected.failure`, and that resolving it is the
maintainer's call. It was resolved, by the review, in the direction that docstring
named as one of the two options. It now says so and says why this verifier sits outside
the panel.

The counts that moved with the retirement and were left behind: ten near misses where
the panel holds eight, 4 of 10, all ten vectors, vectors 04 09 and 10 where 10 left the
set two days ago, three thin rules where the table shows two, eleven correctly signed
records where there are eight, and a 932-test figure from a suite that is now 1717.

The new control's own docstring carried a number from the wrong instrument: it said
four vectors refused the generic verifier, which was the panel's figure. Measured at
the adapter, which is what that control runs, it is every one of the seven refusal
vectors. Re-run, not recalled.

The diagnostics module carried markers and a lambda for two labels no vector uses any
more, left behind when 03 and 08 were retired. Both are gone, and the test that checked
every used label has a marker now checks the other direction too, which is the one that
went quiet.

The README told an adapter author to look in `FAILURE_MARKERS`, which is no longer in
the adapter, and claimed every signature is asserted independently. Four of the eight
records are re-verified; the other four are refused by the record schema before the
signature is read, and the file that two places sent the reader to for them has never
existed in this repository. The README now says what is covered, and the two pointers
are gone rather than left promising a file.

The retired-vectors test had a control whose comment described a call it did not make.
The call is there now, and the control below it says what it controls for.

Also merges main again, which had moved one commit further.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
… third

Three assertions survive in test_nothing_validates_a_record_against_the_superseded_schema
and the comment recording what was taken out called it the third. The pull request
description had it right and the file had it wrong, which is the wrong way round.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
… unique to it" was false

Checking the pull request's quotations against issue 116 turned up one more claim of the
same shape as the rest, in the same module: the docstring said vector 09 had nothing
unique to it while 09 has had the "declared set checked, last member only" column to
itself throughout. Every one of the four live vectors is the sole entry in some column,
so on this measurement none of them is deletable.

Written into a test rather than into the paragraph that was wrong, because that
paragraph is the third claim in this module to have gone stale without anything noticing.
test_no_live_vector_is_redundant fails in both directions: a vector that stops being
anyone's only catch, and a vector that becomes one.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
…ger here

A wider grep than the last one, over every file the review's commits touch rather than
the five a private checklist happened to name.

`gen_vectors.py` told a reader of vector 04 to "see vector 08" for why the v0.1
identifier is excluded for a different reason. 08 was deleted by the same change. That
file is published with the set, so the pointer breaks in somebody else's hands rather
than in ours; it now names the cutover and where its coverage lives.

The separation module described 10 and 08 in the present tense as one separating and the
other not, written when 10 had left and 08 had not. Both have, and the paragraph says so.

`test_the_published_artifacts_name_no_vector_that_is_not_here` is the guard: every vector
id the README or the generator names has to be a file in the directory. History belongs
in the test modules, which are read by us; these two describe the set as it stands.
Proven by putting "see vector 08" back and watching it fail.

The fixture for 04 is regenerated, since the text that changed is inside it, and
test_generator_reproduces_its_committed_fixtures is what caught that rather than a
reading of the diff.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
…f one per defect

Three defects of the same shape shipped inside this change, and each was answered with a
guard aimed at that one shape: a constant that had moved modules, a test file that never
existed, a vector id that had been deleted. This is their general form, so the fourth
does not need its own.

test_every_name_this_set_cites_resolves recovers the citations from the files rather
than from a list: every repository path, every backticked test name and every backticked
constant the set's own surface names has to exist. Proven three ways, by pointing a path,
a test name and a constant at something that is not there.

The first thing it caught was a false positive of its own making. A repository name
ending in one of the path words, followed by a slash, yields a match starting mid-word,
and the comment explaining that spelled the bad match out, so the guard tripped on its
own explanation. The lookbehind fixes the class and the comment is worded rather than
quoted, because a guard whose explanation trips it teaches the reader to weaken it.

The two guards added earlier were mutated to check they still guard: an unused marker in
the diagnostics table fails its test, and giving vector 09's column a second entry fails
the redundancy test.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
…ve label once

A reader grepping the portable adapter for `expected.failure` finds one use and could
reasonably conclude the contract is looser than the module's own docstring claims. It is
test_the_precondition_check_fires_and_covers_every_vector_that_needs_one, which uses the
label to derive which fixtures must declare a premise: an assertion about the fixtures,
which no run of any implementation can change. Said where that reader lands rather than
in a reply they will not have.

Signed-off-by: LouieLuNZ <48041247+lywinged@users.noreply.github.com>
@lywinged

Copy link
Copy Markdown
Collaborator Author

Done, on 963b392.

failure is informative now. The portable adapter asserts the verdict and the statement and
nothing about which rule refused. The control you asked for is test_a_generic_refusal_passes_this_set: it runs every vector through a verifier that applies every
agreed rule and reports one generic refusal, and the eight pass. There is a second at the panel's
level, test_a_generic_refusal_is_separated_by_nothing. Both fail if an assertion on the cause
comes back, which I checked by putting one back rather than assuming. This library's own refusal
messages are in tests/test_verifier_compatibility_diagnostics.py, whose docstring closes by saying
that nothing in that file may be read as a conformance requirement.

On the four and the three, both numbers were wrong for where they sat, and the disagreement was
the real finding.
SEPARATION recorded 04, 06 and 09 for that near miss, three, and it was
the panel's figure. The README's four was in a paragraph about the adapter, and at the adapter the
figure was neither: a verifier that erases the cause failed every refusal vector, seven of the eight
today. So the README was not a stale copy of the panel, it was a different instrument reported with
the panel's kind of number. All of it goes with the entry itself: comparing the refusal's cause was
the widening that put that near miss in the panel, and reverting it is what making failure
informative means at that level. One other recorded figure moved and only one, the membership
reading, which no longer fails 06.

03 and 08 are retired into the cutover's own coverage, ids retired rather than reused, following
10. The explicit configuration each one stated is pinned in tests/test_sign.py beside the cutover's
other tests. That coverage is preserved is measured rather than asserted: removing the record-side
guard fails four tests there, removing the set-side guard fails three, and the new test fails under
both.

The set is eight and still separates four, so the two that left were inert, and the rules carried
by a single vector drop from three to one. Every figure remeasured rather than carried over:

before now
Vectors 10 8
Separated from a verifier implementing none of the obligations 4: 01, 04, 06, 09 4: 01, 04, 06, 09
Inert 6 4: 02, 05, 07, 11
Wrong implementations in the panel 9 8
Wrong implementations no vector catches 1 1
Refusal rules carried by a single vector 3 1

Six further passes found the files still saying the old thing, and that is worth naming
rather than leaving for you to find.
_v_wrong_reason's docstring still read "the vector set is
meanwhile stricter than the text it encodes ... the adapter asserts it", forty lines from the
control that now proves the opposite. Seven counts had moved with the retirement and been left
behind, among them ten near misses for a panel of eight and eleven signed records for a set of
eight. The new control's own docstring quoted the panel's four where its own instrument measures
seven. The README sent an adapter author to FAILURE_MARKERS, which is no longer in the adapter,
and two places pointed at a signature test that has never existed in this repository. One claim in
the separation module was simply false: it said vector 09 had nothing unique to it, and 09 has had
a column of the panel to itself throughout. And the generator told a reader of vector 04 to "see
vector 08", which the same change had deleted.

The ones a test can recognise are pinned by one now, since a paragraph is what failed each time.
Three guards came out of individual defects: every label the diagnostics module carries
must be used by a vector and every label used must be carried, no live vector may stop being some
column's only catch without the module saying so, and no vector id may be named in the README or
the generator unless it is a file in the directory. A fourth is their general form, since three
defects of one shape deserve one guard rather than three:
test_every_name_this_set_cites_resolves recovers every repository path, test name and constant
the set's own surface cites and requires each to exist. Each was proven by reintroducing the defect
and watching it fail, and the general one first caught a false positive of its own making.

The counts and that docstring are pinned by nothing, and I would rather say so than imply the
class is closed: a number that has quietly stopped being true is not a shape a test recognises,
and what stands in for a guard there is recomputing them rather than carrying them over, which is
what the figures above are.

Seven commits are mine: 67c3ef1 is the change you asked for, and 6f63406, e8b1c45, 97232b7,
591dee4, 412dfed and 963b392 are passes over prose and the guards that came out of them. git show on each is
the whole of what I changed. main is merged rather than rebased, twice,
because it moved again while this was in hand, so nothing of mine has moved underneath what you
read. The changelog conflict is resolved in the first of those merges, 7803701, by keeping all
three entries under Unreleased. The vectors are regenerated from gen_vectors.py and the remaining
eight come back byte-identical. ruff check src tests scripts, tools/check_dashes.py and
mypy src/agentrust_trace, which are what CI runs, are clean, and the suite is 1720 passed and 6
skipped: test_version declining to compare an installed distribution, and five diagnostics cases
whose refusal is not a complaint about a member of the declared set.

One thing I have added to the description rather than assumed: the gate check passes here by
skipping, because the workflow returns early for a maintainer author, so this can merge on CI alone
and on my own say-so. I am not merging it.

The description carries the same figures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:UNKNOWN Contributor check flagged UNKNOWN risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants