refactor(signing): walk AgentCard descriptors when pruning the canonical payload

[aeoess]

Description

Part of #1278 🦕

This refactors canonical payload pruning to walk the AgentCard descriptors alongside the JSON values. It prepares the code for handling field requirements without introducing that behavior here.

The REQUIRED-default change has been reverted pending a2aproject/A2A#2122. The resulting tree is identical to 3f90712. No canonical-byte differences were found on the tested inputs.

Evidence

• The tree at the head a091c83 is identical to the tree at 3f90712.
• tests/utils/test_signing.py: 25 passed (22 before). The 3 new tests come from the first commit.
• A differential run over 6000 randomly populated cards, two seeds, found no difference between the new pruning and _clean_empty. The run is not part of the test suite.
• On the 24 vectors in test(conformance): add Agent Card signature vectors (a2a-card-sign-v01) a2a-tck#246 at 4568f93, 3f90712 is identical to fad0482 in verdict and canonical bytes. @kuangmi-bit reproduced this independently.
• The frozen JS-signed card from AgentCard JWS: v2.6.0 canonicalizes the served JSON, the JS SDK canonicalizes the proto projection; signatures do not cross-verify a2a-go#445 (9420 bytes, sha256 2df33ff1…) canonicalizes to 6410 bytes / c5d5384a… on fad0482 and on the first commit.
• The 11 tests of test(signing): add cross-SDK signature vectors #1286 pass at the head.
• The whole suite outside tests/integration/cross_version on Python 3.10: 2169 passed in one of three runs. In the other two, tests/utils/test_telemetry.py::test_trace_function_sync_attribute_extractor_error_logged failed under -n 4. It passed in three runs on its own. This PR does not touch telemetry.
• ruff check, ruff format --check and ty check pass on the changed module.

Not in this PR

• The REQUIRED default change. It waits for [Bug]: §8.4.1 canonicalization is under-determined - two SDKs produce different bytes for the same card, and neither reproduces the section's own worked example A2A#2122.

• Optional fields with explicit presence that are set to their default (for example iconUrl: "") are still pruned. That is a separate change with its own byte impact.

• Follow the CONTRIBUTING Guide.

• Make your Pull Request title in the Conventional Commits specification.

• Ensure the tests and linter pass.

• Appropriate docs were updated (not needed, no public API change).

[github-actions]

🧪 Code Coverage (vs main)

⬇️ Download Full Report

	Base	PR	Delta
src/a2a/server/cluster/database_event_stream.py	96.94%	92.86%	🔴 -4.08%
src/a2a/utils/signing.py	95.77%	93.42%	🔴 -2.35%
Total	92.99%	92.93%	🔴 -0.05%

Generated by coverage-comment.yml

[ogasurfproject-jpg]

I measured both commits of this PR against the interop matrix from #1278 and against the signed corpus in a2aproject/a2a-tck#246. Other two SDKs as of today: @a2a-js/sdk 1.3.0 and a2a-go main 534a60fc. Baseline: a2a-sdk 1.2.1, frozen in results_20261001.json.

3f90712 (refactor): all 54 matrix cells are identical to the 1.2.1 baseline, and the corpus reading is unchanged. This agrees with "no canonical bytes change" on these cards.

cdee28e (fix): 12 cells move, all in the three cases with an empty REQUIRED field:

case	python > js	js > python	python > go	go > python
description: ""	pass to FAIL	pass to FAIL	FAIL to pass	FAIL to pass
skills: []	pass to FAIL	pass to FAIL	FAIL to pass	FAIL to pass
skills[0].tags: []	pass to FAIL	pass to FAIL	FAIL to pass	FAIL to pass

• Unchanged:
  • The cases with an empty optional field (capabilities.extensions: [], securityRequirements: [{}]). Python still agrees with JS there, not with Go.
  • The control card.
  • The frozen JS-signed production card from AgentCard JWS: v2.6.0 canonicalizes the served JSON, the JS SDK canonicalizes the proto projection; signatures do not cross-verify a2a-go#445, which still passes on all three SDKs.
• Signed corpus: the set of vectors this commit's verifier accepts equals the rule-1-as-written reading exactly.
  • S1-001, S1-003 and S1-005 are now accepted.
  • S1-002, S1-004 and S1-006 are now rejected.
  • S1-007 is accepted and S1-008 is rejected, as before.
  • On 1.2.1 and on 3f90712 the same set equals prune-empty, which is also what @a2a-js/sdk follows. a2a-go follows served-as-is.

So the expectation in the description holds on these cards. With the second commit, a card that has an empty REQUIRED field verifies between Python and Go, and stops verifying between Python and JS until the JS SDK canonicalizes the same way.

Section 8.4.1 worked example: still not reproduced byte for byte. The only difference is that REQUIRED fields absent from the input are emitted at their default:

expected  {"capabilities":{"pushNotifications":false,"streaming":false},"description":"","name":"Example Agent","skills":[]}
cdee28e   {"capabilities":{"pushNotifications":false,"streaming":false},"defaultInputModes":[],"defaultOutputModes":[],"description":"","name":"Example Agent","skills":[],"supportedInterfaces":[],"version":""}

The example's input omits those four fields and so does its output. Whether a REQUIRED field that was never set should appear in the signed bytes seems to belong with a2aproject/A2A#2122. It makes no difference on cards that carry every REQUIRED field, which is the case for the corpus and for the production card.

Reproduce (from a checkout of ogasurfproject-jpg/horizon-shield at main, workers/a2a-card-sign/interop-matrix):

pip install "a2a-sdk[signing] @ git+https://github.com/aeoess/a2a-python@cdee28e53f2166ca04fc360868cae9fc2b7a3182" rfc8785==0.1.4
npm install && (cd go && go build -o interop-go .)
python matrix.py && python watch.py

Use 3f907120f0969b5ef965ff521cf6d6554c01c6e1 for the first commit. watch.py prints the moved cells against the 1.2.1 baseline and the reading each verifier follows on the corpus.

[aeoess]

@ogasurfproject-jpg thank you, this is really useful. I reproduced the same cdee28e output for the worked example.

One thing I checked after your comment: by the time verification runs, the card is already parsed into protobuf, and at that point description: "" and no description look the same. So the SDK cannot tell those two cases apart anymore.

That seems like the real question for a2aproject/A2A#2122.

[ogasurfproject-jpg]

Confirmed on a2a-sdk 1.2.1. In the AgentCard descriptor, name, description, version and all seven repeated fields have no field presence; only provider, documentation_url, capabilities and icon_url do. ParseDict of the same card with description: "" and with no description gives equal messages and identical wire bytes, and the same holds for skills: [] against no skills. create_signature_verifier takes the parsed AgentCard, so the JSON as served never reaches the verifier.

So on the Python public API, the open point in a2aproject/a2a-tck#245 is settled before the spec settles it. Of its two resolutions, presence-preserving cannot be implemented over the message; only inject-required-defaults can, which is what cdee28e does, and it is why the section 8.4.1 worked example still differs in the REQUIRED fields it omits. a2a-go canonicalizes the served JSON, so it could implement either.

That gives a2aproject/A2A#2122 a concrete form: does a verifier canonicalize the card as received, or the card as parsed? If parsed, the spec has to say that an absent REQUIRED field is signed at its default value, and the worked example changes. If received, the Python verifier needs the served bytes (or the dict before ParseDict) alongside the message.

Nothing in a2a-card-sign-v01 (a2aproject/a2a-tck#246) depends on this: every vector there carries every REQUIRED field, so both resolutions give the same bytes on it. If it would help #2122, I can add a group where a REQUIRED field is absent, with one signature per resolution.

[kuangmi-bit]

Independent verification — heads of #1287 (cdee28e5) and #1286 — second pair of eyes, nothing below comes from your test files.

Method: pulled the frozen production card + JWKS at the pinned commit (hashes re-checked), the 13-vector corpus and the published results_20261001.json from #1286; ran a pre-patch copy and this branch side by side through the public _canonicalize_agent_card path (ignore_unknown_fields=True where a served card carries fields the proto lacks); compared the produced bytes against each vector's signed bytes; and re-derived the ES256 verification myself with cryptography (I did not call the repo's verifier — JWS input = protected || '.' || b64url(canonical_bytes), ES256 r||s assembled by hand).

The harness is anchored to a third-party run, not to my expectations: my pre-patch numbers reproduce the published a2a-sdk 1.2.1 column exactly (434/d2988a65, 376/a4b11173, 344/dc62b9c1, 417/36ee28e6).

Frozen production card (9420 / 2df33ff1…, JWKS 256 / 692fd49d…)

check	result
canonical bytes	6410 / c5d5384a… — byte-identical to the JS form, unchanged by this patch ✅
signatures[1] (JS-side) verified with the committed JWKS	PASS ✅
signatures[0] (served-as-is form)	still FAIL — unchanged, expected

Corpus (13 vectors / 6 cases) — signed-bytes = produced bytes equal the bytes the signature covers; sig = independent ES256 over our bytes.

vector	case	pre-patch	this branch	signed-bytes	sig
S0-001…004	control	434 / d2988a65	434 / d2988a65	MATCH → MATCH	PASS → PASS
S0-REJECT-005	reject	435 / c112a711	435 / c112a711	MATCH → MATCH	FAIL → FAIL
S1-001	empty_description	376 / a4b11173	393 / 9a36f162	DIFF → MATCH	FAIL → PASS
S1-002	empty_description (prune-empty signed)	376	393	MATCH → DIFF	PASS → FAIL
S1-003	empty_skills	344 / dc62b9c1	356 / cd2ef2a7	DIFF → MATCH	FAIL → PASS
S1-004	empty_skills (prune-empty signed)	344	356	MATCH → DIFF	PASS → FAIL
S1-005	empty_skill_tags	417 / 36ee28e6	427 / 7248fcdf	DIFF → MATCH	FAIL → PASS
S1-006	empty_skill_tags (prune-empty signed)	417	427	MATCH → DIFF	PASS → FAIL
S1-007 / S1-008	empty_extensions	434 / d2988a65	434 / d2988a65	unchanged	unchanged

Complete leaf delta across the whole corpus — every added path is REQUIRED, nothing else moves

S1-001/S1-002  +  /description        = ""
S1-003/S1-004  +  /skills             = []
S1-005/S1-006  +  /skills/0/tags      = []
control / empty_extensions / reject vectors: no path added or removed

So the four non-REQUIRED leaves from the always_print_fields_with_no_presence experiment (capabilities/extensions/{0,1}/required, supportedInterfaces/{0,1}/tenant) do not come back, and extensions: [] / securityRequirements: [{}] still collapse — that 6638-byte form is not reachable from this patch.

Three things worth recording

1. Python converges on rule-1-as-written and stays narrower than Go. Our three rule-1 outputs are exactly the published Go/rule-1 numbers (9a36f162 / cd2ef2a7 / 7248fcdf), but on empty_extensions Go keeps a non-REQUIRED empty (450 / afbd88a7) where we still collapse (434 / d2988a65).
2. This commit relocates the interop break rather than removing it. The prune-empty-signed twins flip PASS → FAIL — i.e. cards signed today by @a2a-js/sdk 1.3.0 and a2a-sdk 1.2.1 stop verifying in Python. No single canonical form satisfies both readings, so whoever signs with one and verifies with the other breaks; that is a verification-policy call (#2122: one reading, or accept-both-with-fallback), not a coding one. If it stays single-reading, the JS side has to move with us, otherwise Python fails half of this corpus in one direction or the other.
3. Spec 8.4.1's worked example is still not reproduced — now for the opposite reason. The example's input omits version, defaultInputModes, defaultOutputModes, supportedInterfaces, so post-parse those REQUIRED fields sit at their defaults and rule 1 keeps them: we emit six keys the example doesn't show. The example is only reproducible if "keep REQUIRED defaults" is scoped to fields the served JSON carried, never if it's scoped to the descriptor. All three SDKs fail it today (matches: python false / js false / go false), so it's worth pinning in #2122.

tests/utils/test_signing.py on this head: 39 passed. The harness (side-by-side base/patched runner + the standalone JWS verifier, ~60 lines) is available if you want to run the same sweep — say the word and I'll paste it here or as a gist.

Verdict from my side: commit 1 (descriptor walk + optional-default pruning) does what it claims and needs nothing from me; the open item is the reading switch you split into commit 2, which is exactly the #2122 dependency you flagged. Happy to run the same sweep against a two-reading/fail-open variant so #2122 has numbers for both options.

[ogasurfproject-jpg]

Thank you, @kuangmi-bit. Your 13 lengths and hashes are the canonical bytes recorded in a2a-card-sign-v01 (canonical_utf8_hex), vector for vector: S0-001 to S0-004 434/d2988a65, S0-REJECT-005 435/c112a711, S1-001 393/9a36f162, S1-002 376/a4b11173, S1-003 356/cd2ef2a7, S1-004 344/dc62b9c1, S1-005 427/7248fcdf, S1-006 417/36ee28e6, S1-007 434/d2988a65, S1-008 450/afbd88a7. So the corpus now has a reproduction that shares no code with the generator, and the ES256 side was checked without the repo's verifier as well.

On the worked example, I agree with how you scope it. "Keep REQUIRED defaults only for fields the served JSON carried" is the presence-preserving resolution in a2aproject/a2a-tck#245. On the Python public API it needs the served bytes, because ParseDict erases presence for those fields (previous comment).

A sweep against a two-reading variant would be useful for a2aproject/A2A#2122. The S1 pairs are built for it: each pair carries one signature per canonical form over the same card, so a verifier that tries both readings should accept both vectors of S1-001 to S1-006, and S1-007/S1-008 show what happens on a non-REQUIRED empty field.

[kuangmi-bit]

Addendum to the verification above — conformance view, now that #1286's corpus is public.

Taking each corpus reading as a candidate resolution and scoring verifiers against it (independent ES256, candidates = rule-1-as-written vs prune-empty vs served-as-is):

verifier	rule-1-as-written	prune-empty	served-as-is
this branch (rule-1)	13/13, 0 false accepts	7/13, 3 fa	11/13, 1 fa
prune-empty (1.2.1 / @a2a-js 1.3.0 today)	7/13, 3 fa	13/13, 0 fa	5/13, 4 fa
accept-both readings	10/13, 3 fa	10/13, 3 fa	8/13, 4 fa

So with #1287 applied, Python is fully conformant against the corpus under rule-1-as-written, and every leaf it adds is REQUIRED-only. Two things it does not settle, both now written up with numbers on #2122:

• the three prune-empty-signed twins (S1-002/004/006) reverse to MUST-REJECT under rule-1-as-written — that is the migration cost, and it is a text decision, not a coding one;
• "accept both readings" scores 10/13 with 3 false accepts against either reading, so leniency can't be used as a quiet bridge and still be called conformant.

Detail + harness pointer: a2aproject/A2A#2122 (comment)

[ogasurfproject-jpg]

Follow-up from the absent-field side, measured against cdee28e. When a REQUIRED field is absent from the served JSON, cdee28e emits it at its default before canonicalizing, so it rejects cards that a2a-sdk 1.2.1, @a2a-js/sdk 1.3.0 and a2a-go all accept today (S2-001, 003, 005, 007 and 009 in the corpus, now in a2aproject/a2a-tck#246). The section's worked example is reproduced only when rule 1 is scoped to the fields the JSON carries, and cdee28e does not accept it either.

Numbers for all 24 vectors and a proposed scoping sentence are on a2aproject/A2A#2122. Commit 1 is unaffected; this is the reading choice in commit 2 again, now with the absent-field case measured.

[aeoess]

I ran both commits against all 24 vectors in a2aproject/a2a-tck#246 at 4568f93, with a separate ES256 check that does not use the SDK verifier. The results match what @ogasurfproject-jpg reported.

3f90712 is identical to base fad0482 on all 24 vectors, both in verdict and canonical bytes. So commit 1 changes no behavior and is only groundwork by itself.

cdee28e changes 16 verdicts. It gets 24/24 under descriptor scope, and 13/24 with 5 false accepts under served scope.

So commit 2 depends on the canonicalization-scope decision in a2aproject/A2A#2122. I can split the PR and leave that commit waiting on the ruling. Whether the behavior-neutral first commit is useful separately is up to you.

[kuangmi-bit]

Independent re-run of both commits against the 24 vectors in a2aproject/a2a-tck#246 at 4568f93, for the split question.

Method: each tree in its own process, _canonicalize_agent_card taken from that tree, and the oracle is the bytes the signature covers — I verify the corpus's ES256 signature over the published canonical_utf8_hex with testkey_jwks.json using cryptography directly, never the SDK's verifier. Harness check first: it reproduces MANIFEST.observed for a2a-sdk 1.2.1 on the 13 s0/s1 vectors (13/13) and for a2a-python#1287 on the 11 s2 vectors (11/11), so the measurement agrees with the corpus's own record before it says anything new.

Results:

• 3f90712 vs base fad0482: byte-identical on 24/24. Not just the same verdicts — the same canonical bytes, zero flips. Commit 1 is behavior-neutral in the strongest sense available here: no verifier pairing and no accept/reject outcome moves.
• cdee28e: 16 flips, and they are the 16 you list (S1-001 … S1-006, S2-001 … S2-010).
• Scored strictly against the corpus's own labels (only the vectors that name a reading): cdee28e matches rule-1-as-written 13/13, rule-1-descriptor-scope 11/11 — 24/24 across the two, as you reported — and matches prune-empty 8/24, served-as-is 12/24.

For the split: commit 1 changes no bytes, so nothing downstream depends on when it lands, and it can merge on its own without touching interop. Commit 2's behavior is the descriptor reading, and a2aproject/A2A#2122 is heading for served scope, so commit 2 is the half that must wait for the ruling — or be rewritten if the ruling lands where the discussion currently points. The shape you propose is the right one.

[aeoess]

Following @kuangmi-bit's rerun, I reverted the second commit in a091c83. The resulting tree is identical to 3f90712, so the PR now carries only the descriptor refactor, with no canonical-byte differences found on the tested inputs. The REQUIRED-default change remains separate, pending the decision in a2aproject/A2A#2122.