Skip to content

fix(evolution): keep map-form outputs in map form on re-encode (#577) - #633

Open
Kshot3000 wants to merge 1 commit into
IntersectMBO:mainfrom
Kshot3000:fix/txout-map-form-preserve
Open

Kshot3000 wants to merge 1 commit into
IntersectMBO:mainfrom
Kshot3000:fix/txout-map-form-preserve

Conversation

@Kshot3000

@Kshot3000 Kshot3000 commented Oct 14, 2026 •

Copy link
Copy Markdown

Summary

Fixes #577. An output written in the post-Alonzo map form, with no script ref, came back from Transaction.fromCBORHex → Transaction.toCBORHex in the legacy array form:

Received Written back
a2 00 [addr] 01 [coin] 82 [addr] [coin]
bf 00 [addr] 01 [coin] ff 82 [addr] [coin]
a3 00 [addr] 01 [coin] 02 8200 5820 [hash] 83 [addr] [coin] 5820 [hash]

Nothing else changed, but the body bytes change, so the transaction id changes and every witness already collected no longer verifies. Same class as #574: the decoder accepts two forms and the encoder writes one without consulting the captured format.

Root cause

TxOut.makeFromCDDL's encoder picks the array form whenever scriptRef is absent and the datum is absent or a hash (canUseShelleyFormat). The captured format node for such an output says map while the encoder produces an array, so the format-guided encoder drops the format for that output and writes the array.

A pure CBOR-level conversion cannot fix this: in the array form the third element is a bare datum hash, while map-form key 2 carries the datum_option [0, hash] — only the TxOut layer knows that mapping.

Fix

Let the captured format decide the form, as the issue prescribes:

  • TxOut.withCapturedForm(encoded, format) — when the format node is a map and the encoded output is an array of 2–3 elements, rebuild it as a map: keys 0/1 take the address and value as they are, and a bare datum hash is wrapped as [0, hash]. Anything else (array format, already a map, an edit that added a script ref or inline datum) is returned unchanged.
  • TransactionBody.alignOutputsWithFormat(cborMap, format) — walks the encoded body map positionally, exactly the way the format-guided encoder consumes format entries, and applies withCapturedForm to the outputs (key 1) and to the collateral return (key 16, the same bug for a single output). Returns the input map untouched when nothing aligns.
  • Both TransactionBody.toCBOR*WithFormat and Transaction.toCBORBytesWithFormat (which also covers the cached default toCBORHex path and addVKeyWitnessesHex) run the alignment before the format-guided encode.

With no captured format — or no map node — encoding is exactly as before, so freshly built transactions do not change.

Tests

RED first on main: each map-form case below failed with the array form written back (e.g. expected …81a200…, received …8182…); the array-form controls passed. After the fix, 9 new checks in test/Transaction-with-format.test.ts:

  • definite map-form output round-trips byte-identically via toCBORHexWithFormat and via the cached default path (fromCBORHex → toCBORHex)
  • indefinite map-form output (bf … ff) round-trips byte-identically
  • map-form output with a datum hash round-trips byte-identically (the [0, hash] wrap)
  • a map-form and an array-form output side by side in one body both keep their forms
  • a map-form collateral return (key 16) round-trips byte-identically
  • body hash of the datum-hash transaction is unchanged — verified via CML hash_transaction
  • addVKeyWitnessesHex leaves the map-form body bytes (and hash) unchanged
  • the same round trip at the TransactionBody WithFormat level

Full package suite: 1752 passed / 75 skipped / 0 failed (test files 107 passed / 1 skipped of 108) — current main's baseline plus the 9 new tests. tsc --noEmit and eslint clean; a patch changeset is included. Prettier already flags the three touched source files on unmodified main; my versions produce exactly main's prettier diff-line counts (28 / 54 / 6), so the change adds no new formatting drift.

Rebase note (2026-10-16): rebased onto current main after the format-cache / gainsLowerKey changes in Transaction.toCBORBytesWithFormat conflicted with this PR. The alignment now runs off upstream's bodyFormat and the aligned body feeds both return paths (the early replay return and the witness-set path); alignOutputsWithFormat / withCapturedForm are unchanged. RED re-confirmed on main and the neighbouring format suites (10 files / 157 tests) re-run green — see the rebase comment below for detail.


I'm an independent builder working on Cardano tooling — not affiliated with the Cardano Foundation, IOG or Intersect.

@Kshot9000 (https://x.com/kshot9000) · Tips: addr1q8hnl6vl5a6k3rw3n5g3jtte696zcl76kfatzv7gpswa9r0dj7fma6klq55y4ffm7tf0em09udnyhuk4ah92pl5x9jpqjae44v

…sectMBO#577)

The TxOut schema encoder picks the legacy array form whenever an output
has no script ref and its datum is absent or a hash, even when the output
was decoded from the post-Alonzo map form. The captured format node says
map while the encoder produces an array, so the format was dropped for
that output: the body bytes changed, the transaction id changed, and
witnesses already collected stopped verifying.

The captured format now decides the form. TxOut.withCapturedForm rebuilds
an array-encoded output as a map when its format node is a map — a bare
datum hash is wrapped as the datum_option [0, hash] — and
TransactionBody.alignOutputsWithFormat applies it to the outputs (key 1)
and the collateral return (key 16), matched positionally the way the
encoder consumes format entries. Transaction.toCBORBytesWithFormat
aligns the body inside the transaction tuple the same way. With no map
format node nothing changes: array-form outputs, outputs with an inline
datum or script ref, and freshly built transactions encode as before.

Tests: 9 new checks in Transaction-with-format.test.ts — definite and
indefinite map-form round trips via WithFormat and the cached default
path, a map-form output with a datum hash, map and array outputs side by
side, a map-form collateral return, CML body-hash equality,
addVKeyWitnessesHex body stability, and a TransactionBody-level round
trip. Full suite: 1687 passed / 75 skipped / 0 failed.
@Kshot3000
Kshot3000 force-pushed the fix/txout-map-form-preserve branch from a6f64d5 to db92912 Compare October 17, 2026 02:59
@Kshot3000

Copy link
Copy Markdown
Author

Rebased onto current main (db92912) — this branch was conflicting in Transaction.ts after the format work that landed upstream since the PR opened (#654 witness-set intact, #658 fake-witness redeemer indices, and the format-cache / gainsLowerKey changes in toCBORBytesWithFormat).

The conflict was semantic rather than line noise, so the fix was re-derived in place rather than marker-resolved:

  • Upstream now splits the root format into bodyFormat / witnessSetFormat up front and added the gainsLowerKey condition to the early replay return. The output alignment now runs off that same bodyFormat, and the aligned body feeds both return paths (previously only the tuple path existed in this shape) — the early return encodes alignedTuple, and the witness-set path returns [body, witnessSet.value, …] as before.
  • TransactionBody.alignOutputsWithFormat and TxOut.withCapturedForm merged unchanged; the fix is not superseded upstream (no equivalent on main).

Verification on the rebased branch:

  • RED on main, GREEN with the fix: the 9 new checks in test/Transaction-with-format.test.ts fail on main (vitest's bail stops the file at the first failure, so three were also confirmed individually: definite map-form via WithFormat, map-form collateral return, and the TransactionBody-level round trip) and all 15 tests in the file pass with the fix.
  • Neighbouring format suites re-run: 10 files / 157 tests pass, including the newer Transaction.CanonicalWitnessAdd, Transaction-format-preservation and CBOR.KeyOrder.
  • Full package suite: 1752 passed / 75 skipped / 0 failed (test files 107 passed / 1 skipped of 108) = current main baseline 1743 + the 9 new tests, counted from this run's own log.
  • tsc --noEmit and eslint clean; prettier parity exact against upstream on all four touched files (28/54/6/0 diff lines, identical before and after).

I'm an independent builder working on Cardano tooling — not affiliated with the Cardano Foundation, IOG or Intersect.

@Kshot9000 (https://x.com/kshot9000) · Tips: addr1q8hnl6vl5a6k3rw3n5g3jtte696zcl76kfatzv7gpswa9r0dj7fma6klq55y4ffm7tf0em09udnyhuk4ah92pl5x9jpqjae44v

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TxOut: map-form outputs without a script ref are re-encoded as arrays, changing the transaction id

1 participant