Skip to content

fix(evolution): decode deeply nested CBOR and Plutus data without recursing per level (#662) - #665

Open
Kshot3000 wants to merge 1 commit into
IntersectMBO:mainfrom
Kshot3000:fix/cbor-deep-nesting-662
Open

Kshot3000 wants to merge 1 commit into
IntersectMBO:mainfrom
Kshot3000:fix/cbor-deep-nesting-662

Conversation

@Kshot3000

Copy link
Copy Markdown

Summary

Fixes #662. The ledger limits the size of each leaf of Plutus data, not its depth (PlutusCore/Data.hs, decodeData), so a datum can nest roughly as deep as the transaction size allows — around 16,000 levels. Both decoders recursed once per level:

  • CBOR.fromCBORHex("81".repeat(1719) + "00") threw RangeError: Maximum call stack size exceeded
  • Data.fromCBORHex("81".repeat(1250) + "00") threw a ParseError wrapping the same RangeError

And since Blockfrost, Koios, Maestro and Kupmios all decode each inline datum while converting UTxOs, one UTxO carrying such a datum made getUtxos() fail for the whole address. Reproduced locally before the fix at several depths; after the fix both entry points decode a list nested 16,000 levels deep and the decoded structure measures exactly 16,000 levels.

Changes

  • CBOR.ts — the offset decoder (decodeItemAt and its array/map/tag helpers) now runs on an explicit stack of open containers instead of recursing. Leaf decoders are untouched, and definite/indefinite handling, maps-as-objects, key-order capture and the format tree keep the exact shapes the recursive version produced (the CBOR-with-format suite covers this).
  • Data.ts — cborValueToPlutusData converts on an explicit action stack. It also stops calling the CBOR.is* guards: those are Schema.is checks that validate the whole subtree recursively, which is the same unbounded recursion in a second place (and quadratic work on top). The structural guards accept exactly the same nodes for decoder-produced values, validation order (a tag's shape is checked before its fields convert, children convert left to right) and every error message are unchanged.
  • Data.ts — Data.fromCBORBytes / Data.fromCBORHex now decode through those two functions directly and wrap failures in a ParseError, as the schema path did. Routing through FromCBORBytes composed with FromCDDL parses the value against the recursive CBOR and Data schemas, and Effect's parser recurses once per level there too, so the schema round trip overflowed before the conversion ever ran.

One scope note, stated plainly: the FromCBORBytes / FromCBORHex / FromCDDL schemas are unchanged and keep their existing validation behaviour — decoding a ledger-depth datum through the schemas remains bounded by Effect's recursive parser, which is library machinery this repo doesn't control. The plain decode functions and the conversion named in the issue are depth-safe.

Tests

  • 10 new regression tests: 16,000-level definite and indefinite lists, maps and tags in CBOR.test.ts (plus the format-tree path at 5,000 and truncated-input rejection), and in Data.test.ts 16,000-level data from hex and from bytes, nested constructors at 5,000 (kept shallower only because building a Constr per level is slow), and the ParseError surface for an unsupported tag.
  • Full suite: 1688 passed / 75 skipped / 0 failed — main's baseline is 1678 passed, so the delta is exactly the 10 new tests. tsc --noEmit and eslint clean. Changeset added (patch).

Independent builder — not affiliated with the Cardano Foundation, IOG or Intersect. — @Kshot9000 (https://x.com/kshot9000) · ADA: addr1q8hnl6vl5a6k3rw3n5g3jtte696zcl76kfatzv7gpswa9r0dj7fma6klq55y4ffm7tf0em09udnyhuk4ah92pl5x9jpqjae44v

…ursing per level (IntersectMBO#662)

The ledger limits the size of each leaf of Plutus data, not its depth, so
a datum can nest roughly as deep as the transaction size allows (around
16,000 levels). CBOR.fromCBORHex threw RangeError from about 1,719 levels
and Data.fromCBORHex failed from about 1,250, and every provider decodes
inline datums when converting UTxOs, so one deeply nested datum made
getUtxos() fail for a whole address.

- CBOR.ts: the offset decoder (decodeItemAt and its array/map/tag
  helpers) now runs on an explicit stack of open containers instead of
  recursing once per level. Leaf decoding, definite/indefinite handling,
  maps-as-objects and the format tree are byte-for-byte the behaviours of
  the recursive version.
- Data.ts: cborValueToPlutusData converts on an explicit action stack,
  with structural guards instead of the CBOR.is* Schema guards (those
  validate the whole subtree recursively). Validation order and error
  messages are unchanged.
- Data.ts: fromCBORBytes/fromCBORHex decode through those two functions
  directly and wrap failures in ParseError, instead of routing through
  the recursive schema parser; the FromCBORBytes/FromCBORHex/FromCDDL
  schemas keep their existing validation behaviour.

Tests: 10 new depth regression tests (16,000-level definite/indefinite
lists, maps and tags, the format path, Data from hex/bytes, nested
constructors, truncated input). Full suite: 1688 passed / 75 skipped /
0 failed (main baseline 1678 + 10 new). tsc --noEmit and eslint clean.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decoding fails for Plutus data nested deeper than about 1,250 levels

1 participant