cardano-rpc: Populate transaction bodies in FetchBlock responses#1247
cardano-rpc: Populate transaction bodies in FetchBlock responses#1247carbolymer wants to merge 9 commits into
Conversation
carbolymer
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
f9cb860 to
812cb1c
Compare
73e1a16 to
534dce1
Compare
812cb1c to
b0dbce2
Compare
2daffe7 to
ecc1f35
Compare
b0dbce2 to
0d12995
Compare
a5413ed to
9396d3b
Compare
0d12995 to
2fec518
Compare
There was a problem hiding this comment.
Pull request overview
This PR extends the cardano-rpc UTxO RPC implementation so FetchBlock returns fully populated transaction bodies (including Byron), backed by new ledger-to-protobuf conversion modules and expanded property/golden test coverage. It also includes supporting cardano-api exports/helpers needed to correctly wire redeemers and access Byron raw blocks.
Changes:
- Populate
FetchBlockresponses withcardano.body.txfor Shelley-based eras and Byron, using new conversion modules (Type.Tx,Type.Byron,Type.Certificate,Type.Governance). - Refactor and extend UTxO RPC type mappings (scripts, protocol parameters, tx outputs) and add extensive property/golden tests for mapping correctness across eras.
- Add
cardano-apiexports for per-eratoScriptIndex*helpers and re-exportbyronBlockRawfor Byron block access.
Reviewed changes
Copilot reviewed 26 out of 28 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/Type.hs | Adds a property test for metadatum integer clamping behavior. |
| cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/FetchBlockTx.hs | New property tests projecting txToUtxoRpcTx results against ledger facts across eras. |
| cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/ByronTx.hs | New property/golden tests for Byron tx/block conversion. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/TxOutput.hs | Extracts reusable multiasset conversion helper. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Tx.hs | New Shelley-based ledger-tx → UTxO RPC Tx conversion including redeemer wiring. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Script.hs | Factors out scriptToUtxoRpcScript for reuse. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/ProtocolParameters.hs | Reworks protocol parameter mapping using HKD setters; adds protocol version conversion helper. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Governance.hs | New mapping for Conway governance proposals/actions to proto. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Certificate.hs | New mapping for Shelley/Conway/Dijkstra certificate variants to proto. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Byron.hs | New conversion for Byron-era txs and block tx extraction. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type.hs | Re-exports newly introduced conversion entry points. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Sync.hs | Wires FetchBlock to include populated transaction bodies. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/Orphans.hs | Adds Inject instance for protocol version mapping. |
| cardano-rpc/README.md | Updates feature matrix to mark FetchBlock as supported. |
| cardano-rpc/cardano-rpc.cabal | Registers new modules and dependencies; wires new test modules. |
| cardano-api/src/Cardano/Api/Tx/Internal/Body.hs | Exports toScriptIndexAlonzo/Conway/Dijkstra and generalises their types. |
| cardano-api/src/Cardano/Api/Tx.hs | Re-exports the per-era toScriptIndex* helpers from the public API. |
| cardano-api/src/Cardano/Api/Experimental/Era.hs | Adds ConwayEraPParams to EraCommonConstraints (changelog marked breaking). |
| cardano-api/src/Cardano/Api/Consensus/Internal/Reexport.hs | Re-exports consensus byronBlockRaw. |
| cardano-api/src/Cardano/Api/Consensus.hs | Exposes byronBlockRaw from the public consensus API. |
| cardano-api/gen/Test/Gen/Cardano/Api/Typed.hs | Adjusts genTx to sign a single body; adds genTxWitnesses. |
| .changes/20260715_cardano_api_gen_tx_single_body.yml | Changelog fragment for generator/testlib adjustment. |
| .changes/20260715_cardano_api_export_to_script_index_variants.yml | Changelog fragment for new toScriptIndex* exports. |
| .changes/20260715_cardano_api_conway_pparams_common_constraints.yml | Changelog fragment marking added constraint as breaking. |
| .changes/20260708_cardano_rpc_fetchblock_tx_bodies.yml | Changelog fragment for FetchBlock tx body population feature. |
| .changes/20260708_cardano_api_byron_block_raw_reexport.yml | Changelog fragment for byronBlockRaw re-export. |
| -- Byron transactions are not representable as cardano-api's 'Tx era', | ||
| -- so they are converted straight from the Byron ledger types | ||
| txs = case block of | ||
| ByronBlock consensusBlock -> | ||
| byronBlockTxs (byronBlockRaw consensusBlock) |
2fec518 to
1cc20c7
Compare
09b776c to
8966469
Compare
palas
left a comment
palas
left a comment
There was a problem hiding this comment.
Nice work! It all looks solid. I still want to look in more depth at the tests tomorrow, cause I didn't have time today. And I don't know about the exporting of per-era toScriptIndex functions, is it necessary?
| , toScriptIndexAlonzo | ||
| , toScriptIndexConway | ||
| , toScriptIndexDijkstra |
There was a problem hiding this comment.
I don't understand why these are needed. Why not use toScriptIndex?
There was a problem hiding this comment.
To avoid using of eons. I can either:
- Use
toScriptIndex - Use eons, which are soft-deprecated at this point.
I'll have another look if I can avoid using those.
There was a problem hiding this comment.
I've removed those reexports, added experimental toScriptPurposeIndex. But I still have to use eons. 😞
ddec16c to
792f6ae
Compare
palas
left a comment
palas
left a comment
There was a problem hiding this comment.
Great job. It all looks very sound to me. I left a couple of really minor nits
| protoProposal ^. U5c.anchor . U5c.url === "https://example.com/anchor" | ||
| protoProposal ^. U5c.anchor . U5c.contentHash | ||
| === L.hashToBytes (L.extractHash (L.hashAnnotated (L.AnchorData "anchor data"))) |
There was a problem hiding this comment.
"https://example.com/anchor" and "anchor data" are duplicated constants which are meant to be the same (they are also defined in 288 and 289). At least the string literals for them should come from a single source
| || not (null (protoTx ^. U5c.witnesses . U5c.vkeywitness)) | ||
| firstTx ^. U5c.maybe'fee === Nothing | ||
|
|
||
| H.note_ "The boundary block fixture decodes and its header hash matches mainnet" |
There was a problem hiding this comment.
I would argue before and after this, they should be separate tests/functions, or at least sub-functions, cause they are completely independent as far as I can see
| expectedCertificateArm = \case | ||
| L.ConwayTxCertDeleg delegCert -> | ||
| case delegCert of | ||
| L.ConwayRegCert _ L.SNothing -> |
There was a problem hiding this comment.
It seems the ConwayRegCert _ L.SNothing and ConwayUnRegCert _ L.SNothing branches are not exercised by genTx. Not sure if it is worth tuning genTx so that they are
Classifies a ledger redeemer pointer into its `PlutusScriptPurpose` category and index given an `AlonzoEraOnwards` witness - the read direction, the inverse of `toPlutusScriptPurpose`. Also adds the `GuardingScript` purpose for Dijkstra guard scripts.
792f6ae to
1af9cd2
Compare
Jimbo4350
left a comment
Jimbo4350
left a comment
There was a problem hiding this comment.
There is some use of the legacy api here that should be addressed.
| -- plutus purposes and pins the era, so no constraints or type applications | ||
| -- are needed at use sites. | ||
| toPlutusScriptPurposeIndex | ||
| :: AlonzoEraOnwards era |
There was a problem hiding this comment.
This can be parameterized on ShelleyBasedEra era and return Maybe (PlutusScriptPurpose, Word32). Where it is called has access to a ShelleyBasedEra era value.
|
|
||
| -- | Convert a script to the UTxO RPC 'UtxoRpc.Script' message, dispatching on | ||
| -- the script language to select the corresponding oneof field. | ||
| scriptToUtxoRpcScript :: Script lang -> Proto UtxoRpc.Script |
There was a problem hiding this comment.
It is possible to use AnyScript era instead here? Script lang is the old api.
| txAuxDataScripts | ||
| :: ShelleyBasedEra era | ||
| -> L.TxAuxData (ShelleyLedgerEra era) | ||
| -> [ScriptInEra era] |
There was a problem hiding this comment.
Is it possible to use AnyScript era here? ScriptInEra era is the old api.
Context
Part of the node kernel access work (ADR-019):
FetchBlockpreviously returned raw block bytes, the block header and the timestamp, but leftbody.txempty.This PR populates the transaction bodies. All 14 fields of the UTxO RPC
Txmessage are mapped: inputs, outputs, reference inputs, certificates, withdrawals, mint, witnesses, collateral, fee, validity, successful, auxiliary data, hash and governance proposals.successfulreflects the phase-2IsValidflag.Certificatemodule maps all 19 certificate variants (Dijkstra-safe via concrete-era dispatch); a newGovernancemodule maps all 7 governance actions including partial parameter updates.Tx erahas no Byron constructor. Byron fees are implicit in the UTxO and stay unset; epoch boundary blocks yield no transactions. This makesFetchBlockserve transaction data for the full mainnet history.The first four commits are small cardano-api prerequisites, each with its own changelog fragment: re-export
byronBlockRawfromCardano.Api.Consensus, addConwayEraPParamstoEraCommonConstraints, export the per-eratoScriptIndexvariants, and makegenTxsign the transaction body it returns.Known limitations, deliberate and documented in code:
TxInput.as_outputis unset (resolving it requires UTxO lookups).feeis unset (implicit in Byron; requires UTxO lookups).WitnessSet;native_bytesis the fidelity escape hatch.bytes-field encoding divergences (TxOutput.addressas text, inlineDatum.hashcarrying CBOR) are tracked separately in gRPC:bytesfields encoded as text or wrong payload (TxOutput.address,Datum.hash) #1246 and not changed here.How to trust this PR
This PR is integrated and tested end-to-end in IntersectMBO/cardano-node#6579.
The conversion is covered at three levels:
cardano-rpc-test, 77 tests): per-era field projections of generated transactions against the ledger values across all seven Shelley-based eras, certificate arm routing, injected-redeemer wiring (spending/rewarding/certifying at known indices), a wire-level protobuf encode/decode roundtrip forcing total evaluation across all eras, Byron golden tests decoding real block CBOR, and a forged-annotation guard pinning Byron transaction ids to the on-chain bytes rather than a re-serialisation.FetchBlockand asserts the mapped fields (hash, fee, inputs, outputs, validity, mint, witness set contents) against the submitted values, plus theNOT_FOUNDandINVALID_ARGUMENTerror paths.Review commit by commit; every commit builds and passes tests on its own. The conversion lives in
Cardano.Rpc.Server.Internal.UtxoRpc.Type.{Tx,Certificate,Governance,Byron}and the handler wiring inCardano.Rpc.Server.Internal.UtxoRpc.Sync.Run the tests:
Checklist
.changes/