Skip to content

Commit 3072c0e

Browse files
authored
feat: x-algokit-byte-length support and block model restructuring for cross-SDK alignment (#242)
* feat: add x-algokit-byte-length vendor extension support for fixed-length byte validation - Add byte_length and list_inner_byte_length fields to TypeInfo in builder.py - Update _build_metadata() to generate fixed-length byte serde helpers - Add encode_fixed_bytes_base64/decode_fixed_bytes_base64 helpers for 32/64 byte validation - Add encode_fixed_bytes_sequence/decode_fixed_bytes_sequence for array validation - Regenerate API clients with fixed-length byte fields (group, lease, signature, etc.) - Update OAS_BRANCH to fix/byte-len-validation for spec generation * refactor(algod-client): restructure block models to align with JS SDK Align Python block model structure with the JavaScript SDK and OpenAPI spec for cross-SDK consistency. Structural changes: - Rename GetBlock -> BlockResponse (matches OAS spec + JS/TS SDKs) - Add TxnCommitments nested type for transaction roots (txn, txn256) - Add RewardState nested type for reward fields (fees, rwd, earn, etc.) - Add UpgradeState nested type for protocol upgrade state - Add UpgradeVote nested type for upgrade vote parameters - Refactor BlockHeader to use flattened nested types - Change BlockEvalDelta.bytes -> bytes_value (avoid Python keyword) - Fix inner_txns type: SignedTxnInBlock -> SignedTxnWithAD - Make previous_block_hash and genesis_hash non-optional with defaults - Fix get_block() to return BlockResponse (typed) instead of Block - Export typed BlockResponse from _block.py instead of untyped version BREAKING CHANGE: - GetBlock removed, use BlockResponse instead - Field access patterns changed in BlockHeader: - header.fee_sink -> header.reward_state.fee_sink - header.current_protocol -> header.upgrade_state.current_protocol - header.upgrade_propose -> header.upgrade_vote.upgrade_propose - header.transactions_root -> header.txn_commitments.transactions_root - Block response structure: result.header -> result.block.header * refactor: excludes block models from generated code Excludes block models from the generated client code to allow separate handling. This change introduces a mechanism to conditionally include block models during code generation based on a client configuration. It also updates the AVM debugger source map generation by incorporating hashes in the standard AVM debugger format. Skips tests that depend on msgpack handling until mock server is fixed. * fix(algod-client): restore block model types to public exports Ensures all block-related types (ApplyData, BlockEvalDelta, BlockStateDelta, SignedTxnWithAD, TxnCommitments, RewardState, UpgradeState, UpgradeVote, etc.) remain publicly exported from algokit_algod_client.models for type hints, isinstance checks, and IDE autocomplete. These types are returned by the public API and must be accessible to users. Aligns with TypeScript SDK behavior.
1 parent 22bde9a commit 3072c0e

34 files changed

+635
-154
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -184,3 +184,5 @@ api/specs/
184184
.polytest*/
185185

186186
references/
187+
188+
AGENTS.md

MIGRATION-NOTES.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,73 @@ A collection of notes to consolidate todos during decoupling efforts (similar do
4040

4141
- BoxReference used to be a subclass of algosdk BoxReference, now there is a direct equivalent imported from transact package. Do we want to keep aliasing the BoxReference to transact version or do we want to deprecate and advice consumers to import directly from transact?
4242

43+
## Block Model Restructuring (TS Alignment)
44+
45+
### Breaking Changes
46+
47+
- `GetBlock` removed → use `BlockResponse`
48+
- `BlockHeader` fields reorganized into nested types:
49+
- `header.fee_sink``header.reward_state.fee_sink`
50+
- `header.rewards_pool``header.reward_state.rewards_pool`
51+
- `header.rewards_level``header.reward_state.rewards_level`
52+
- `header.rewards_rate``header.reward_state.rewards_rate`
53+
- `header.rewards_residue``header.reward_state.rewards_residue`
54+
- `header.rewards_recalculation_round``header.reward_state.rewards_recalculation_round`
55+
- `header.current_protocol``header.upgrade_state.current_protocol`
56+
- `header.next_protocol``header.upgrade_state.next_protocol`
57+
- `header.next_protocol_approvals``header.upgrade_state.next_protocol_approvals`
58+
- `header.next_protocol_vote_before``header.upgrade_state.next_protocol_vote_before`
59+
- `header.next_protocol_switch_on``header.upgrade_state.next_protocol_switch_on`
60+
- `header.upgrade_propose``header.upgrade_vote.upgrade_propose`
61+
- `header.upgrade_delay``header.upgrade_vote.upgrade_delay`
62+
- `header.upgrade_approve``header.upgrade_vote.upgrade_approve`
63+
- `header.transactions_root``header.txn_commitments.transactions_root`
64+
- `header.transactions_root_sha256``header.txn_commitments.transactions_root_sha256`
65+
- `BlockEvalDelta.bytes``BlockEvalDelta.bytes_value` (avoid Python keyword)
66+
- `BlockAppEvalDelta.inner_txns` type changed: `SignedTxnInBlock``SignedTxnWithAD`
67+
- `previous_block_hash` and `genesis_hash` now non-optional with `bytes(32)` defaults
68+
69+
### Migration
70+
71+
```python
72+
# Before
73+
from algokit_algod_client.models import GetBlock
74+
response: GetBlock = algod_client.get_block(...)
75+
fee_sink = response.block.header.fee_sink
76+
protocol = response.block.header.current_protocol
77+
proposal = response.block.header.upgrade_propose
78+
79+
# After
80+
from algokit_algod_client.models import BlockResponse
81+
response: BlockResponse = algod_client.get_block(...)
82+
fee_sink = response.block.header.reward_state.fee_sink
83+
protocol = response.block.header.upgrade_state.current_protocol
84+
proposal = response.block.header.upgrade_vote.upgrade_propose
85+
```
86+
87+
## Fixed-Length Byte Validation
88+
89+
### Breaking Changes
90+
91+
- Runtime validation for fixed-length byte fields (32/64 bytes)
92+
- `ValueError` raised if byte length doesn't match expected length
93+
- Affects: `group`, `lease`, signatures, hashes, keys, commitment roots
94+
95+
### Migration
96+
97+
```python
98+
# Validation now enforced at encode/decode
99+
# 32-byte fields: group, lease, transaction hashes, block hashes, keys
100+
# 64-byte fields: signatures, SHA-512 hashes
101+
102+
# Before: silently accepted wrong lengths
103+
txn.group = bytes(10) # No error
104+
105+
# After: raises ValueError
106+
txn.group = bytes(10) # ValueError: Expected 32 bytes, got 10
107+
txn.group = bytes(32) # OK
108+
```
109+
43110
## Account and Signer Type Renames (TS Alignment)
44111

45112
### Breaking Changes

api/oas-generator/src/oas_generator/builder.py

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@ class TypeInfo:
3333
is_locals_reference: bool = False
3434
is_holding_reference: bool = False
3535
needs_datetime: bool = False
36+
byte_length: int | None = None # Fixed byte length from x-algokit-byte-length
37+
list_inner_byte_length: int | None = None # Fixed byte length for list items
3638
imports: set[str] = field(default_factory=set)
3739

3840

@@ -170,7 +172,10 @@ def resolve(self, schema: ctx.RawSchema, *, hint: str = "Inline") -> TypeInfo:
170172
if schema_type == "string":
171173
fmt = schema.get("format")
172174
if fmt in {"byte", "binary"} or schema.get("x-algokit-bytes-base64"):
173-
info = TypeInfo(annotation="bytes", is_bytes=True)
175+
# Extract fixed byte length if present
176+
byte_length_val = schema.get("x-algokit-byte-length")
177+
byte_length = int(byte_length_val) if byte_length_val is not None else None
178+
info = TypeInfo(annotation="bytes", is_bytes=True, byte_length=byte_length)
174179
elif fmt == "date-time":
175180
info = TypeInfo(
176181
annotation="datetime",
@@ -219,6 +224,7 @@ def _resolve_array(self, schema: ctx.RawSchema, *, hint: str) -> TypeInfo:
219224
list_inner_model=inner.model,
220225
list_inner_enum=inner.enum,
221226
list_inner_is_bytes=inner.is_bytes,
227+
list_inner_byte_length=inner.byte_length,
222228
is_signed_transaction=inner.is_signed_transaction,
223229
is_box_reference=inner.is_box_reference,
224230
is_locals_reference=inner.is_locals_reference,
@@ -462,6 +468,10 @@ def _build_model(self, entry: SchemaEntry) -> ctx.ModelDescriptor: # noqa: C901
462468
imports.add("from ._serde_helpers import decode_bytes_base64, encode_bytes_base64")
463469
if "encode_bytes_sequence" in field.metadata or "decode_bytes_sequence" in field.metadata:
464470
imports.add("from ._serde_helpers import decode_bytes_sequence, encode_bytes_sequence")
471+
if "encode_fixed_bytes_base64" in field.metadata or "decode_fixed_bytes_base64" in field.metadata:
472+
imports.add("from ._serde_helpers import decode_fixed_bytes_base64, encode_fixed_bytes_base64")
473+
if "encode_fixed_bytes_sequence" in field.metadata or "decode_fixed_bytes_sequence" in field.metadata:
474+
imports.add("from ._serde_helpers import decode_fixed_bytes_sequence, encode_fixed_bytes_sequence")
465475
if "nested(" in field.metadata:
466476
uses_nested = True
467477
if "flatten(" in field.metadata:
@@ -585,6 +595,15 @@ def _build_metadata(self, wire_name: str, type_info: TypeInfo, *, required: bool
585595
" )"
586596
)
587597
if type_info.is_list and type_info.list_inner_is_bytes:
598+
# Handle fixed-length bytes in sequences
599+
if type_info.list_inner_byte_length is not None:
600+
return (
601+
"wire(\n"
602+
f' "{alias}",\n'
603+
f" encode=lambda v: encode_fixed_bytes_sequence(v, {type_info.list_inner_byte_length}),\n"
604+
f" decode=lambda raw: decode_fixed_bytes_sequence(raw, {type_info.list_inner_byte_length}),\n"
605+
" )"
606+
)
588607
return (
589608
"wire(\n"
590609
f' "{alias}",\n'
@@ -593,6 +612,15 @@ def _build_metadata(self, wire_name: str, type_info: TypeInfo, *, required: bool
593612
" )"
594613
)
595614
if type_info.is_bytes:
615+
# Handle fixed-length bytes
616+
if type_info.byte_length is not None:
617+
return (
618+
"wire(\n"
619+
f' "{alias}",\n'
620+
f" encode=lambda v: encode_fixed_bytes_base64(v, {type_info.byte_length}),\n"
621+
f" decode=lambda raw: decode_fixed_bytes_base64(raw, {type_info.byte_length}),\n"
622+
" )"
623+
)
596624
return (
597625
"wire(\n"
598626
f' "{alias}",\n'
@@ -908,11 +936,11 @@ def _build_response(self, responses: dict[str, Any], operation_id: str) -> ctx.R
908936
self.uses_block_models = True
909937
media_types = media_types or ["application/json"]
910938
return ctx.ResponseDescriptor(
911-
type_hint="models.GetBlock",
939+
type_hint="models.BlockResponse",
912940
media_types=media_types,
913941
description=payload.get("description"),
914942
is_binary=False,
915-
model="GetBlock",
943+
model="BlockResponse",
916944
)
917945
if schema is None:
918946
return None

api/oas-generator/src/oas_generator/renderer/engine.py

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,14 @@ class TemplateRenderer:
2626
"BlockStateProofTracking",
2727
"ParticipationUpdates",
2828
"SignedTxnInBlock",
29+
"SignedTxnWithAD",
30+
"TxnCommitments",
31+
"RewardState",
32+
"UpgradeState",
33+
"UpgradeVote",
2934
"BlockHeader",
3035
"Block",
31-
"GetBlock",
36+
"BlockResponse",
3237
]
3338
LEDGER_STATE_DELTA_EXPORTS: ClassVar[list[str]] = [
3439
"LedgerTealValue",
@@ -87,7 +92,9 @@ def render(self, client: ctx.ClientDescriptor, config: GeneratorConfig) -> dict[
8792
files[models_dir / "__init__.py"] = self._render_template("models/__init__.py.j2", context)
8893
files[models_dir / "_serde_helpers.py"] = self._render_template("models/_serde_helpers.py.j2", context)
8994
ledger_model_names = set(LEDGER_STATE_DELTA_MODEL_NAMES)
90-
models = [model for model in context["client"].models if model.name not in ledger_model_names]
95+
block_model_names = set(self.BLOCK_MODEL_EXPORTS) if client.include_block_models else set()
96+
excluded_models = ledger_model_names | block_model_names
97+
models = [model for model in context["client"].models if model.name not in excluded_models]
9198
for model in models:
9299
model_context = {**context, "model": model}
93100
files[models_dir / f"{model.module_name}.py"] = self._render_template("models/model.py.j2", model_context)
@@ -132,10 +139,12 @@ def _build_context(self, client: ctx.ClientDescriptor, config: GeneratorConfig)
132139
model_exports.append("SuggestedParams")
133140
metadata_usage = self._collect_metadata_usage(client)
134141
ledger_model_names = set(LEDGER_STATE_DELTA_MODEL_NAMES)
142+
block_model_names = set(self.BLOCK_MODEL_EXPORTS) if client.include_block_models else set()
143+
excluded_models = ledger_model_names | block_model_names
135144
model_modules = [
136145
{"module": model.module_name, "name": model.name}
137146
for model in client.models
138-
if model.name not in ledger_model_names
147+
if model.name not in excluded_models
139148
]
140149
enum_modules = [{"module": enum.module_name, "name": enum.name} for enum in client.enums]
141150
alias_modules = [{"module": alias.module_name, "name": alias.name} for alias in client.aliases]

api/oas-generator/src/oas_generator/renderer/templates/models/_serde_helpers.py.j2

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,22 @@ def decode_bytes_base64(raw: object) -> bytes:
3737
raise TypeError(f"Unsupported value for bytes field: {type(raw)!r}")
3838

3939

40+
def encode_fixed_bytes_base64(value: BytesLike, expected_length: int) -> str:
41+
"""Encode fixed-length bytes to base64, validating the length."""
42+
coerced = _coerce_bytes(value)
43+
if len(coerced) != expected_length:
44+
raise ValueError(f"Expected {expected_length} bytes, got {len(coerced)}")
45+
return base64.b64encode(coerced).decode("ascii")
46+
47+
48+
def decode_fixed_bytes_base64(raw: object, expected_length: int) -> bytes:
49+
"""Decode base64 to fixed-length bytes, validating the length."""
50+
decoded = decode_bytes_base64(raw)
51+
if len(decoded) != expected_length:
52+
raise ValueError(f"Expected {expected_length} bytes, got {len(decoded)}")
53+
return decoded
54+
55+
4056
def decode_bytes_map_key(raw: object) -> bytes:
4157
if isinstance(raw, bytes | bytearray | memoryview):
4258
return bytes(raw)
@@ -77,6 +93,36 @@ def decode_bytes_sequence(raw: object) -> list[bytes | None] | None:
7793
return decoded or None
7894

7995

96+
def encode_fixed_bytes_sequence(
97+
values: Iterable[BytesLike | None] | None, expected_length: int
98+
) -> list[str | None] | None:
99+
"""Encode a sequence of fixed-length bytes to base64, validating each element's length."""
100+
if values is None:
101+
return None
102+
encoded: list[str | None] = []
103+
for value in values:
104+
if value is None:
105+
encoded.append(None)
106+
continue
107+
if not isinstance(value, bytes | bytearray | memoryview):
108+
raise TypeError(f"Unsupported value for bytes field sequence: {type(value)!r}")
109+
encoded.append(encode_fixed_bytes_base64(value, expected_length))
110+
return encoded or None
111+
112+
113+
def decode_fixed_bytes_sequence(raw: object, expected_length: int) -> list[bytes | None] | None:
114+
"""Decode a sequence of base64 strings to fixed-length bytes, validating each element's length."""
115+
if not isinstance(raw, list):
116+
return None
117+
decoded: list[bytes | None] = []
118+
for item in raw:
119+
if item is None:
120+
decoded.append(None)
121+
continue
122+
decoded.append(decode_fixed_bytes_base64(item, expected_length))
123+
return decoded or None
124+
125+
80126
def encode_model_sequence(values: Iterable[object] | None) -> list[dict[str, object]] | None:
81127
if values is None:
82128
return None

0 commit comments

Comments
 (0)