Skip to content

feat(bundle): emit bundle v2 next to v1 (RFC-070 F1) - #5

Merged
0xmanhnv merged 1 commit into
mainfrom
feat/bundle-v2-emit
Oct 10, 2026
Merged

0xmanhnv merged 1 commit into
mainfrom
feat/bundle-v2-emit

Conversation

@0xmanhnv

Copy link
Copy Markdown
Member

Emits bundle format v2 (signed pointer, signed manifests, content-addressed gzip JSONL chunks keyed by record-id ranges) alongside the unchanged v1 output for one release.

  • Writer: sdk-go pkg/transfer/bundle, pinned to the pseudo-version of sdk-go main 17ba39a (PR 233). No tagged sdk-go release has it yet: replace the pin with a tag when one exists.
  • Streams: programs (id = program id) and changes (id = program#n); default chunking (about 2,000 records), deterministic by record id.
  • Signing: same key and key set; build writes unsigned v2 manifests, the sign step signs them and writes the signed pointer (it pins the signed manifests' digests). A local-only bundle is never signed; it gets an unsigned snapshot.v2.manifest.json with local_only in meta and no pointer.
  • verify also checks v2 (pointer, manifest pins, every chunk).
  • Workflow unchanged: release uploads out/*, so the v2 files and chunks are included; the previous-release download only needs v1.
  • Tests: round trip with the SDK consumer, unchanged input gives identical chunks (also across sequences), one changed program changes one chunk, v1 files and verification unchanged, local-only stays unsigned.

Real local build (diodb, bounty-targets, pd-bounty): 4169 programs -> programs stream 2 chunks (2521 records / 636 KB, 1648 / 98 KB), changes stream 1 chunk (4169 / 27 KB); v1 programs file 733 KB.

RFC-070 compatibility rule: one release cycle with both formats. v2 uses
the SDK writer (sdk-go pseudo-version of main, to be replaced by a tag),
the same signing keys and key set, chunks by record id (deterministic: same
programs give the same chunks, one changed program changes one chunk).
Signing writes the signed manifests and pointer; verify checks v2 too.
Local-only builds write an unsigned v2 snapshot manifest.
@0xmanhnv
0xmanhnv merged commit 31d6b3a into main Oct 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant