feat(pypirsf): PyPI RSF record layout and dependency-blob decoder

[jonyoder]

What

Ports the PyPI record layout and the dependency-blob decode side out of Package Manager, so there is one implementation shared by the standalone resolver and by PPM.

Step 1 of the direction settled with Jonathan: a standalone runnable resolver that resolves from an RSF file it is given, with PPM keeping its own tight in-process integration.

Why here and not in repository-snapshot-format

That library is deliberately generic — records, fields, indexes, and nothing about any ecosystem. A PyPI record schema is Python-packaging knowledge, not format knowledge, so it belongs with the code that understands Python packaging.

The dependency direction already works out: PPM imports this module, and this module imports the generic format library, so PPM → go-pyresolver → repository-snapshot-format stays acyclic.

The neutral type is three fields, because that was the whole coupling

PPM's depsblob referenced storetypes.PyPIVersionMetadata for exactly three fields. So VersionDeps carries RequiresDist, RequiresPython, ProvidesExtra — as raw unparsed strings.

That keeps this package's only dependency a zstd implementation and leaves PEP 508/440 parsing to the caller, which also lets a consumer that just echoes the strings back into an API response skip parsing entirely. PPM's type additionally carries Blocked, BlockingRule, Vulns and MetadataMD5Sum — none of which the codec ever touched.

Two properties carried over deliberately, both now tested

Hardening. Every length-prefixed read is bounded against bytes actually remaining, so a crafted length errors instead of allocating, and MaxDecompressedBytes (64 MiB) caps zstd expansion. This matters more here than in a server: a standalone tool decodes a file the user downloaded, and nothing upstream vouched for it.

The load-bearing field order. ReleaseDate must sit between Version and Summary, with Summary last, because released readers navigate subfields by name using a forward-only skip and therefore silently depend on Summary being last. Violating it leaks bytes into the next element and desynchronizes every shipped reader — which already happened in production. Documented as wire format rather than struct layout.

The golden fixtures are the point

testdata/golden_blob.bin and golden_depsdict.bin are copied verbatim from the producer's own test vectors. That is what makes the conformance test meaningful: PPM's codec_test.go asserts the same expected decode against the same fixture, so both decoders are pinned to one producer-authored vector rather than each merely agreeing with itself.

TestGoldenCoversDictionaryCompressionBothWays additionally asserts the fixture exercises both requirement encodings — dictionary-referenced (flask>=2.0) and inline-name (unknownpkg>=1.0) — so a decoder that broke one could not pass the golden test by luck.

The fixture also covers the empty-case distinction that matters to a resolver: version 3.0.0 decodes to a zero VersionDeps, meaning captured and declares nothing, which is different from a version being absent from the map, meaning nothing was captured. Collapsing those would let a resolver silently drop a subtree.

Three deliberate API differences from PPM's version

• Dict.Decoder() is not exposed. PPM used it only so a test could assert the decoder was released. TestDecodeAfterCloseErrors covers the same ground by decoding through a closed Dict, without leaking zstd types into a public API. PPM's migration will need to adapt that one test.
• Names() is exported, since the dep-name table doubles as a bounded set usable for integer-id comparison instead of string comparison.
• decompress guards an empty field rather than indexing field[0] on faith. Unreachable today because DecodePackage checks first, but a future caller gets an error rather than a panic.

Testing

• 86.5% statement coverage on pypirsf/; -race -count=1 clean; gofmt -l . silent; golangci-lint run ./... at CI's pinned v2.11.2 → 0 issues from the module root.
• New dependency: github.com/klauspost/compress (zstd), attributed in NOTICE along with the port itself.

Not in this PR

• The RSF-file-backed MetadataIndex built on this — next.
• PPM's migration onto this package, which is its own reviewed PR. Until it lands, two copies of the decoder exist; the shared golden fixture is what keeps them honest in the meantime, and closing that window is why the migration shouldn't be deferred indefinitely. One thing to watch there: src/rsf/python_listing_index.go calls ParseDepsdictField during index build, and Rev 15's design depends on the listing hot path never materializing the trailing deps fields — so the migration needs to confirm it doesn't change when that dict is parsed.

🤖 Generated with Claude Code