Skip to content

feat(data): aprender-contrastive-data — deterministic, leakage-safe pair construction - #2702

Open
guyernest wants to merge 2 commits into
paiml:mainfrom
guyernest:contrastive-data
Open

feat(data): aprender-contrastive-data — deterministic, leakage-safe pair construction#2702
guyernest wants to merge 2 commits into
paiml:mainfrom
guyernest:contrastive-data

Conversation

@guyernest

Copy link
Copy Markdown
Contributor

aprender-contrastive-data — deterministic, leakage-safe pair construction

Contrastive/Siamese data construction as a general capability: class buckets,
balanced few-shot selection, bounded positive/negative pair sampling, typed split
roles, dataset fingerprints, and the cross-split leakage checks that make those
trustworthy.

Independent of #2618. This does not depend on aprender-core at all — its
whole dependency closure is 28 packages. Review and merge in either order.

Why a separate crate

Not folded into aprender-data: that crate carries arrow/parquet/zstd/s3/
streaming/federated weight which must not flow into a training crate or a
serverless consumer. SetFit is this crate's first consumer, not its owner.

The bytes boundary

The public API is bytes-in / bytes-out and typed values. No filesystem access, no
sockets, no path-shaped parameters — not even in tests. apr-cli owns every
filesystem adapter.

That is not stylistic. The destination for these artifacts is object storage,
where a manifest is an S3 object rather than a file; an API that speaks &Path
makes such a consumer a rewrite instead of a wrapper.

It is enforced, not asserted: make contrastive-data-boundary checks the
resolved dependency closure against a positive allowlist and bans
std::fs/std::net/std::path/Path/PathBuf under src/. Wired into tier3,
because a target outside the tiers is a target that stops being run.

Determinism

Every random decision is a pure function of its draw ordinal, from the
counter-based Philox generator in aprender-rand. Worker-count independence is
structural rather than tested-for: draw i cannot depend on how many draws
preceded it, because nothing precedes it.

Five typestate misuses are proven unrepresentable with trybuild compile-fail
cases and committed .stderr snapshots, rather than asserted in prose.

One fix to the gate itself

allowed-deps.txt listed provable-contracts-macros — the workspace dependency
key. The crate is aprender-contracts-macros (the key carries
package = "..."), and the crate name is what cargo tree prints, so the entry
could never match. The gate was RED from the commit that scaffolded it.

Corrected to the real package name. This is a name fix, not a widening — the
reviewed decision is unchanged, and the gate proved non-vacuous by catching it.

Verification

rc captured directly, never through a pipe:

cargo check  -p aprender-contrastive-data --all-targets     rc=0
cargo test   -p aprender-contrastive-data                   249 passed / 0 failed
cargo clippy -p aprender-contrastive-data --all-targets
             -- -D warnings                                 rc=0
make contrastive-data-boundary                              rc=0
cargo fmt --all -- --check                                  rc=0
pv validate contracts/contrastive-pair-protocol-v1.yaml     0 errors, 0 warnings

Note for whoever merges second

#2618 and this PR both add aprender-rand to [workspace.dependencies]. That
one line will conflict textually; everything else is disjoint. Whichever lands
second needs a one-line rebase, and I'm happy to do it.

🤖 Generated with Claude Code

guyernest and others added 2 commits August 25, 2026 06:50
Schema before implementation, as the repo does for phase contracts. Defines the
deterministic contrastive data protocol: class buckets, balanced few-shot
selection, bounded pair sampling, typed split roles, dataset fingerprints and
the cross-split leakage checks.

`contracts/aprender/binding.yaml` gains the 24 entries that bind its equations
to the modules landing in the next commit.

  pv validate contracts/contrastive-pair-protocol-v1.yaml   0 errors, 0 warnings

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…air construction

Contrastive/Siamese DATA CONSTRUCTION as a general capability: class buckets,
balanced few-shot selection, bounded positive/negative pair sampling, typed
split roles, dataset fingerprints, and the cross-split leakage checks that make
those trustworthy. SetFit is its first consumer, not its owner.

Deliberately a separate crate rather than part of aprender-data: that crate
carries arrow/parquet/zstd/s3/streaming/federated weight which must not flow
into a training crate or a serverless consumer.

BYTES BOUNDARY. The public API is bytes-in / bytes-out and typed values. No
filesystem access, no sockets, no path-shaped parameters — not even in tests;
apr-cli owns every filesystem adapter. The destination for these artifacts is
object storage, where a manifest is an S3 object, not a file: an API that speaks
`&Path` makes such a consumer a rewrite instead of a wrapper. The boundary is
ENFORCED, not asserted — `make contrastive-data-boundary` checks the resolved
dependency closure against a positive allowlist and bans std::fs/net/path
symbols under src/. Wired into tier3, because a target outside the tiers is a
target that stops being run.

DETERMINISM. Every random decision is a pure function of its draw ordinal, from
the counter-based Philox generator in aprender-rand. Worker-count independence
is structural rather than tested-for: draw i cannot depend on how many draws
preceded it, because nothing precedes it.

Five typestate misuses are proven unrepresentable with trybuild compile-fail
cases and committed .stderr snapshots, rather than asserted in prose.

ONE FIX TO THE GATE ITSELF. allowed-deps.txt listed `provable-contracts-macros`
— the workspace dependency KEY. The crate is `aprender-contracts-macros` (the
key carries `package = "..."`), and the crate name is what `cargo tree` prints,
so the entry could never match and the gate was RED from the commit that
scaffolded it. Corrected to the real package name. This is a name fix, not a
widening: the reviewed decision is unchanged, and the gate proved non-vacuous by
catching it.

Verified (rc captured directly, never through a pipe):
  cargo check  -p aprender-contrastive-data --all-targets    rc=0
  cargo test   -p aprender-contrastive-data                  249 passed, 0 failed
  cargo clippy -p aprender-contrastive-data --all-targets
               -- -D warnings                                rc=0
  make contrastive-data-boundary                             rc=0 (deps subset + symbol ban)
  cargo fmt --all -- --check                                 rc=0
  pv validate contracts/contrastive-pair-protocol-v1.yaml    0 errors, 0 warnings

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor

I will look tomorrow and merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants