feat(data): aprender-contrastive-data — deterministic, leakage-safe pair construction - #2702
Open
guyernest wants to merge 2 commits into
Open
feat(data): aprender-contrastive-data — deterministic, leakage-safe pair construction#2702guyernest wants to merge 2 commits into
guyernest wants to merge 2 commits into
Conversation
Schema before implementation, as the repo does for phase contracts. Defines the deterministic contrastive data protocol: class buckets, balanced few-shot selection, bounded pair sampling, typed split roles, dataset fingerprints and the cross-split leakage checks. `contracts/aprender/binding.yaml` gains the 24 entries that bind its equations to the modules landing in the next commit. pv validate contracts/contrastive-pair-protocol-v1.yaml 0 errors, 0 warnings Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…air construction
Contrastive/Siamese DATA CONSTRUCTION as a general capability: class buckets,
balanced few-shot selection, bounded positive/negative pair sampling, typed
split roles, dataset fingerprints, and the cross-split leakage checks that make
those trustworthy. SetFit is its first consumer, not its owner.
Deliberately a separate crate rather than part of aprender-data: that crate
carries arrow/parquet/zstd/s3/streaming/federated weight which must not flow
into a training crate or a serverless consumer.
BYTES BOUNDARY. The public API is bytes-in / bytes-out and typed values. No
filesystem access, no sockets, no path-shaped parameters — not even in tests;
apr-cli owns every filesystem adapter. The destination for these artifacts is
object storage, where a manifest is an S3 object, not a file: an API that speaks
`&Path` makes such a consumer a rewrite instead of a wrapper. The boundary is
ENFORCED, not asserted — `make contrastive-data-boundary` checks the resolved
dependency closure against a positive allowlist and bans std::fs/net/path
symbols under src/. Wired into tier3, because a target outside the tiers is a
target that stops being run.
DETERMINISM. Every random decision is a pure function of its draw ordinal, from
the counter-based Philox generator in aprender-rand. Worker-count independence
is structural rather than tested-for: draw i cannot depend on how many draws
preceded it, because nothing precedes it.
Five typestate misuses are proven unrepresentable with trybuild compile-fail
cases and committed .stderr snapshots, rather than asserted in prose.
ONE FIX TO THE GATE ITSELF. allowed-deps.txt listed `provable-contracts-macros`
— the workspace dependency KEY. The crate is `aprender-contracts-macros` (the
key carries `package = "..."`), and the crate name is what `cargo tree` prints,
so the entry could never match and the gate was RED from the commit that
scaffolded it. Corrected to the real package name. This is a name fix, not a
widening: the reviewed decision is unchanged, and the gate proved non-vacuous by
catching it.
Verified (rc captured directly, never through a pipe):
cargo check -p aprender-contrastive-data --all-targets rc=0
cargo test -p aprender-contrastive-data 249 passed, 0 failed
cargo clippy -p aprender-contrastive-data --all-targets
-- -D warnings rc=0
make contrastive-data-boundary rc=0 (deps subset + symbol ban)
cargo fmt --all -- --check rc=0
pv validate contracts/contrastive-pair-protocol-v1.yaml 0 errors, 0 warnings
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
I will look tomorrow and merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
aprender-contrastive-data— deterministic, leakage-safe pair constructionContrastive/Siamese data construction as a general capability: class buckets,
balanced few-shot selection, bounded positive/negative pair sampling, typed split
roles, dataset fingerprints, and the cross-split leakage checks that make those
trustworthy.
Independent of #2618. This does not depend on
aprender-coreat all — itswhole dependency closure is 28 packages. Review and merge in either order.
Why a separate crate
Not folded into
aprender-data: that crate carries arrow/parquet/zstd/s3/streaming/federated weight which must not flow into a training crate or a
serverless consumer. SetFit is this crate's first consumer, not its owner.
The bytes boundary
The public API is bytes-in / bytes-out and typed values. No filesystem access, no
sockets, no path-shaped parameters — not even in tests.
apr-cliowns everyfilesystem adapter.
That is not stylistic. The destination for these artifacts is object storage,
where a manifest is an S3 object rather than a file; an API that speaks
&Pathmakes such a consumer a rewrite instead of a wrapper.
It is enforced, not asserted:
make contrastive-data-boundarychecks theresolved dependency closure against a positive allowlist and bans
std::fs/std::net/std::path/Path/PathBufundersrc/. Wired into tier3,because a target outside the tiers is a target that stops being run.
Determinism
Every random decision is a pure function of its draw ordinal, from the
counter-based Philox generator in
aprender-rand. Worker-count independence isstructural rather than tested-for: draw i cannot depend on how many draws
preceded it, because nothing precedes it.
Five typestate misuses are proven unrepresentable with
trybuildcompile-failcases and committed
.stderrsnapshots, rather than asserted in prose.One fix to the gate itself
allowed-deps.txtlistedprovable-contracts-macros— the workspace dependencykey. The crate is
aprender-contracts-macros(the key carriespackage = "..."), and the crate name is whatcargo treeprints, so the entrycould never match. The gate was RED from the commit that scaffolded it.
Corrected to the real package name. This is a name fix, not a widening — the
reviewed decision is unchanged, and the gate proved non-vacuous by catching it.
Verification
rc captured directly, never through a pipe:
Note for whoever merges second
#2618 and this PR both add
aprender-randto[workspace.dependencies]. Thatone line will conflict textually; everything else is disjoint. Whichever lands
second needs a one-line rebase, and I'm happy to do it.
🤖 Generated with Claude Code