Commit f856559
feat(writer): pco E6 — IntMult mode (mode=1) with triple-GCD detection
Adds IntMult encoding mode to PcoEncodingEncoder. For integer dtypes,
samples the chunk, computes triple-GCDs over disjoint triples, applies
statistical filter (z-score > 3 vs. uniform-mod-GCD null), and verifies
> 0.5 bits/element savings before switching from Classic.
When chosen:
mult[i] = latent[i] / base
adj[i] = latent[i] % base
Both streams encoded independently with bin DP + tANS; chunk meta
carries mode=1, base, primary bins, secondary bins. Wire format:
page header: primary states (4), secondary states (4), align
per batch: primary ANS, primary offsets, secondary ANS, secondary offsets
Refactored single-stream Classic page encoding into a StreamData helper
shared between Classic and IntMult paths.
FloatMult/FloatQuant deferred — marginal gain over existing Classic+ALP
cascade, significant algorithmic complexity (approx pair-GCD, false
position root finder, trailing-zero detector).
Java→Rust integration test confirms wire format: values × 1000 in I64
column round-trips through Rust JNI decoder.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1 parent 14c98be commit f856559
5 files changed
Lines changed: 503 additions & 58 deletions
File tree
- docs/adr
- integration/src/test/java/io/github/dfa1/vortex/integration
- writer/src
- main/java/io/github/dfa1/vortex/writer/encode
- test/java/io/github/dfa1/vortex/writer/encode
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
Lines changed: 17 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1454 | 1454 | | |
1455 | 1455 | | |
1456 | 1456 | | |
| 1457 | + | |
| 1458 | + | |
| 1459 | + | |
| 1460 | + | |
| 1461 | + | |
| 1462 | + | |
| 1463 | + | |
| 1464 | + | |
| 1465 | + | |
| 1466 | + | |
| 1467 | + | |
| 1468 | + | |
| 1469 | + | |
| 1470 | + | |
| 1471 | + | |
| 1472 | + | |
| 1473 | + | |
1457 | 1474 | | |
1458 | 1475 | | |
1459 | 1476 | | |
| |||
0 commit comments