Export impl_hll_bucket_list! so callers can build any HLL precision

[GordonYuanyc]

Closes #94 (part (a) only — see Not in scope below).

What

Six commits:

1. bd558fc — export the macro. Mark impl_hll_bucket_list! #[macro_export].
   Downstream crates can now generate an HllRegisterStorage type at any
   precision (e.g. lg_k = 18, which sketch-bench's lkarger search space
   needs) instead of being stuck with the built-in HllBucketListP12/P14/P16.
   The generated type is the same monomorphized fast path, not a slow fallback.

2. c614acd — test it. tests/hll_custom_precision.rs.

3. debe577 — make ErtlMLE::estimate generic, so custom precisions can
   query, not just insert and merge.

4. f60317d — heap-allocate register storage, so large precisions don't
   overflow the stack in debug builds.

5. 72c190e — harden the macro against caller context (from review): qualify
   Result/Box/Default/unreachable!/derive paths, and assert
   num_registers == 1 << precision at compile time.

6. 02ec700 — tighten the tests (from review): non-vacuous lg_k=4 tolerance,
   ASAPv1 envelope round-trip for all three variants at a custom precision.

asap_sketchlib::impl_hll_bucket_list!(HllBucketListP18, 18, 1_usize << 18);

let mut hll = HyperLogLogImpl::<Classic, HllBucketListP18>::new();

Commits 3 and 4 came out of checking the full variant × operation matrix at a
custom precision. Result after this PR — every cell works:

variant	insert	query	merge	ser/de
Classic	✓	✓	✓	✓
ErtlMLE	✓	✓ (was ✗, commit 3)	✓	✓
HIP	✓	✓	n/a — no merge method, by design	✓

Commit 1 — note on "one line"

#[macro_export] is the one functional line, but it can't ship alone: the macro
body referenced Serialize, Serializer, Deserialize, Deserializer,
serde_big_array::BigArray, Index, IndexMut, Range and
HllRegisterStorage by bare name, so the expansion only compiled inside this
module. The rest is mechanical path-qualification — $crate::__private::serde,
$crate::HllRegisterStorage, ::std::ops::* — exactly the pattern
impl_fixed_matrix! already uses (748f179). Without it a downstream caller
would have to import all eight names and take a serde-big-array dependency of
their own.

Also in commit 1: a doc example on the macro (runs as a doctest, i.e. from
outside the crate), a CHANGELOG entry, and one bullet in docs/features.md
next to impl_fixed_matrix!.

Commit 3 — ErtlMLE at custom precisions

estimate for ErtlMLE came from impl_ertl_mle_estimate!, instantiated only
for P12/P14/P16, so a custom-precision sketch could insert and merge but
estimate did not exist:

error[E0599]: no method named `estimate` found for struct `HyperLogLogImpl<ErtlMLE, P18>`

The only thing forcing the macro was the exact-sized histogram
[u32; REGISTER_BITS + 2], unwritable in a generic impl while
generic_const_exprs is unstable. It doesn't need to be exact: REGISTER_BITS + 2
peaks at 66, so one [u32; 66] covers every precision and the loop bounds stay
Registers::REGISTER_BITS — still a compile-time constant per monomorphization.
The extra slots are zeros the algorithm never reads, so estimates at existing
precisions are unchanged. Net −4 lines; also drops the two #[cfg(test)]
harness macros that only existed to work around the same limitation.

Commit 4 — the debug stack overflow

Default built [0_u8; N] on the stack and copied it into the box; Deserialize
did the same via BigArray. Release elides those temporaries — debug does not.
On a 2 MiB test thread:

operation	before (debug)	after (debug)
default() / new()	aborts at lg_k ≥ 22	fine (no stack ceiling; tested to lg_k 30 / 1 GiB)
rmp_serde::from_slice	aborts at lg_k ≥ 17	fine
deserialize_from_bytes (ASAPv1)	fine	fine

Not a new bug — it's equally true of the shipped HllBucketListP16 — but
exporting the macro makes the precisions where it bites reachable, and
fatal runtime error: stack overflow is a bad first experience for someone who
just generated a P18 type. Fix routes both paths through a heap allocation that
never names an [u8; N] value, so behavior no longer depends on the optimizer.

Wire compatibility: unchanged. Serialize still uses BigArray; the new
visitor reads the same fixed-length sequence via deserialize_tuple. Verified
directly — bytes written by the pre-change implementation decode to identical
registers under the new one and re-encode byte-for-byte (16388 bytes, P14).

Review follow-ups (commits 5–6)

A review pass turned up two real defects in the exported macro, both reproduced
before fixing and re-verified after:

• Unqualified Result broke the expansion in ordinary caller modules. With
  use std::io::Result or use std::fmt::Result in scope — the aliases take a
  different number of generic arguments — the macro failed with 8 errors
  (E0053/E0107/E0277/E0308). Commit 1 had qualified serde, std::ops, vec!
  and write! but missed Result in three signatures, plus Box, Default
  and unreachable!. Now verified against a downstream crate that shadows
  Result two ways and declares its own Default trait, under deny(warnings).

• Nothing tied precision to num_registers.
  impl_hll_bucket_list!(Foo, 18, 1_usize << 20) compiled and silently produced
  wrong estimates forever: the bucket index is (hash >> REGISTER_BITS) & P_MASK,
  so only 1 << 18 registers are ever reachable while estimate() divides by
  1 << 20. Measured on the broken pair: 83,131 of 1,048,576 registers ever
  touched. No panic, no bounds violation — it just returns bad numbers. A
  const _: () block now rejects it at compile time:

  error[E0080]: evaluation panicked: impl_hll_bucket_list!: num_registers must equal 1 << precision
  

  precision >= 1 is checked the same way, and HllRegisterStorage now
  documents the relations hand-written impls must uphold.

The review also independently confirmed the PR's three load-bearing claims:
ErtlMLE estimates byte-identical across P12/P14/P16 × 8 cardinalities; MessagePack
output byte-identical (16388 B, cmp clean, and bytes cross-decode both
directions); insert throughput unchanged.

Known, deliberately not fixed here: impl_fixed_matrix! in the same file has
the identical bare-Result flaw (pre-existing, and it is already exported). It
deserves its own pass over all of its unqualified names rather than a partial fix
smuggled into this PR.

Test coverage

tests/hll_custom_precision.rs is an integration test, so it compiles as a
separate crate against the public surface only — invoking the macro there is
itself the check that the export works downstream. It instantiates lg_k
4, 8, 10, 13, 18, and 22 (the last only in the allocation test) and asserts:

• storage constants (PRECISION, NUM_REGISTERS, REGISTER_BITS, P_MASK,
  len, zeroed on default())
• Classic, ErtlMLE, and HIP estimates within tolerance at every precision,
  over deterministic inputs (so the assertions can't flake); tolerances are ~4x
  the 1.04/sqrt(m) standard error, and measured errors sit well inside them
  (e.g. lg_k=18: classic 0.01%, ErtlMLE 0.01%, HIP 0.15%)
• register ranks stay within REGISTER_BITS + 1
• merge of two overlapping lg_k=18 sketches recovers the union cardinality
• Index/IndexMut for usize and Range, and IntoIterator
• MessagePack round-trip at lg_k=13, and an ASAPv1 envelope round-trip at a
  custom precision for all three variants — including HIP's running scalars,
  plus a negative case (lg_k=18 bytes must not decode as lg_k=13)
• stack regression at lg_k 18 and 22 — this test aborts without commit 4

cargo test and cargo test --release both fully green (505 unit +
6/7/12/4/1 integration + 21 doctests). cargo fmt --check and
cargo clippy --all-targets clean.

Not in scope

Part (b), the Vec<u8> runtime-sized fallback, is not here. Worth a separate
PR — with (a) in place, every precision is reachable, so (b) becomes a
convenience (no type declaration needed up front) rather than the only way to
run at lg_k=18. Note the portable wire type message_pack_format::portable::HllSketch
already carries a runtime precision, so there's prior art for the shape.

🤖 Generated with Claude Code