fix(fsst): correct decoder output-buffer size contract to 8x

[LuciferYang]

What

The FSST decoder needs its output buffer to be at least 8x the compressed input, but the size guard only required 3x and the public decompress doc claimed "at least 3 times". This raises the guard to 8x (with 32-bit overflow safety), fixes the docs, documents the unsafe invariants, and aligns the in-tree example.

Why

decompress_bulk writes a full 8-byte word per code but advances the output cursor by only the symbol length, relying on a later write (or spare capacity) to overwrite the slack — the standard FSST decode trick:

// fsst.rs, final symbol of a run — no following write to cover the slack
let code = compressed_strs[in_curr] as usize;
unsafe {
    ptr::write_unaligned(out.as_mut_ptr().add(*out_curr) as *mut u64, src); // writes 8 bytes
}
*out_curr += lens[code] as usize; // advances by len (1..=8)

MAX_SYMBOL_LENGTH is 8, so a 1-byte code expands to at most 8 bytes and the whole decoded output is at most 8x the input. The last code of a run is the tightest case: it has no following write, so its 8-byte store must still land inside the buffer. Both facts require the buffer to be at least 8x.

The guard in FsstDecoder::init was:

// when decoder_switch_on is true, we make sure the out_buf is at least 3 times the size of the in_buf,
if self.decoder_switch_on && in_buf.len() * 3 > out_buf.len() {
    return Err(...);
}

3x is too weak: it accepts a buffer between 3x and 8x that the decode loop then writes out of bounds. This is not reachable from the in-tree decompressors (FsstPerValueDecompressor, FsstMiniBlockDecompressor, and the v2.0 FsstPageDecoder) — they already allocate in_buf.len() * 8, so the 3x check never fires for them. But the crate's own benchmark example allocated 3x and would now be rejected, and the guard was unsound for any future caller that trusted the "3 times" wording.

Changes

• Raise the guard from * 3 to * 8, using checked_mul(8) so a >= 512 MiB input can't wrap len * 8 on a 32-bit target and bypass the check (overflow is treated as "too small"). This is a threshold comparison, not an allocation — no extra memory: existing 8x callers still pass, and a <8x buffer is now rejected instead of silently overflowing.
• Fix the misleading "3 times" wording in the guard comment and the decompress pub-doc to 8x, with the 8-byte-write reason.
• Add // SAFETY: comments to the unaligned load/store blocks in fsst_unaligned_load_unchecked and decompress_bulk, per the repo convention of documenting every unsafe block. The comments honestly disclose which conditions are proven (loop guards, the 8x output-buffer check, the symbol-table size check) versus which are trusted preconditions for well-formed input (lens[code] <= 8 and offset values are loaded verbatim from the symbol table / offsets and are not re-validated on decode). Hardening the decoder against corrupt on-disk structures is a separate concern, out of scope here.
• Update the benchmark example's decode buffer from 3x to 8x so it satisfies the corrected guard.
• Add a regression test test_decompress_rejects_undersized_output_buffer asserting a 1-byte-short buffer is rejected and an exact-8x buffer succeeds (fails against the old 3x guard).

Test plan

• cargo test -p fsst — 4/4 pass (incl. new test).
• cargo test -p lance-encoding fsst — 12/12 pass (all consumers).
• cargo fmt -p fsst --check — clean.
• cargo clippy -p fsst --tests --examples -- -D warnings — clean.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!