Skip to content

Make the fuzz targets build and run again - #343

Open
MayCXC wants to merge 2 commits into
asg017:mainfrom
MayCXC:fuzz-targets-build
Open

MayCXC wants to merge 2 commits into
asg017:mainfrom
MayCXC:fuzz-targets-build

Conversation

@MayCXC

@MayCXC MayCXC commented Oct 23, 2026 •

Copy link
Copy Markdown

No fuzz target has built since 575371d:

$ make -C tests/fuzz all
Makefile:158: *** missing separator (did you mean TAB instead of 8 spaces?).  Stop.

(The Fuzz workflow is disabled on this repository; its last runs, in March, failed.) Two commits:

Makefile. 575371d, which added the DiskANN targets, dropped the line continuation after ivf_rescore in FUZZ_TARGETS and the ivf_rescore rule's recipe, and 0de765f removed numpy.c, which the numpy target still compiles. The list continues again, ivf_rescore has its recipe, and the numpy target and its dictionary are gone. make -C tests/fuzz all builds all 36 targets.

rescore_quantize_edge. Mode 3 asserted that a vector of one repeated value quantizes to all zeros, as the min/max int8 quantizer did for a zero range; 575371d replaced that quantizer with a fixed map of [-1, 1] onto [-128, 127], which gives 0 only for values near 0.004, so the target aborts within a few hundred inputs on main (a vector of NaN gives 127). It now asserts that every element gets the same code, and the comments on modes 2 and 4 describe the current quantizer's clamps. Modes 1 and 2 read the fuzz bytes as floats from one byte past the mode byte, which UBSan reports as a misaligned load in rescore_quantize_float_to_int8(); they copy the bytes to a float array first.

Test. Each of the 36 targets ran for 60 seconds as the workflow runs them (ASAN_OPTIONS=detect_leaks=1, the target's dictionary), with clang 19 on Linux arm64:

It merges cleanly with #327 to #331 and #336 to #342. #355 builds on it to compile the IVF index into the eight IVF targets.

Origin. The mode 3 rewrite is the rescore port's to photostructure/sqlite-vec, photostructure#8 (cd4bb09), which wrote the uniform-vector case for the current quantizer; the Makefile repair and the copies in modes 1 and 2 are new here. vlasky/sqlite-vec@3768246 ports this suite to vlasky's fork without the DiskANN, IVF and rescore targets, and mceachen's photostructure/sqlite-vec@b84a2c3 runs those twelve targets on Linux; neither touches the lines repaired here. No other open or closed PR or issue here repairs the fuzz build.

575371d, which added the DiskANN targets, dropped the line continuation
after ivf_rescore in FUZZ_TARGETS and the ivf_rescore rule's recipe, so
make stops at the first DiskANN name and builds nothing:

    $ make -C tests/fuzz all
    Makefile:158: *** missing separator (did you mean TAB instead of 8
    spaces?).  Stop.

0de765f then removed numpy.c, which the numpy target still compiles. The
list continues past ivf_rescore again, ivf_rescore has its recipe, and the
numpy target and its dictionary are gone. make -C tests/fuzz all builds all
36 targets.
575371d replaced rescore's min/max int8 quantizer with a fixed map of
[-1, 1] onto [-128, 127], and mode 3 still asserted that a vector of one
repeated value quantizes to all zeros, as the replaced quantizer did for a
zero range. The current quantizer gives 0 only for values near 0.004, so
the target aborts within a few hundred inputs (a vector of NaN gives 127).
Mode 3 now asserts that every element gets the same code, which holds for
any element-wise quantizer, and the comments on modes 2 and 4 describe the
clamps the current quantizer has.

Modes 1 and 2 read the fuzz bytes as floats from one byte past the mode
byte, a misaligned load UBSan reports in rescore_quantize_float_to_int8();
they copy the bytes to a float array first. The target runs 3.7 million
inputs in 60 seconds without a report.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant