Repository navigation
Conversation
This was referenced Oct 23, 2026
The Makefile enabled NEON only on macOS arm64, so a Linux arm64 build, the linux-aarch64 release binary among them, ran every distance's scalar loop. NEON is part of the arm64 baseline: GCC and Clang define __ARM_NEON for aarch64 with no -m flag, so the Linux block needs only the define, beside its AVX check and inside its guard against Android cross-compiles. OMIT_SIMD=1 still builds without it. The define follows the compiler's target, `$(CC) -dumpmachine`, not uname, as mceachen's photostructure/sqlite-vec@350c67a has it: 32-bit Raspberry Pi OS runs a 64-bit kernel, so uname says aarch64 while gcc targets 32-bit ARM, which lacks the AArch64-only intrinsics the NEON kernels use. Under an aarch64 kernel, `make loadable CC=arm-linux-gnueabihf-gcc` fails when uname decides (implicit declaration of vaddvq_f32, unknown type name float64x2_t) and builds a 32-bit ARM extension when the compiler does; for gcc and clang targeting aarch64 the build commands are the same either way. GCC cannot build the NEON code on main, because l1_int8_neon() relies on implicit signed/unsigned vector conversions; this builds on asg017#322, which makes them explicit. Built with GCC 14 at -O3 on Linux arm64, a KNN query on a 100,000-row float[768] table takes 19.6 ms instead of 59.2 ms for k = 10 and 24.2 ms instead of 65.8 ms for k = 100 with the L2 metric, and 17.5 ms instead of 85.3 ms and 23.3 ms instead of 90.6 ms with cosine (medians of 20 queries, each build in its own process: the extension exports its internal functions, so a second build loaded into the same process runs the first one's KNN code). L1 and Hamming distances were bit-identical to the scalar loops' on 2,000 random pairs at each of six lengths from 8 to 1,536, though the float32 L1 kernel adds in another order than the scalar loop, both in double, so the two can differ in the last digit on other inputs. Float32 L2 and cosine distances differ in the last digits, since the NEON kernels split their sums across 16 partial sums where the scalar loops keep one; for L2 the NEON result is closer to the exact value, a mean relative error of 3.0e-8 instead of 1.7e-7 at 768 dimensions. So a KNN query can order rows with nearly equal distances differently, as the macOS arm64 build already does; 4 of the 2,000 ranks of the k = 100 L2 queries swapped such rows. Int8 cosine distances can differ in the last digit as well, since cosine_int8_neon() finishes in float and the scalar loop in double. The Test workflow's build-linux-aarch64-extension job already runs the suite on an arm64 runner, so it now tests the NEON build. Locally the suite passes on it (478 passed, 132 skipped, 204 snapshots).
MayCXC
force-pushed
the
neon-linux-arm64
branch
from
October 23, 2026 16:05
eba5572 to
8d133f2
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This builds on #322 and carries its commit: GCC cannot build the NEON code on main (#299), because
l1_int8_neon()relies on implicit signed/unsigned vector conversions, and #322 makes them explicit. Once #322 is merged, this is the one commit after it.The Makefile enables NEON only on macOS arm64, so a Linux arm64 build, the linux-aarch64 release binary among them, runs every distance's scalar loop. NEON is part of the arm64 baseline: GCC and Clang define
__ARM_NEONfor aarch64 with no-mflag, so the Linux block needs only the define, beside its AVX check and inside its guard against Android cross-compiles.OMIT_SIMD=1still builds without it.The define follows the compiler's target,
$(CC) -dumpmachine, notuname -m: 32-bit Raspberry Pi OS runs a 64-bit kernel, so uname says aarch64 while gcc targets 32-bit ARM, which lacks the AArch64-only intrinsics the NEON kernels use. Under an aarch64 kernel,make loadable CC=arm-linux-gnueabihf-gccfails when uname decides (implicit declaration of function 'vaddvq_f32',unknown type name 'float64x2_t') and builds a 32-bit ARM extension when the compiler does. For gcc and clang targeting aarch64 the build commands are the same either way.Built with GCC 14 at
-O3on Linux arm64 (medians of 20 queries on a 100,000-rowfloat[768]table, each build in its own process: the extension exports its internal functions, so a second build loaded into the same process runs the first one's KNN code):What changes in the results, on 2,000 random pairs at each of six lengths from 8 to 1,536:
float32L1 kernel adds in another order than the scalar loop, both in double, so the two can differ in the last digit on other inputs.float32L2 and cosine distances differ in the last digits, since the NEON kernels split their sums across 16 partial sums where the scalar loops keep one; for L2 the NEON result is closer to the exact value, a mean relative error of 3.0e-8 instead of 1.7e-7 at 768 dimensions. So a KNN query can order rows with nearly equal distances differently, as the macOS arm64 build already does; 4 of the 2,000 ranks of the k = 100 L2 queries above swapped such rows.int8cosine distances can differ in the last digit, sincecosine_int8_neon()finishes in float and the scalar loop in double.int8L2 distances differ only past about 1,536 random dimensions, where the scalar loop's float sum stops being exact; the kernel's 32-bit sum overflows invec_distance_l2()past 33,026 elements at opposite extremes, as it already does on macOS arm64, and vec0 tables allow at most 8,192 dimensions.The Test workflow's
build-linux-aarch64-extensionjob already runs the suite on an arm64 runner, so it now tests the NEON build. Locally the suite passes on it (478 passed, 132 skipped, 204 snapshots).Origin. This is photostructure/sqlite-vec@5fb0b3d (photostructure#6), with mceachen's photostructure/sqlite-vec@350c67a, which took the define from the compiler's target there and noted the L1 summation order. vlasky#21 turns NEON on for vlasky's fork. No open or closed PR or issue here turns NEON on for Linux arm64.
It merges cleanly with #338 and with #327 through #331, #336 and #337. It conflicts with stumpylog's #307 in the Makefile's Linux block, which #307 limits to x86_64 for its AVX change.