feat(distance): Add AVX512-FP16 native distance kernels for Float16 data - #365
Draft
yuejiaointel wants to merge 1 commit into
Draft
feat(distance): Add AVX512-FP16 native distance kernels for Float16 data#365yuejiaointel wants to merge 1 commit into
yuejiaointel wants to merge 1 commit into
Conversation
Adds hardware-native fp16 distance computation for CPUs with AVX512-FP16 support (Granite Rapids, Sapphire Rapids). Without this fix, the AVX512-FP16 dispatch tier on capable hardware silently fell through to a scalar generic, causing ~30% QPS regression vs gcc11 baseline. Changes: - distance_core.h: add AVX512_FP16 to AVX_AVAILABILITY enum - avx_detection.h: CPUID check for fp16 (bit 23, not 29) - inner_product.h/euclidean.h/cosine.h: AVX512_FP16 dispatch tier with fallback inheritance to AVX512 for unspecialized types - avx512_fp16.cpp: IPNativeHalfOp32 (Float16×Float16, vfmaddph, 32-lane) and IPMixedFloatToHalfOp32 (float×Float16, converts query on-the-fly) - cmake/multi-arch.cmake: wire avx512_fp16.cpp into cascadelake build - avx512.cpp: extern-template declarations for AVX512_FP16 tier
yuejiaointel
marked this pull request as draft
August 18, 2026 16:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds hardware-native fp16 distance computation for CPUs with AVX512-FP16 support (Granite Rapids, Sapphire Rapids). Without this fix, the AVX512-FP16 dispatch tier on capable hardware silently fell through to a scalar generic, causing ~30% QPS regression vs gcc11 baseline.
Changes
distance_core.h: addAVX512_FP16toAVX_AVAILABILITYenumavx_detection.h: CPUID check for AVX512-FP16 (bit 23, was incorrectly bit 29)inner_product.h/euclidean.h/cosine.h: AVX512_FP16 dispatch tier with fallback inheritance to AVX512 for unspecialized typesavx512_fp16.cpp:IPNativeHalfOp32(Float16×Float16,vfmaddph, 32-lane) andIPMixedFloatToHalfOp32(float×Float16, converts query on-the-fly to fp16)cmake/multi-arch.cmake: wireavx512_fp16.cppinto cascadelake buildavx512.cpp: extern-template declarations for AVX512_FP16 tierRelated
Follows #361 (prefetch fence fix). Together these two PRs restore full performance on gcc-12 compiled builds with AVX512-FP16 hardware.