fix(hip): release Q8 memo before pool teardown#522
Draft
dusterbloom wants to merge 1 commit into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Release
LUCE_Q8_MEMOallocations before the CUDA/HIP context destroys the pools that own them.This keeps backend shutdown on the normal resource-lifecycle path instead of aborting in the legacy HIP pool destructor. It changes no inference kernels, model backends, cache formats, or generated values.
Root cause
ggml_backend_cuda_context::luce_q8_memostoresggml_cuda_pool_allocowners. The context declares its pool array after the memo vector, so C++ member destruction destroys the pools first. A memo allocation retained after the final graph evaluation therefore remains checked out when the legacy pool destructor checkspool_size == 0.On gfx1151, that shutdown assertion aborted the HIP process. The abnormal exit could poison the KFD workqueue and prevent the next HIP process from initializing until reboot.
Changes
ggml_backend_cuda_context's destructor while its pools are still alive.LUCE_Q8_MEMO=1and then frees the backend with the memo allocation still retained.Relationship to the stacked DS4 PRs
This fix was checked against the complete open stack #502 -> #503 -> #496:
ggml-cuda.cufor ROCmFPX types, DS4 HC/SwiGLU, and dispatch, but does not change the context destructor or allocation ownership.luce_q8_memois still declared beforepoolsand the destructor still leaves the final memo entries alive. The teardown bug therefore remains present.server/CMakeLists.txt.The fix is complementary to that DS4 work and does not introduce another backend or duplicate its inference changes.
Validation
test_cuda_pool_shutdownwith-j4ctest -R '^cuda_pool_shutdown$' --output-on-failurepassedmainwith-j4ctest -R '^cuda_pool_shutdown$' --output-on-failurepassedThe known-aborting baseline was not intentionally rerun on gfx1151 because reproducing that abort can poison KFD and require a machine reboot.