Skip to content

fix(hip): release Q8 memo before pool teardown#522

Draft
dusterbloom wants to merge 1 commit into
Luce-Org:mainfrom
dusterbloom:agent/fix-hip-pool-shutdown
Draft

fix(hip): release Q8 memo before pool teardown#522
dusterbloom wants to merge 1 commit into
Luce-Org:mainfrom
dusterbloom:agent/fix-hip-pool-shutdown

Conversation

@dusterbloom

@dusterbloom dusterbloom commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Release LUCE_Q8_MEMO allocations before the CUDA/HIP context destroys the pools that own them.

This keeps backend shutdown on the normal resource-lifecycle path instead of aborting in the legacy HIP pool destructor. It changes no inference kernels, model backends, cache formats, or generated values.

Root cause

ggml_backend_cuda_context::luce_q8_memo stores ggml_cuda_pool_alloc owners. The context declares its pool array after the memo vector, so C++ member destruction destroys the pools first. A memo allocation retained after the final graph evaluation therefore remains checked out when the legacy pool destructor checks pool_size == 0.

On gfx1151, that shutdown assertion aborted the HIP process. The abnormal exit could poison the KFD workqueue and prevent the next HIP process from initializing until reboot.

Changes

  • Drain Q8 memo entries in LIFO order from ggml_backend_cuda_context's destructor while its pools are still alive.
  • Preserve the LIFO release requirement of VMM pools.
  • Add a CUDA/HIP regression test that performs a Q4_0 matvec with LUCE_Q8_MEMO=1 and then frees the backend with the memo allocation still retained.

Relationship to the stacked DS4 PRs

This fix was checked against the complete open stack #502 -> #503 -> #496:

The fix is complementary to that DS4 work and does not introduce another backend or duplicate its inference changes.

Validation

  • CUDA 12.6, GCC 11, RTX 3090, sm_86:
    • built test_cuda_pool_shutdown with -j4
    • ctest -R '^cuda_pool_shutdown$' --output-on-failure passed
  • ROCm 7.2.4, Strix Halo gfx1151:
    • clean build from current main with -j4
    • ctest -R '^cuda_pool_shutdown$' --output-on-failure passed
    • two serial test processes both initialized and shut down successfully; the second HIP initialization remained healthy after the first teardown

The known-aborting baseline was not intentionally rerun on gfx1151 because reproducing that abort can poison KFD and require a machine reboot.

Review in cubic

@davide221 davide221 added bug Something isn't working ds4flash labels Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ds4flash

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants