Vulkan: export allocator for custom contexts by xFile3160 · Pull Request #8871 · halide/Halide

xFile3160 · 2025-11-17T21:55:24Z

This keeps the PR focused on the allocator/context part of #8715.

When an application owns the Vulkan instance, device, and queue, it still
needs a Halide allocator for shader modules, staging buffers, descriptor
resources, and runtime allocations. Today that allocator is private, so an
override of halide_vulkan_acquire_context() cannot safely keep and return
the allocator that Halide created for the application's Vulkan handles.

This adds public acquire/release helpers for the opaque Vulkan memory
allocator:

halide_vulkan_acquire_memory_allocator()
halide_vulkan_release_memory_allocator()

The acquire helper can create an allocator for caller-supplied Vulkan
handles, or validate and reuse an allocator returned by a later context
override. The release helper tears down Halide-owned allocator state and
shader-module cache entries without destroying externally owned Vulkan
objects.

The Vulkan compilation cache is keyed by allocator instead of raw VkDevice
so shader module cleanup follows the allocator lifetime. For Halide-owned
contexts this keeps the same practical behavior, because the default
allocator and default device have the same lifetime. For externally managed
contexts it avoids sharing cache ownership through a bare device handle.

The imported-buffer metadata and offset handling from #8715 is split into
#9110 to keep this review limited to allocator ownership.

Validation on Linux:

git diff --check
direct freestanding compile of src/runtime/vulkan.cpp
direct freestanding compile of src/runtime/runtime_api.cpp
cmake -S . -B build/pr-8871-vulkan-upstream -G Ninja -DCMAKE_BUILD_TYPE=Release -DHalide_WASM_BACKEND=OFF -DWITH_PYTHON_BINDINGS=NO
cmake --build build/pr-8871-vulkan-upstream

The CMake build completed 3972/3972. Python bindings were disabled
because this local machine does not have pybind11 configured for the
default Halide CMake build.

xFile3160 · 2025-11-17T21:56:48Z

Full discussion available here: #8715 (comment)

alexreinking · 2025-11-18T16:51:19Z

@derek-gerstmann could you review this PR? Seems you were discussing the related issue.

xFile3160 · 2026-01-20T21:42:35Z

Any news about this? Should I rebase?

alexreinking · 2026-01-20T23:14:32Z

Hi @xFile3160 -- happy new year! Yes, please rebase (looks like you just did). I'll ping @derek-gerstmann again to look at this since he has in-depth knowledge of the Vulkan backend.

derek-gerstmann · 2026-01-20T23:27:22Z

Thanks for the reminder! I'll look this over this week!

derek-gerstmann · 2026-01-20T23:49:21Z

+//   with the same locking used by the custom acquire/release implementations. This allows the allocator to be
+//   saved for future halide_vulkan_acquire_context calls that Halide will automatically issue to retrieve
+//   the custom context.
+extern int halide_vulkan_export_memory_allocator(void *user_context,


I don't understand the need for this method, or for the corresponding release method. The allocator should be stored in your custom context, and held onto for the lifetime of the context. The context manages lifespan of the allocator.

derek-gerstmann · 2026-01-20T23:49:32Z

+// - halide_vulkan_memory_allocator_release
+//   releases the internally allocated memory allocator, important for proper memory cleanup. Must have overridden halide_vulkan_acquire_context
+//   and halide_vulkan_release_context, and must coordinate with the same locking as the custom implementations.
+extern int halide_vulkan_memory_allocator_release(void *user_context,


See above comment.

derek-gerstmann · 2026-01-20T23:50:38Z

    return is_initialized;
 }

+WEAK int halide_vulkan_export_memory_allocator(void *user_context, halide_vulkan_memory_allocator *allocator) {


This doesn't actually do anything other than check to see if the allocator is null.

derek-gerstmann · 2026-01-20T23:52:00Z

    return destroy_status;
 }

+WEAK int halide_vulkan_memory_allocator_release(void *user_context,


Not sure I understand the intent ... was it to have a public method to invoke the destructor for the allocator?

derek-gerstmann · 2026-01-20T23:57:12Z

            error = halide_error_code_device_interface_no_device;
            halide_error_no_device_interface(user_context);
        }
+        // If user overrode halide_vulkan_acquire_context and returned nullptr for allocator,


This class shouldn't be doing anything other than holding a lock on the context. It's just a convenient wrapper for the internal methods to have a lock that lives within a scope.

derek-gerstmann · 2026-04-22T20:51:17Z

+//   with the same locking used by the custom acquire/release implementations. This allows the allocator to be
+//   saved for future halide_vulkan_acquire_context calls that Halide will automatically issue to retrieve
+//   the custom context.
+extern int halide_vulkan_export_memory_allocator(void *user_context,


I'd suggest following the conventions of the context methods and naming this halide_vulkan_acquire_memory_allocator.

derek-gerstmann · 2026-04-22T20:51:39Z

+// - halide_vulkan_memory_allocator_release
+//   releases the internally allocated memory allocator, important for proper memory cleanup. Must have overridden halide_vulkan_acquire_context
+//   and halide_vulkan_release_context, and must coordinate with the same locking as the custom implementations.
+extern int halide_vulkan_memory_allocator_release(void *user_context,


Same as above. I'd suggest I'd suggest naming this halide_vulkan_release_memory_allocator.

derek-gerstmann · 2026-04-22T20:52:36Z

 }

+WEAK int halide_vulkan_export_memory_allocator(void *user_context, halide_vulkan_memory_allocator *allocator) {
+    halide_mutex_lock(&thread_lock);


This default implementation doesn't actually do anything ... shouldn't it return the allocator associated with the context?

derek-gerstmann · 2026-04-22T20:54:55Z

+        return halide_error_code_buffer_argument_is_null;
+    }
+
+    return vk_release_memory_allocator(user_context, (VulkanMemoryAllocator *)allocator,


Lifetime management is an issue here. How do we know there are no remaining uses for the allocator? Also, allocators are specific to the context, so we need to make sure the given allocator matches the one associated with the given context.

derek-gerstmann · 2026-04-22T20:57:11Z

+            halide_start_clock(user_context);
+#endif
+            // make sure halide vulkan is loaded BEFORE creating allocator
+            debug(user_context) << "VulkanContext: Loading Vulkan function pointers for context override...\n";


This is not the right place to initialize device function pointers. They are specific to the context, and should only be initialized once, which is why they are only done in the acquire_context method.

xFile3160 · 2026-04-22T21:19:04Z

I've changed this patch quiet a bit actually. But @derek-gerstmann your comments make absolutely sense, and I'm going to address/explain the intent and the new API a bit better soon. Sorry, I haven't followed up either updating this patch.

When an embedder owns the Vulkan instance, device, and queue, it still needs a Halide allocator for shader modules, staging buffers, and other runtime allocations. Add acquire/release helpers for the opaque Vulkan allocator so embedders can store it with their context and return it from later acquire calls. The helpers reload Vulkan function pointers for the supplied context, validate that a reused allocator still matches the device, and release only Halide-owned allocator and shader-module state. Key the Vulkan compilation cache by allocator instead of VkDevice so external allocators have independent shader-module lifetimes even when they share the same device handle.

alexreinking requested a review from halidebuildbots November 17, 2025 22:31

xFile3160 force-pushed the main branch from b0258f8 to 5b36964 Compare January 20, 2026 21:56

xFile3160 closed this Feb 16, 2026

xFile3160 force-pushed the main branch from f802490 to 1877f41 Compare February 16, 2026 15:49

xFile3160 reopened this Feb 16, 2026

alexreinking requested a review from derek-gerstmann February 16, 2026 16:36

derek-gerstmann requested changes Apr 22, 2026

View reviewed changes

xFile3160 force-pushed the main branch from 5bb201a to eaa2054 Compare April 25, 2026 08:40

xFile3160 mentioned this pull request Apr 25, 2026

Vulkan: wrap external buffers as regions #9110

Draft

Conversation

xFile3160 commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xFile3160 commented Nov 17, 2025

Uh oh!

alexreinking commented Nov 18, 2025

Uh oh!

xFile3160 commented Jan 20, 2026

Uh oh!

alexreinking commented Jan 20, 2026

Uh oh!

derek-gerstmann commented Jan 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xFile3160 commented Apr 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

xFile3160 commented Nov 17, 2025 •

edited

Loading