Replace convertJSArrayToNumberVector with direct WASM frame buffer#29
Open
Replace convertJSArrayToNumberVector with direct WASM frame buffer#29
Conversation
Allocate a persistent uint8_t* buffer in WASM linear memory once via initFrameBuffer(colorSpace), expose its byte offset via getFrameBufferPtr(), and write each video frame directly with HEAPU8.set() on the JS side. This eliminates the per-frame heap allocation and element-by-element copy that convertJSArrayToNumberVector<uint8_t> performed on every processFrame call. - emscripten/WebARKitJS.h: add initFrameBuffer, getFrameBufferPtr, destructor, m_frameBuffer/m_frameBufferSize private members; update processFrame signature - emscripten/WebARKitJS.cpp: implement buffer allocation, bppForColorSpace helper, null guard in processFrame, destructor frees buffer - emscripten/bindings.cpp: expose initFrameBuffer and getFrameBufferPtr to JS - src/WebARKitController.js: add initFrameBuffer wrapper, lazy-init in processFrame, HEAPU8.set() direct write instead of passing JS array to WASM - examples/worker_threejs.js: explicit initFrameBuffer(RGBA) after tracker load - examples/worker.js: explicit initFrameBuffer(GRAY) - examples/worker_bufferCopy_threejs.js: explicit initFrameBuffer(RGBA) - examples/worker_jsfeatNext.js: explicit initFrameBuffer(GRAY) - build/dist: rebuilt artifacts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
convertJSArrayToNumberVector<uint8_t>()walked the JS typed array element-by-element and created a freshstd::vectoron every frame. For a 1280×720 RGBA frame that's ~3.7 MB allocated and freed on every call.uint8_t*buffer once viainitFrameBuffer(colorSpace), exposes its WASM linear-memory byte offset viagetFrameBufferPtr(), and JS writes video data directly withHEAPU8.set()before callingprocessFrame.processFrameinWebARKitController.jsauto-callsinitFrameBufferon first use if not called explicitly, so existing callers keep working without changes.worker_threejs.js,worker.js,worker_bufferCopy_threejs.js,worker_jsfeatNext.jsall callinitFrameBufferexplicitly with the correct colorspace after tracker load.Changes
emscripten/WebARKitJS.hinitFrameBuffer,getFrameBufferPtr, destructor, private buffer members; updateprocessFramesignatureemscripten/WebARKitJS.cppbppForColorSpacehelper, null guard, destructoremscripten/bindings.cppinitFrameBufferandgetFrameBufferPtrto JS via embindsrc/WebARKitController.jsinitFrameBufferwrapper, lazy-init +HEAPU8.set()inprocessFrameexamples/worker_threejs.jsinitFrameBuffer(RGBA)examples/worker.jsinitFrameBuffer(GRAY)examples/worker_bufferCopy_threejs.jsinitFrameBuffer(RGBA)examples/worker_jsfeatNext.jsinitFrameBuffer(GRAY)build/,dist/Pitfalls addressed
HEAPU8is read fresh each frame (not cached) to surviveALLOW_MEMORY_GROWTHheap relocationsinitFrameBuffercolorspace must matchprocessFramecolorspace — all examples verified and corrected (found and fixed aGRAY/RGBAmismatch inworker_jsfeatNext.js)initTrackerGrayintentionally left unchanged (one-time call, copy overhead negligible)Test plan
npm run build-docker— compiled cleanly, no new warningsnpm run build-es6— webpack bundled successfullyexamples/teblid_example.htmlorthreejs_worker_ES6.jsin browser and verify marker tracking still works🤖 Generated with Claude Code