Add wasm-bindgen support#23493
Conversation
Does rustc then read the wasm to find those function names, and pass those names to In general if we need to read metadata-type info from the wasm, then we have a minimal parser in tools/webassembly.py. If we need something more complex, a binaryen pass is an option. |
|
wasm-bindgen itself is two pieces: a library that allows you to annotate your rust code marking things to be exported, and a tool that consumes a .wasm file and reads those annotations to produce a companion js file. rustc knows about those function names because wasm-bindgen as a library provided the annotations. If rustc invokes the linker itself, it's able to pass that information along. However, because we need to also build C++, we're only using rustc to compile and not drive the whole process, so we need to have it output that information elsewhere. One (very naive) possibility is to have rustc invoke a fake linker that just writes the |
Export all C symbols, except perhaps those from system libraries.
Automatically infer what symbols to export for wasm-bindgen
…arget
`WorkerPool::spawn` calls `wasm_bindgen::module()` to forward the
current Wasm module handle to a newly-spawned Web Worker. That
intrinsic is only supported by wasm-bindgen-cli's OutputMode::{Web,
NoModules, Node, Module} (see `wasm-bindgen-cli-support/src/js/mod.rs`
`Intrinsic::Module`, which bails out for Bundler and Emscripten).
An emscripten fork with `-sWASM_BINDGEN` integration (e.g. the
`walkingeyerobot/emscripten` draft upstream of
emscripten-core/emscripten#23493) inserts the
`__wasm_bindgen_emscripten_marker` custom section, which forces
wasm-bindgen-cli into `OutputMode::Emscripten` unconditionally. In that
mode the `__wbindgen_module` intrinsic bails, so post-link fails with:
failed to generates bindings for import of
`__wbindgen_placeholder__::__wbg___wbindgen_module_<hash>`
This happens even when the caller never spawns a worker — the mere
reference in the dep graph is enough. Web Workers aren't usable from
the emscripten target regardless (single-threaded model), so fall back
to a null module handle when `target_os = "emscripten"`. Any runtime
call to `spawn` would have failed on that target anyway; this just
keeps the import out of the wasm so wasm-bindgen post-link can complete.
Verified on `wasm32-unknown-emscripten` with wasm-bindgen-cli 0.2.118 +
the walkingeyerobot emscripten fork: `cargo build --release --bin
nobodywho_flutter_web --target wasm32-unknown-emscripten` now produces
a ~15 MB .wasm + matching JS glue that loads and instantiates in Node.
…arget
`WorkerPool::spawn` calls `wasm_bindgen::module()` to forward the
current Wasm module handle to a newly-spawned Web Worker. That
intrinsic is only supported by wasm-bindgen-cli's OutputMode::{Web,
NoModules, Node, Module} (see `wasm-bindgen-cli-support/src/js/mod.rs`
`Intrinsic::Module`, which bails out for Bundler and Emscripten).
An emscripten fork with `-sWASM_BINDGEN` integration (e.g. the
`walkingeyerobot/emscripten` draft upstream of
emscripten-core/emscripten#23493) inserts the
`__wasm_bindgen_emscripten_marker` custom section, which forces
wasm-bindgen-cli into `OutputMode::Emscripten` unconditionally. In that
mode the `__wbindgen_module` intrinsic bails, so post-link fails with:
failed to generates bindings for import of
`__wbindgen_placeholder__::__wbg___wbindgen_module_<hash>`
This happens even when the caller never spawns a worker — the mere
reference in the dep graph is enough. Web Workers aren't usable from
the emscripten target regardless (single-threaded model), so fall back
to a null module handle when `target_os = "emscripten"`. Any runtime
call to `spawn` would have failed on that target anyway; this just
keeps the import out of the wasm so wasm-bindgen post-link can complete.
Verified on `wasm32-unknown-emscripten` with wasm-bindgen-cli 0.2.118 +
the walkingeyerobot emscripten fork: `cargo build --release --bin
nobodywho_flutter_web --target wasm32-unknown-emscripten` now produces
a ~15 MB .wasm + matching JS glue that loads and instantiates in Node.
|
I believe this is ready for review! :D |
walkingeyerobot
changed the title
|
Sorry for the delay reviewing this. Just getting to it now. Is the PR description up-to-date with current state of things? |
| 'CACHE', | ||
| 'PORTS', | ||
| 'COMPILER_WRAPPER', | ||
| 'WASM_BINDGEN', |
There was a problem hiding this comment.
Could we avoid the new config setting completely and just rely on wasm-bindgen being in the PATH when a user added -sWASM_BINDGEN.
This is what we do for tsc I believe. I'm loath to add more config setting if we can possibly avoid it.
| # We use an out-of-tree cache directory so it can be part of the | ||
| # persisted workspace (see below). | ||
| echo "CACHE = os.path.expanduser('~/cache')" >> .emscripten | ||
| echo "WASM_BINDGEN = os.path.expanduser('~/.cargo/bin/wasm-bindgen')" >> .emscripten |
There was a problem hiding this comment.
Can we just add ~/.cargo/bin to the PATH instead? (see commend in config.py)
| for file_path in bindgen_tsd: | ||
| with open(file_path, encoding='utf-8') as file: | ||
| for line in file: | ||
| out += f'{line}' |
There was a problem hiding this comment.
Can these 3 lines be replaced with just out += read_file(file_path)?
|
|
||
| if settings.WASM_BINDGEN: | ||
| building.run_wasm_bindgen(in_wasm) | ||
| settings.JS_LIBRARIES += [get_emscripten_temp_dir() + '/bindgen_out/library_bindgen.js'] |
There was a problem hiding this comment.
Can run_wasm_bindgen return the filename here?
bindgen_jslib = building.run_wasm_bindgen(in_wasm)
settings.JS_LIBRARIES.append(bindgen_jslib)
| """ | ||
|
|
||
| value: str | ||
| is_file: int |
There was a problem hiding this comment.
Perhaps putting this in cmdline.py would make more sense?
I'm still not clear why we need to move this, but trying to understand now.
| def test_rust_integration_basics(self): | ||
| copytree(test_file('rust/basics'), '.') | ||
| self.run_process(['cargo', 'build', '--target=wasm32-unknown-emscripten']) | ||
| self.run_process(['cargo', 'build']) |
There was a problem hiding this comment.
Why isn't --target needed here anymore? .. Oh I see its in the cargo.toml file now.
Maybe just revert these changes to test_rust_integration_basics ? Since they seem unrelated?
5 participants
This is an early draft PR for the purposes of gathering feedback early. There are also pending changes to wasm-bindgen.
How this works:
wasm32-unknown-emscripteninto a.afile..afile.wasm-ldto link the C++ and Rust into a.wasmfile..wasmfile, producing a new.wasmfile, alibrary.jsfile, and apre.jsfile..js, integrating the wasm-bindgen.jsfiles.You can see a demo more easily at https://github.com/walkingeyerobot/cxx-rust-demo.
library_wbg.jsandpre.jsare approximately what will be produced by wasm-bindgen for consumption by Emscripten.Some TODOs:
wasm-ldso they're not removed in the final.wasmbut that may not necessarily be present after wasm-bindgen processes the.wasm. wasm-bindgen at compile time puts the information it needs to generate JS inside the.wasmfile itself in the form of_describefunctions. These functions are then removed after JS generation..jsfiles produced by wasm-bindgen. This shouldn't be that hard; I just haven't gotten around to it yet. This would simplify the code for both Emscripten and wasm-bindgen.-unknownand-emscriptenthat I'll have to fix.I'm mostly looking for feedback on the first point about exported symbols and about the general addition of
-sWASM_BINDGENto Emscripten. Again, this is very early, but it's a pretty big feature, so I thought it best to start discussions now.cc @daxpedda, who I've been working with on the wasm-bindgen side.