feat(build): --size, a mode that optimises for artifact bytes (#1729) - #1731
Merged
Conversation
ae build had --quick (-O0 -g), --profile (-O2 -g -fno-omit-frame-pointer)
and --coverage (-O0 -g --coverage) -- all debug-oriented -- with the
default -O2 between them. Nothing pointed the other way, so shipping a
small library meant emitting the C and hand-compiling it, which is
exactly the hand-rolled script this is meant to delete.
--size compiles with -Oz -g0 and strips at link with
-Wl,--strip-all -Wl,--gc-sections.
THE ROOT CAUSE, which was not where the report guessed: `zig cc` emits
DWARF BY DEFAULT, even at -O2, and the cross backend passed no -g0.
Confirmed directly -- the same TU is 1,349 bytes at
`zig cc -target wasm32-wasi -O2` and 477 bytes with -g0 added. Nothing
in ae_cross.c was adding -g; the optimised path is a bare "-O2".
Measured on a two-function wasi library, 97.4% of the module was
.debug*/name sections and code+data were the remaining 2.6%. --size
takes it from 956,573 to 24,942 bytes -- 38x -- and the result is
behaviourally identical: it instantiates under node's WASI, exports
every symbol, and still traps correctly on the fail-stop panic path.
The equivalent native .so has ZERO .debug* sections, so this was a
cross-path problem rather than something every target shipped. Native
still gains ~14%, from -Oz and the symbol table.
Two deliberate limits:
- Not the default. A 38x difference is discoverable; anyone shipping to
a browser will find the flag. Stripping every build by default would
make the first "why can't I get a stack trace from my wasm module"
report hard to diagnose. Named modes keep the trade-off visible at
the point of choosing it.
- No link-time stripping for --emit=obj / --emit=csrc. Neither links,
and an object file's symbols are precisely what whoever links it next
needs. They still get -Oz -g0 for the compile.
--profile wins if both are passed: asking for a small artifact and a
profileable one is contradictory, and the debug-oriented reading is
safer. Same precedence shape --coverage already has over --profile.
Note the second flag path: tools/ae.c has a base_opt that bypasses
opt_flags() and carries its own "keep this in sync" comment. --size had
to be added to BOTH, or the mode would silently do nothing on the path
that build_gcc_cmd takes.
make test 394/0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The native path used -Oz, which gcc did not support until GCC 12. ubuntu-22.04 -- the CI baseline -- ships GCC 11, where it is a hard error, so `ae build --size` failed on every leg that runs the full suite: Linux GCC, Linux Clang, Linux Hardened, macOS ARM64. It passed locally because this box has GCC 12. -Os is supported by every gcc and clang we target and gives nearly the same result. The CROSS path keeps -Oz, and that is safe for a reason worth stating: zig bundles its own clang, so the version is not the host compiler's to vary. The cross artifact is unchanged at 24,942 bytes. Also stop the test swallowing the compiler's message. It ran the build with >/dev/null 2>&1 and reported only "--size build failed", which hid `unrecognized command line option '-Oz'` -- the one line that would have named the cause. It now keeps the log and prints the first 15 lines on failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
--strip-all and --gc-sections are GNU ld options. Apple's ld rejects
them outright:
ld: unknown options: --strip-all --gc-sections
clang: error: linker command failed with exit code 1
so `ae build --size` failed on macOS ARM64 -- the one leg still red after
the -Oz fix. The Mach-O equivalents are -x (strip local symbols; -S would
also drop debug info, which -g0 already prevents) and -dead_strip.
Split on both paths: the native one with the same #if defined(__APPLE__)
that harden_ldflags already uses, and the cross one with the existing
is_apple test, since cross-compiling TO macOS or iOS hits the same
linker. Everything else -- ELF, PE, wasm -- goes through an LLD that
takes the GNU spellings.
Verified aarch64-macos cross builds with and without --size (both OK;
an earlier failure on that target turned out to be a library-shaped
source built as an exe, unrelated and pre-existing). wasm --size
unchanged at 24,942 bytes. make test 394/0.
The captured build log added in the previous commit is what named this
one line: without it the failure was again just "--size build failed".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The check used `nm -D --defined-only`. BSD/macOS nm has no -D at all, so
on the mac legs it exited non-zero with EMPTY output -- which the test
read as "the symbols are gone" and reported as
[FAIL] size_mode: --emit=lib --size stripped the dynamic symbols
on a library that was perfectly fine. The build itself succeeded; only
the probe was broken.
Two changes:
- Pick the spelling by platform: `nm -gU` on Darwin (defined external
symbols), `nm -D --defined-only` elsewhere.
- Treat an EMPTY listing as inconclusive and skip, rather than as
proof of a strip. A probe that cannot run must not masquerade as a
regression -- that is precisely what cost this cycle.
The distinction is now explicit and tested both ways: a non-empty
listing missing the symbol still fails (verified with a stub nm that
reports unrelated symbols), while an unusable nm skips (verified with a
stub that always exits 1).
Worth recording why the original passed locally: after --strip-all the
STATIC symbol table is gone, so `nm -g` finds nothing, but the DYNAMIC
table survives and `nm -D` finds the exports. GNU nm made the right
answer available; BSD nm makes it available under a different name.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1729. From an ask by the html-sanitizer downstream, whose other half merged as #1728.
The root cause was not where we thought
Both the ask and #1729 said "something in the zig cross path is adding
-g— or not passing whatever suppresses it." It's the second, and it isn't ours:zig ccemits DWARF by default, even at-O2. Confirmed directly on one translation unit:Nothing in
ae_cross.cwas adding-g— the optimised path is a bare"-O2". We simply never passed-g0, and zig's default filled the gap.The fix
ae build --size:-Oz -g0at compile,-Wl,--strip-all -Wl,--gc-sectionsat link.--target=wasm32-wasi --emit=lib--size38×, and behaviourally identical — the 24 KB module instantiates under node's WASI, exports every symbol, and still traps correctly on the fail-stop panic path from #1728:
That lands almost exactly on the ask's hand-stripped figure, which is the sanity check I wanted: we're recovering what stripping recovers, not accidentally dropping something else.
Two deliberate limits
Not the default. A 38× difference is discoverable — anyone shipping to a browser will find the flag — whereas stripping every build by default would make the first "why can't I get a stack trace from my wasm module" report genuinely hard to diagnose. Named modes keep the trade-off visible at the point of choosing it. (This was the opinion I gave in #1729; happy to be overruled.)
No link-time stripping for
--emit=obj/--emit=csrc. Neither links, and an object file's symbols are precisely what whoever links it next needs. They still get-Oz -g0for the compile. Tested —nmstill finds the symbols in a--emit=obj --sizeoutput.--profilewins if both are passed: asking for a small artifact and a profileable one is contradictory, and the debug-oriented reading is safer. Same precedence shape--coveragealready has over--profile.A trap worth knowing about
tools/ae.chas a second optimisation-flag path —base_optat line 2530 — that bypassesopt_flags()and carries its own "Keep this in sync" comment.--sizehad to be added to both. Had I only doneopt_flags(), the mode would have silently done nothing on the pathbuild_gcc_cmdtakes, and the native half of the test would have passed anyway because the cross path is separate again.What I chose not to do
The ask offered
--cflags=/--ldflags=passthrough as an alternative. I went with the named mode because a passthrough makes the toolchain's flag surface part of our public contract by accident, and "what doesaepass by default" becomes something users must reverse-engineer to know what they're overriding. The mode is also discoverable inae build's usage output, which a passthrough isn't.Verification
make test: 394 passed, 0 failedtests/integration/size_mode/, asserting in rising order of what would actually break a user: the binary runs;--emit=libkeeps its dynamic symbols (a stripped library with none links fine and is useless);--emit=objkeeps its symbols; the wasm artifact is dramatically smaller and still a valid module with its exports intact. The size assertion is a conservative 4× against a measured 38×, so a partial regression — say-g0lost but stripping kept — still trips it.tests/ae_sweep_prune.txt(nomain), and the prune verified rather than assumed.make check-docsclean;--sizedocumented indocs/build-system.mdbeside--profile, and added toae build's usage text.🤖 Generated with Claude Code