feat: add TextEncoder/TextDecoder and atob/btoa on a lazy-global tier (#2026)

Author: edusperoni

docs/README.md

@@ -13,6 +13,10 @@
   layered on the runtime's `EventTarget`, the GC contract (weak timers and
   `any()` links, listener-driven persistence), and the `DOMException`
   stand-in (name-patched `Error` reasons).
+- [TextEncoder / TextDecoder and atob / btoa](text-encoding.md) — the WHATWG
+  encoding and base64 globals (`TextEncoder`, `TextDecoder`, `atob`, `btoa`),
+  the supported encodings with their label sets, streaming decode semantics,
+  and the lazy-global tier that runs their builtins only on first use.
 - [Error handling](error-handling.md) — global `error`/`unhandledrejection` events, `reportError`, catching Java exceptions in JS (`error.nativeException`), forwarding JS throws to Java callers (`interop.escapeException`), JS stacks on Java exceptions (`com.tns.JavaScriptStackTrace`), configuration flags, and crash-reporter integration.
 - [structuredClone](structured-clone.md) — the WHATWG `structuredClone(value, { transfer })` global: what clones, how graph identity and cycles are preserved, `ArrayBuffer` transfer, and the `DataCloneError`-named `Error` that stands in for `DOMException`.
 - [Implementing additional Chrome DevTools protocol Domains](extending-inspector.md)

docs/ns-builtin-modules.md

@@ -59,6 +59,7 @@ Rules:
 |---|---|
 | `inspect(value[, options])` | Formats any value for human consumption: depth-limited, output-capped, cycle-safe, never invokes getters (except a guarded `error.stack` read and custom `toString` overrides, which are honored). `options.depth` (number) overrides the default depth of 2. Other option keys are reserved. |
 | `format(fmt, ...args)` | Node-style printf formatting: `%s`, `%d`, `%i`, `%f`, `%j`, `%o`, `%O`, `%%`. Extra arguments are appended space-separated, objects rendered via `inspect`. When `fmt` is not a string or contains no substitutions, all arguments are formatted and joined with spaces. `console.*` routes its arguments through this, so `console.log("%d apples", 3)` works. |
+| `TextEncoder` / `TextDecoder` | The WHATWG encoding interfaces, **the very same class objects the globals of those names hold** (`require("ns:util").TextDecoder === globalThis.TextDecoder`). Reading either member is what materializes them, so requiring the module costs nothing extra. |
 
 ```js
 const { inspect, format } = require("ns:util");
@@ -78,6 +79,13 @@ format("%j", { ok: true }); // '{"ok":true}'
 format("100% sure", "extra"); // "100% sure extra"  (no placeholder consumed)
 ```
 
+```js
+const { TextEncoder, TextDecoder } = require("ns:util");
+
+TextDecoder === globalThis.TextDecoder; // true
+new TextDecoder().decode(new TextEncoder().encode("héllo")); // "héllo"
+```
+
 **Stability caveat (verbatim from Node's contract):** the output of `inspect`
 (and therefore `format`'s object rendering) may change between runtime
 versions for readability; it is intended for humans and must not be parsed
@@ -400,7 +408,7 @@ unmodified where a shim exists:
 
 | module | exports | notes |
 |---|---|---|
-| `node:util` | `inspect`, `format` | Re-exports `ns:util`'s members unchanged (`nodeUtil.inspect === nsUtil.inspect`) from a **distinct, separately frozen module object**. Documented as partial. |
+| `node:util` | `inspect`, `format`, `TextEncoder`, `TextDecoder` | Re-exports `ns:util`'s members unchanged (`nodeUtil.inspect === nsUtil.inspect`) from a **distinct, separately frozen module object**. `TextEncoder`/`TextDecoder` are the globals of those names, as they are in Node. Documented as partial. |
 | `node:url` | `fileURLToPath`, `pathToFileURL` | Node-strict converters between `file:` URLs and paths. Documented as partial — no `URL`/`URLSearchParams` re-exports (both are globals), no legacy `url.parse`/`format`/`resolve`. |
 | `node:module` | `createRequire` | Re-exports `ns:module`'s `createRequire` unchanged from a **distinct, separately frozen module object**. `createPumpingRequire` is deliberately absent: it has no Node counterpart, so code written against this shim keeps running on Node. `require.resolve`/`.cache`/`.main` are not implemented, and neither is any other `node:module` member (`Module`, `builtinModules`, `isBuiltin`, `register`, `syncBuiltinESMExports`). Documented as partial. |
 

docs/text-encoding.md

@@ -0,0 +1,70 @@
+# TextEncoder / TextDecoder and atob / btoa
+
+Native, WHATWG-conformant `TextEncoder`, `TextDecoder`
+([Encoding Standard](https://encoding.spec.whatwg.org)) and `atob` / `btoa`
+([HTML Standard §8.3](https://html.spec.whatwg.org/multipage/webappapis.html#atob))
+globals, and the **lazy-global tier** they ride on.
+
+## Lazy globals
+
+These globals are registered on the global template as lazy data properties
+(`LazyGlobals`, `test-app/runtime/src/main/cpp/LazyGlobals.cpp`): the builtin
+behind a name is not compiled, run, or allocated until app code first reads it,
+and V8 then replaces the property with a plain data property so later reads
+cost nothing. Sibling names from one builtin (`TextEncoder` + `TextDecoder`)
+share a single run per isolate. Workers get the same globals — the tier is
+registered in every isolate's template. Assigning over one of these names
+before its first read replaces the global, like any other writable global.
+
+The tier is the intended home for further web globals (`Blob`, `fetch`,
+`crypto`, `DOMException`, …) with zero cost when unused; see
+`test-app/runtime/src/main/cpp/js/README.md` for the rules a lazy builtin
+lives by.
+
+The per-isolate exports cache behind the tier (`BuiltinLoader::GetExports`) is
+shared with the `ns:`/`node:` module registry: `require("ns:util").TextDecoder`
+and `require("node:util").TextDecoder` are the very class objects the globals
+hold, whichever entry point is reached first
+(see [ns-builtin-modules](ns-builtin-modules.md)).
+
+## TextEncoder / TextDecoder
+
+Node's split: `js/text-encoding.js` owns the WebIDL surface (brand checks via
+private fields, enumerable prototype members, `Symbol.toStringTag`),
+`TextEncoding.cpp` owns the bytes.
+
+- **Decoder encodings**: the `TextDecoder` constructor resolves utf-8,
+  utf-16le, utf-16be and windows-1252, each with its complete WHATWG label
+  set; an unknown label throws `RangeError`. (Precedent: Node without ICU
+  ships utf-8/utf-16le; utf-16be and windows-1252 are cheap, and windows-1252
+  covers the `ascii`/`latin1`/`iso-8859-1` aliases web code actually uses.)
+  `TextEncoder` is UTF-8-only and takes no label, as the spec defines it.
+- **Streaming**: full `decode(…, { stream: true })` support. Incomplete
+  sequences (split BOMs and split utf-16 code units included) carry across
+  calls in a 16-byte `Uint8Array` the builtin owns — no per-instance native
+  handle, no finalizer.
+- **Replacement semantics**: WHATWG utf-8 state machine with one U+FFFD per
+  maximal invalid subpart; `fatal: true` throws `TypeError`; `ignoreBOM`
+  honored.
+- `encode()` / `encodeInto()` with correct USV conversion and partial-write
+  boundaries (`encodeInto` never splits an encoded code point).
+- **Fast paths**: pure-ASCII utf-8 and C1-free windows-1252 decode straight
+  through `String::NewFromOneByte`; results downgrade to one-byte strings when
+  possible. `encodeInto` registers a V8 Fast API overload
+  (`NATIVESCRIPT_ENABLE_FAST_API`, default on), live once a call site tiers
+  up.
+
+## atob / btoa
+
+WHATWG forgiving-base64 (`Base64.cpp`): whitespace stripping, padding rules,
+alphabet validation. With no `DOMException` in the runtime yet, failures throw
+the name-patched `Error` (`InvalidCharacterError`) stand-in the abort-signal
+and performance builtins already use; a follow-up will introduce
+`DOMException` and upgrade these.
+
+## Tests
+
+The shared suite (`test-app/app/src/main/assets/app/shared/TextEncoding`)
+holds the conformance specs, feature-detecting so runtimes without these
+globals report pending rather than failing; it was independently validated
+against Node 24 (full ICU) as a reference.

eslint.config.mjs

@@ -32,7 +32,7 @@ const capturedStatics = [
 
 // Captured constructors. A destructure from `primordials` shadows the global,
 // so these only fire on the unguarded reference.
-const restrictedGlobals = ['Date', 'FinalizationRegistry', 'Map', 'Number', 'Proxy', 'RangeError', 'Set', 'String', 'TypeError', 'WeakRef'].map((name) => ({
+const restrictedGlobals = ['Date', 'FinalizationRegistry', 'Map', 'Number', 'Proxy', 'RangeError', 'Set', 'String', 'TypeError', 'Uint8Array', 'Uint32Array', 'WeakRef'].map((name) => ({
   name,
   message: `Destructure ${name} from primordials — builtins must not read intrinsics off globals user code can replace.`,
 }));

test-app/app/src/main/assets/app/mainpage.js

@@ -20,6 +20,7 @@ shared.runRuntimeTests();
 shared.runWorkerTests();
 shared.runPerformanceTests();
 shared.runStructuredCloneTests();
+shared.runTextEncodingTests();
 require("./tests/testWebAssembly");
 require("./tests/testEventLoop");
 require("./tests/testMultithreadedJavascript");

test-app/app/src/main/assets/app/shared

@@ -0,0 +1,28 @@
+// A worker is a fresh isolate, which is what makes the access order testable:
+// the parent realm has already materialized TextEncoder/TextDecoder by the
+// time any spec runs. Nothing here may touch either name before the handler,
+// or the requested order is lost.
+onmessage = function (msg) {
+    var order = msg.data;
+    var results = { order: order };
+
+    if (order === "global-first") {
+        var globalEncoder = globalThis.TextEncoder;
+        var globalDecoder = globalThis.TextDecoder;
+        var nsUtil = require("ns:util");
+        var nodeUtil = require("node:util");
+        results.encoder = nsUtil.TextEncoder === globalEncoder && nodeUtil.TextEncoder === globalEncoder;
+        results.decoder = nsUtil.TextDecoder === globalDecoder && nodeUtil.TextDecoder === globalDecoder;
+        results.roundTrip = new nsUtil.TextDecoder().decode(new nodeUtil.TextEncoder().encode("ok"));
+    } else {
+        var util = require("ns:util");
+        var node = require("node:util");
+        var utilEncoder = util.TextEncoder;
+        var utilDecoder = node.TextDecoder;
+        results.encoder = globalThis.TextEncoder === utilEncoder && node.TextEncoder === utilEncoder;
+        results.decoder = globalThis.TextDecoder === utilDecoder && util.TextDecoder === utilDecoder;
+        results.roundTrip = new node.TextDecoder().decode(new util.TextEncoder().encode("ok"));
+    }
+
+    postMessage(results);
+};

test-app/app/src/main/assets/app/tests/nsUtilEncodingOrderWorker.js

@@ -12,6 +12,49 @@ describe("ns:util", function () {
         expect(require("ns:util")).toBe(util);
     });
 
+    it("exposes the encoding interfaces the globals expose", function () {
+        expect(typeof util.TextEncoder).toBe("function");
+        expect(typeof util.TextDecoder).toBe("function");
+        // One run of the text-encoding builtin backs both entry points, so the
+        // classes are identical objects no matter which is reached first.
+        expect(util.TextEncoder).toBe(globalThis.TextEncoder);
+        expect(util.TextDecoder).toBe(globalThis.TextDecoder);
+    });
+
+    it("round trips text through the module's encoding interfaces", function () {
+        var bytes = new util.TextEncoder().encode("héllo");
+        expect(bytes instanceof Uint8Array).toBe(true);
+        expect(bytes.length).toBe(6);
+        expect(new util.TextDecoder().decode(bytes)).toBe("héllo");
+    });
+
+    it("keeps the classes identical in a fresh isolate, whichever is touched first", function (done) {
+        var orders = ["global-first", "util-first"];
+        var replies = 0;
+        orders.forEach(function (order) {
+            var worker = new Worker("./nsUtilEncodingOrderWorker.js");
+            worker.onmessage = function (msg) {
+                expect(msg.data).toEqual({
+                    order: order,
+                    encoder: true,
+                    decoder: true,
+                    roundTrip: "ok",
+                });
+                worker.terminate();
+                replies++;
+                if (replies === orders.length) {
+                    done();
+                }
+            };
+            worker.onerror = function (error) {
+                fail("worker (" + order + ") failed: " + error.message);
+                worker.terminate();
+                done();
+            };
+            worker.postMessage(order);
+        });
+    });
+
     it("throws for an unknown builtin", function () {
         expect(function () {
             require("ns:definitely-not-a-module");
@@ -158,6 +201,15 @@ describe("node:util", function () {
         expect(Object.isFrozen(nodeUtil)).toBe(true);
         expect(nodeUtil.inspect).toBe(util.inspect);
         expect(nodeUtil.format).toBe(util.format);
+        expect(nodeUtil.TextEncoder).toBe(util.TextEncoder);
+        expect(nodeUtil.TextDecoder).toBe(util.TextDecoder);
+    });
+
+    it("exposes Node's encoding interfaces, identical to the globals", function () {
+        expect(Object.keys(nodeUtil).sort()).toEqual(["TextDecoder", "TextEncoder", "format", "inspect"]);
+        expect(nodeUtil.TextEncoder).toBe(globalThis.TextEncoder);
+        expect(nodeUtil.TextDecoder).toBe(globalThis.TextDecoder);
+        expect(new nodeUtil.TextDecoder("utf-8").decode(new nodeUtil.TextEncoder().encode("ok"))).toBe("ok");
     });
 
     it("is a singleton per realm", function () {

test-app/app/src/main/assets/app/tests/testNsUtil.js

@@ -69,6 +69,7 @@ include_directories(
 set(RUNTIME_BUILTIN_JS_DIR ${PROJECT_SOURCE_DIR}/src/main/cpp/js)
 set(RUNTIME_BUILTIN_JS
         ${RUNTIME_BUILTIN_JS_DIR}/abort-signal.js
+        ${RUNTIME_BUILTIN_JS_DIR}/base64.js
         ${RUNTIME_BUILTIN_JS_DIR}/blob-url.js
         ${RUNTIME_BUILTIN_JS_DIR}/error-events.js
         ${RUNTIME_BUILTIN_JS_DIR}/events.js
@@ -84,6 +85,7 @@ set(RUNTIME_BUILTIN_JS
         ${RUNTIME_BUILTIN_JS_DIR}/primordials.js
         ${RUNTIME_BUILTIN_JS_DIR}/require-factory.js
         ${RUNTIME_BUILTIN_JS_DIR}/structured-clone.js
+        ${RUNTIME_BUILTIN_JS_DIR}/text-encoding.js
         ${RUNTIME_BUILTIN_JS_DIR}/weak-ref.js
 )
 set(RUNTIME_BUILTINS_GENERATED_DIR ${PROJECT_SOURCE_DIR}/src/main/cpp/generated)
@@ -168,6 +170,7 @@ add_library(
         src/main/cpp/ArrayElementAccessor.cpp
         src/main/cpp/ArrayHelper.cpp
         src/main/cpp/AssetExtractor.cpp
+        src/main/cpp/Base64.cpp
         src/main/cpp/BuiltinLoader.cpp
         src/main/cpp/CallbackHandlers.cpp
         src/main/cpp/ConcurrentQueue.cpp
@@ -189,6 +192,7 @@ add_library(
         src/main/cpp/JsArgConverter.cpp
         src/main/cpp/JsArgToArrayConverter.cpp
         src/main/cpp/JSONObjectHelper.cpp
+        src/main/cpp/LazyGlobals.cpp
         src/main/cpp/Logger.cpp
         src/main/cpp/ManualInstrumentation.cpp
         src/main/cpp/MetadataMethodInfo.cpp
@@ -215,6 +219,7 @@ add_library(
         src/main/cpp/SimpleProfiler.cpp
         src/main/cpp/StructuredClone.cpp
         src/main/cpp/StructuredSerialization.cpp
+        src/main/cpp/TextEncoding.cpp
         src/main/cpp/Util.cpp
         src/main/cpp/V8GlobalHelpers.cpp
         src/main/cpp/V8StringConstants.cpp