diff --git a/frameworks/morojs/Dockerfile b/frameworks/morojs/Dockerfile index 8898ceefb5..56bb513d7a 100644 --- a/frameworks/morojs/Dockerfile +++ b/frameworks/morojs/Dockerfile @@ -10,11 +10,14 @@ COPY --from=build /app/node_modules ./node_modules COPY app.mjs . COPY views ./views ENV NODE_ENV=production -# The libuv threadpool carries the compression and file reads of every worker -# thread in the process. Moro sizes it to 64 itself, but only at listen(), -# after its startup has already initialised the pool at Node's default of 4; -# an environment variable at process start is what Node reads. 64 matches the -# benchmark cpuset. +# One libuv threadpool serves every worker thread in the process. Little rides +# on it now - compression runs inline on the worker and static files are +# served from the middleware's memory cache, which stats the disk once a +# second at most - but what does (the cache's revalidation, the first read of +# each file, DNS) should not queue behind Node's default of four threads. Moro +# sizes the pool to 64 itself, but only at listen(), after its startup has +# already initialised it; an environment variable at process start is what +# Node reads. 64 matches the benchmark cpuset. ENV UV_THREADPOOL_SIZE=64 EXPOSE 8080 CMD ["node", "app.mjs"] diff --git a/frameworks/morojs/README.md b/frameworks/morojs/README.md index 4629694f5b..bdae43da75 100644 --- a/frameworks/morojs/README.md +++ b/frameworks/morojs/README.md @@ -6,7 +6,7 @@ MoroJS on its native HTTP engine, clustered by the framework itself. - **Language:** JavaScript - **Runtime:** Node.js 26 -- **Framework:** [MoroJS 1.8](https://github.com/Moro-JS/moro) on `@morojs/engine`, its own native HTTP engine +- **Framework:** [MoroJS 1.8](https://github.com/Moro-JS/moro) on `@morojs/engine` 1.1.16, its own native HTTP engine - **Build:** Multi-stage on `node:26-trixie-slim`; the engine ships prebuilt binaries, so no toolchain ## Endpoints @@ -16,12 +16,12 @@ MoroJS on its native HTTP engine, clustered by the framework itself. | `/pipeline` | GET | Returns `ok` (plain text) | | `/baseline11` | GET/POST | Sums query parameter values, plus the body for POST | | `/baseline2` | GET | Sums query parameter values (behind the gateway proxies) | -| `/delay/:ms` | GET | Waits `ms` milliseconds on an awaited timer, then returns `ms` | -| `/json/:count` | GET | Serializes a slice of the dataset, brotli or gzip when the client asks for it | +| `/delay/:ms` | GET | Answers `ms` from a timer callback `ms` milliseconds later | +| `/json/:count` | GET | Serializes a slice of the dataset, brotli (quality 3) or gzip (level 3) when the client asks for it | | `/echo` | POST | Returns the request body back verbatim (TLS listener) | | `/ws` | WebSocket | Echoes every text and binary frame unchanged | -| `/async-db` | GET | Reads from PostgreSQL through `pg`, prepared statement, pool sized under `DATABASE_MAX_CONN` | -| `/fortunes` | GET | Reads 200 rows from PostgreSQL, appends the runtime row, sorts and renders `views/fortunes.ejs` | +| `/async-db` | GET | Reads from PostgreSQL through `pg`, prepared statement, pool sized under `DATABASE_MAX_CONN` (240 at most), the driver's callback answering the request | +| `/fortunes` | GET | Reads the fortune rows from PostgreSQL, appends the runtime row, sorts and renders `views/fortunes.ejs`, compiled once per worker | | `/static/:filename` | GET | Serves a file from disk through the framework's `staticFiles` middleware (TLS listener) | | `/public/baseline`, `/public/json/:count` | GET | The baseline and JSON routes as the production-stack edge forwards them, no auth | | `/api/items/:id` | GET/POST | Cache-aside read with `X-Cache: HIT` or `MISS`, or a JSON-body update that invalidates the entry and answers 204 | @@ -40,10 +40,27 @@ MoroJS on its native HTTP engine, clustered by the framework itself. `--cpuset-cpus`. - `json-comp` is the framework's `compression()` middleware attached to the `/json` route alone, so no other endpoint pays for the encoder. It negotiates off `Accept-Encoding` per request and - leaves the body alone when the header is absent. gzip is preferred over brotli through the - middleware's `encodings` option: at the default level it costs about half the CPU of brotli at - the default quality for a body a tenth larger, and the profile scores bytes squared against - rate; brotli remains available to a client that accepts nothing else. + leaves the body alone when the header is absent. brotli at quality 3 is preferred, through the + middleware's `encodings` and `brotli` options, with the encoder's window fitted to the body by + the framework: the profile scores bytes squared against rate, and measured on the board gzip at + level 3 ran 7% more requests than brotli 3 but sent 10% more bytes per reply, a 9% lower score. + gzip at level 3 remains available to a client that accepts nothing else. The middleware keeps one + encoder per setting per worker and resets it between bodies (`reuseEncoders`, the default): on the + Node 26 the image ships, brotli's reset keeps its quality, so a reply costs the compression alone + and not an encoder built and torn down around it (measured on these bodies: 30 us against 45 or + more per body). +- The proxied stacks (`compose.gateway.yml`, `compose.gateway-h3.yml`, `compose.production-stack.yml`) + run the server with `MORO_ENGINE_SERVE_ORDER=auto`: the engine's epoll transport serves a loop + turn's sockets grouped by the CPU their peer sends from, an order its design notes measured to lose + about a quarter at 256 connections against arrival order when every request enters JavaScript under + a peer that oversubscribes its cores, which is what an edge proxy is. `auto` has each loop measure + both orders and keep the faster one. The standalone container keeps the engine's default. +- `server.backlog: 16384`: the edges in front of the gateway and production-stack profiles keep an + idle pool of 2048 upstream connections under tens of thousands of requests in flight, so they + dial and drop thousands of connections per wave. At the engine's default backlog of 512 a wave + overflows a worker's accept queue, the kernel drops the SYN and the proxy retries it a second + later, which was those profiles' tail latency. Measured on a 4096-dialer churn: p99 1019 ms at + 512, 52 ms at 4096, same throughput. The harness raises `somaxconn` to 65535 for this. - `UV_THREADPOOL_SIZE=64` is set in the Dockerfile. The libuv pool serves the compression and file reads of every worker thread in the process; Moro sets 64 itself but only at `listen()`, after its startup has initialised the pool at Node's default of 4, and Node reads the variable @@ -57,16 +74,38 @@ MoroJS on its native HTTP engine, clustered by the framework itself. validated and swapped onto the running listener without a restart and without touching connections already established. Only started when `/certs-tls` is mounted, which validate.sh alone does. +- `baseline-h2`, `static-h2`, `baseline-h2c` and `json-h2c` run on Moro's HTTP/2 server, which is + Node's http2 module behind the framework's own request and response objects (`engine: 'node'` + with `http2: true`); the native engine speaks HTTP/1.1 only. `:8443` terminates TLS with ALPN + `h2` (HTTP/1.1 still accepted), `:8082` is cleartext prior-knowledge HTTP/2. Each is its own + process for the same one-configuration-per-process reason, clustered as node:cluster processes, + which is Moro's transport for its Node servers. - `/echo` sends back `req.rawBody`, the framework's undecoded copy of the request body, so the - bytes are the bytes that arrived, `Content-Length` or chunked alike. + bytes are the bytes that arrived, `Content-Length` or chunked alike; a Buffer handed to + `res.send()` goes out `application/octet-stream` through the framework's prepared reply. - `/ws` is the engine's own RFC 6455 support through `app.websocket()` with `{ raw: true }`, so a text frame reaches the handler as the string it carried and `socket.send()` returns it as one. -- Static files go through `staticFiles({ root: '/data/static', precompressed: true })`, which - stats and reads the file on every request, so a replaced file is served on the next response, - and serves the `.br` or `.gz` sidecar on disk when the client accepts it. It is mounted on the - TLS listener behind a path check so the JSON route on that port does not pay for the lookup. -- `fortunes` goes through EJS directly: Moro ships no view layer, so the template is rendered by - the engine's own `renderFile`, a separate file with `<%= %>` escaping every cell. + The engine corks the connection for the duration of each read (`@morojs/engine` 1.1.14), so + the echoes of a pipelined batch of frames leave in one write, each frame built straight into + that write (1.1.15). The upgrade itself is the engine's as well (1.1.15, MoroJS 1.8.17): a valid + handshake is answered and the socket opened without the request ever entering JavaScript, and + the handshake's headers, query and peer address stay readable on the socket on demand. +- Static files go through `staticFiles({ root: '/data/static', precompressed: true, cache: true })`: + the framework's handler with its in-memory cache. A file and its `.br` and `.gz` sidecars are + read once and served from memory with their `ETag` and `Last-Modified`, the sidecar negotiated + per request from `Accept-Encoding`. The disk is stat'ed again at most once a second, and as soon + as the directory reports a change, so a replaced file (same length or not) is served promptly. + Without the cache the handler resolves, stats and reads the file and its sidecars on every + request, each a libuv threadpool round trip shared by every worker thread of the process. It is + mounted on the TLS listener behind a path check so the JSON route on that port does not pay for + the lookup; the check calls `next()` synchronously, which keeps that route on the server's + synchronous dispatch. +- `fortunes` goes through EJS directly: Moro ships no view layer, so the template file is compiled + once per worker with `ejs.compile` and rendered per request, `<%= %>` escaping every cell. +- `/async-db` and `/fortunes` answer from the driver's callback rather than from an async handler: + the reply is the same, and the request stops paying three promise hops (the handler's, the + router's, the server's). `pg-native`, which the uWebSockets-based entries use, loads its libpq + addon once per process and refuses the second worker thread, so it cannot serve a thread cluster. - The gateway and production-stack edges are stock nginx and Caddy, configured as the fulmine entry configures them: `/static/*` off disk at the edge, everything else forwarded over loopback h1 with a keepalive pool, and `/api/*` past the shared JWT verifier first. diff --git a/frameworks/morojs/app.mjs b/frameworks/morojs/app.mjs index ddfdaf86cf..26f0440ffb 100644 --- a/frameworks/morojs/app.mjs +++ b/frameworks/morojs/app.mjs @@ -11,6 +11,7 @@ import { availableParallelism } from 'node:os'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; import { isMainThread } from 'node:worker_threads'; +import { constants as zlibConstants } from 'node:zlib'; import { createApp, middleware, MemoryCacheAdapter, RedisCacheAdapter } from '@morojs/moro'; import ejs from 'ejs'; import pg from 'pg'; @@ -18,17 +19,28 @@ import pg from 'pg'; const HERE = dirname(fileURLToPath(import.meta.url)); const FORTUNES_VIEW = join(HERE, 'views', 'fortunes.ejs'); const DATASET_PATH = process.env.DATASET_PATH || '/data/dataset.json'; -const STATIC_ROOT = '/data/static'; +const STATIC_ROOT = process.env.STATIC_ROOT || '/data/static'; +// The harness mounts the pair at /certs; the override is for running the +// entry outside the container. +const CERTS_DIR = process.env.CERTS_DIR || '/certs'; // One process per listener, because Moro locks one configuration per // process. 'plain' serves :8080; 'tls' serves json-tls, static-tls and 8gbit // on :8081 from /certs; 'tlscheck' serves the opt-in TLS hardening section on -// :9000 from /certs-tls, reloading the pair whenever the files change. +// :9000 from /certs-tls, reloading the pair whenever the files change. The +// two HTTP/2 listeners run on Moro's own HTTP/2 server (Node's http2 module, +// `engine: 'node'` with `http2: true`): the native engine speaks HTTP/1.1 +// only. 'h2' is TLS with ALPN h2 on :8443 for baseline-h2 and static-h2; +// 'h2c' is cleartext prior-knowledge HTTP/2 on :8082 for baseline-h2c and +// json-h2c. Both cluster as node:cluster processes, Moro's transport for the +// Node servers. const ROLE = process.env.MORO_ROLE || 'plain'; const LISTENERS = { plain: { port: 8080 }, - tls: { port: 8081, certs: '/certs' }, + tls: { port: 8081, certs: CERTS_DIR }, tlscheck: { port: 9000, certs: '/certs-tls', watch: true }, + h2: { port: 8443, certs: CERTS_DIR, http2: true }, + h2c: { port: 8082, http2: true }, }; const pairIn = dir => ({ keyFile: join(dir, 'server.key'), certFile: join(dir, 'server.crt') }); const hasPair = dir => existsSync(pairIn(dir).keyFile) && existsSync(pairIn(dir).certFile); @@ -47,7 +59,8 @@ function cpuCount() { } catch {} return availableParallelism(); } -const workers = cpuCount(); +// WORKERS overrides the count, for running the entry outside the container. +const workers = parseInt(process.env.WORKERS, 10) || cpuCount(); // A worker thread on the native engine, a node:cluster process on the // fallback transport. The primary serves nothing, so it loads no dataset and @@ -55,11 +68,12 @@ const workers = cpuCount(); const isWorker = !isMainThread || cluster.isWorker; const isPrimary = !isWorker; -// The harness mounts /certs for the TLS profiles and /certs-tls for the -// tls_check section only; a listener whose pair is absent is not started. +// The harness mounts /certs for the TLS and HTTP/2 profiles and /certs-tls +// for the tls_check section only; a listener whose pair is absent is not +// started. h2c needs no material and always starts. if (ROLE === 'plain' && isPrimary) { - for (const role of ['tls', 'tlscheck']) { - if (!hasPair(LISTENERS[role].certs)) continue; + for (const role of ['tls', 'tlscheck', 'h2', 'h2c']) { + if (LISTENERS[role].certs && !hasPair(LISTENERS[role].certs)) continue; const child = spawn(process.execPath, [fileURLToPath(import.meta.url)], { stdio: 'inherit', env: { ...process.env, MORO_ROLE: role }, @@ -84,10 +98,12 @@ if (isWorker) { // Postgres for /async-db, /fortunes and the production-stack api, all served // on :8080. The pool is sized from DATABASE_MAX_CONN across the workers, so -// the cluster as a whole stays inside what Postgres allows. +// the cluster as a whole stays inside what Postgres allows; 240 at most +// leaves the server's own reserve (DATABASE_MAX_CONN is its max_connections, +// and a cluster sitting exactly on it is refused its last connections). let pool = null; if (isWorker && ROLE === 'plain' && process.env.DATABASE_URL) { - const maxConn = parseInt(process.env.DATABASE_MAX_CONN, 10) || 256; + const maxConn = Math.min(parseInt(process.env.DATABASE_MAX_CONN, 10) || 256, 240); pool = new pg.Pool({ connectionString: process.env.DATABASE_URL, max: Math.max(1, Math.floor(maxConn / workers)), @@ -95,6 +111,12 @@ if (isWorker && ROLE === 'plain' && process.env.DATABASE_URL) { pool.on('error', () => {}); } +// The fortunes template, compiled once per worker: EJS escapes every +// interpolated value, so the row carrying a