Skip to content

ecs: order on System.render, typed input, importer's copy of the runtime, faster first tick (0.1.1) - #222

Open
krisnye wants to merge 127 commits into
mainfrom
knye/ecs-load-tick
Open

krisnye wants to merge 127 commits into
mainfrom
knye/ecs-load-tick

Conversation

@krisnye

@krisnye krisnye commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Combines #220, #221, the first-tick work and typed input into one PR, rebased on main. Sets @adobe/ecs to 0.1.1.

Contract change, for review

  • order on System.render: order?: Index<{ sorted: true; key: Component.Key[] }>; with it tables is one table of all matching rows in index order. GPU kernels reject order.
  • The plugin is Compiler.plugin(), the one export of @adobe/ecs/compiler; Compiler.plugin({ library: true }) builds a library's registrations and input schemas, leaving kernels as source.
  • Declarations hold only what bodies can't show. components: { with, optional, not, added, changed } (with replaces read, write, has; optional keys never narrow matching and read undefined where absent); resources: { changed, is, not }; lookups { with, optional, not, changed } by entity or { index, changed }; up: { through }. The build infers reads, writes and reductions from bodies; a key used but not required fails the build. No name anywhere; labels are bindings, and failures carry id.
  • A system's input is the type of its body's parameter after its context; the build reads its schema.
  • Keys are declared once, in the augmentation beside their Symbol(); there is no public registration. Precision and ranges come from named types (Schema.Type<S>; F32, I32, U32, the vector types). Starts: a literal default, the type's readonly T.default, a reduction's identity, null, zero values, or a fresh ReadonlyMap/ReadonlySet; schemas hold no defaults. Invalid declarations error once at each World.create call.
  • Archetypes are as const lists; Archetype.create is gone and the build indexes lists.
  • kernel?: boolean | "gpu". Omitted compiles eligible row systems (f64x2 lanes, results matching the body as written); true asserts and keeps f32x4 lanes; false runs as written. Async and generator bodies fail the build.
  • Ids are name@version/path#binding; a module outside a named package fails the build.
  • Joined finalizers read every resource as of before their stops' sync point applied.

Fixes

  • Joined finalizers read every resource as of before their stops' sync point applied. Object resources come from the thread's mirror; strings and linear values come from copies in the stop list. Before, an object resource could read as undefined after joining or a move, and a later write could leak into a finalizer's read.

  • Two or more string resources in one joined finalizer no longer crash the thread on a misaligned view.

  • Joined-thread races: a task's rebuild is retried only for its own reason, after a turn of the event loop; copyRest counts snapshots atomically; a pool worker checks for posts again before leaving a closed window; a lent worker's owner ending it is reported as terminated by its owner.

  • Plugin tests install a copy of the package built from its sources, so they no longer depend on dist.

  • Compiled code imports the runtime from the importer's copy of the package. The kernel plugin finds a module's @adobe/ecs copy from its nearest package.json named @adobe/ecs, so two installed copies no longer mix runtimes.

Performance

  • up trees and orders are built in bulk after a load.

  • A sorted index rebuild skips its sort when entries are already in key order.

  • An ordered run binds a row in place when its archetype changes.

  • First tick in the editor load tick 50K benchmark: 32 → 20 ms. In a 50K board app, generate to first frame went from about 355 to about 325 ms. Ordered render frames are unchanged: 0.030–0.036 ms when nothing changes, 0.27–0.29 ms with one key changed at 50K.

  • Hash index buckets are left unsorted by a change and sorted before anything reads them by position: moving one entity between two 50K buckets 418 → 5.6 µs, reading in order after it 416 → 69 µs. Lookups still give rows in entity order.

  • Block ticks, one per 64 rows, so change filters skip a changed chunk's unchanged rows: eight systems filtering on one changed row of 100K 34.9 → 5.0 µs, every row changed 8.5 → 4.7 ms; writes within noise.

  • A world's first 256 runs spawn their reports through one reused buffer; benchmarks timing a run after another warm up past them.

  • Joined index lookups check whether an index was replaced once per run instead of once per row: joined observer 10K 10–13% faster.

Docs

  • The README opens with a summary and an outline of the features. The guide is complete through section 19.
  • A generated API reference (typedoc) publishes to https://adobe.github.io/data/ecs/. The Pages workflow no longer names packages: each declares a pages field and scripts/collect-pages.mjs assembles the site; CI runs pnpm -r run docs.

Checks

packages/ecs: build and 1,405 tests pass, in node, threads, chromium and a project running the row-path suites with bodies as written.

🤖 Generated with Claude Code

krisnye and others added 7 commits October 22, 2026 20:31
Contract change (authorized): System.render's declaration takes
`order?: Index<{ readonly sorted: true; readonly key: readonly Component.Key[] }>`.
With it, the body's `tables` holds exactly one table whose `count` is every
matching row across archetypes (has/not apply) and whose `bind` makes instance
i the i-th of them in the index's order. Shader accessors keep their names and
`fn name(row: u32)` signatures; `row` is the position in the order and
`ecs_count()` the instance count. Gpu.wgsl emits, for an ordered system, the
record `struct EcsTable { ecs_rows, ecs_order, ecs_tables, ecs_shift }` and an
`ecs_address` helper; unordered output is unchanged. A non-sorted or unknown
index is a type error and throws at creation; a run whose matching archetype
holds more rows than 32 bits name beside its slot fails in its report.

Implementation: an order block of one u32 per instance (slot above the
record's shift, row below) and a tables block of each slot's column word
addresses, both heap blocks through uploadIndexes. The order is listed and
written only when the index's `changed` tick or a new `moved` tick (an entry
relocated with its key unchanged) moves, or the matches grow; the tables block
only when an address changes; the record only when a word does. An unchanged
run writes nothing and allocates nothing.

Numbers, chromium: gpu ordered render unchanged 50K 0.030-0.036 ms a run
(unordered two tables 0.045-0.049); gpu ordered render one key 50K
0.268-0.294 ms (listing 50K entries 0.17 ms, 200 KB write 0.07 ms).

Tests: render-order.browser.test.ts (two archetypes interleaved in depth,
has/not, despawn row move, spawn, key change, a hidden entity shown, one
table when nothing matches, writes only on change, ordered WGSL compiles),
system.type-test.ts (sorted, hash, unknown and non-index orders), a creation
check in create.test.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e package

The kernel plugin built the runtime hook paths and the namespace module sets from its
own import.meta.url, so a module importing a different copy of @adobe/ecs went
unrecognized or bound to the plugin's copy. Copies are now found from the files an
importer resolves into, by the nearest package.json named @adobe/ecs, cached per
directory; the plugin's own copy keeps its own layout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New benchmark `editor load tick 50K`: a 2D editor's first tick after
spawning a 50,201-node tree, with two filtered `up` row systems, a hash
index on the parent and a sorted paint-order index. 32 ms before.

- A filtered run after its tree was built again runs the whole order,
  rather than marking every row and scanning for the marks: the tick
  32 -> 25.1 ms.
- The tree's build reads each archetype's parent column once: the
  opacity `up` 5.35 -> 4.5 ms.
- The build sets every level in one walk per entity: the opacity `up`
  4.77 -> 4.49 ms.
- An order build's scratch is shared by every `up` of a thread: the tick
  20.6 -> 20.0 ms.

`up load` 222 -> 184 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A load in document order lists a paint-order index's entries in key
order already; one pass that stops at the first pair out of order checks
it. `editor load tick 50K`'s paint system 5.6-5.8 -> 5.1-5.3 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A main-thread system run in an index's order looked its shared row up
again whenever the archetype changed, at every row when archetypes
interleave in paint order. `editor load tick 50K`'s paint system
4.85 -> 4.5 ms.

perf.md records the editor load work: `editor load tick 50K`
31.4-33.2 -> 20.0-20.4 ms and `up load` 221-223 -> 182-186 ms against
826c45a, alternating, with the attempts not kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@krisnye
krisnye force-pushed the knye/ecs-load-tick branch from 6d49b50 to 38b10b2 Compare October 23, 2026 03:33
@krisnye krisnye changed the title ecs: faster first tick after a bulk load ecs: order on System.render, importer's copy of the runtime, faster first tick (0.1.1) Oct 23, 2026
@krisnye
krisnye changed the base branch from knye/render-order to main October 23, 2026 03:33
krisnye and others added 4 commits October 22, 2026 20:40
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontext

The declaration's input schema goes away: the plugin reads the schema the runtime needs from the parameter's
TypeScript type, before types are stripped, and passes it to create. Checks on the input move to the body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@krisnye krisnye changed the title ecs: order on System.render, importer's copy of the runtime, faster first tick (0.1.1) ecs: order on System.render, typed input, importer's copy of the runtime, faster first tick (0.1.1) Oct 23, 2026
krisnye and others added 12 commits October 22, 2026 23:08
…obal class

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s full count of reports

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…up past the kept reports

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… index, then reading lengths or rows in order

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e anything reads them by position

A change to a bucket of at least 64 entries of an index that isn't unique leaves a hole or
joins a tail instead of shifting the bucket, so moving an entity between two long buckets is
O(log bucket) to find and O(1) amortized to move. Lists sort their bucket on the world's
thread at the first `at`; kernels, workers and joined threads get every bucket sorted before
their parameters are written. Long moves copy 8-byte words over shared memory.

bucket move 100K length 418 -> 5.6 us, in order 416 -> 69 us.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cate nothing, and a hole first in a bucket leaves rows to be made ahead later

A bucket sorted and left unsorted again while another stayed unsorted was listed once more
each time, and every listing then compacted the whole list. A move by an odd number of words
goes through the scratch's views only from 1,024 words. growPool no longer marks a list's rows
made ahead when its first entry is a hole. Only writeIndexParameters sorts for a kernel. A
joined observer system reading unsorted buckets is tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mirror

A finalizer on a joined thread read its thread's object resource at the
resource entity's row in the entity table, which can be ahead of the
mirror: before the thread's first rebuild, when the mirror holds no rows,
and when a later published sync moved the resource entity to another
archetype, it read undefined. It now reads the value the thread holds:
at the resource entity's row in the mirror as replayed so far, and
before a rebuild by the resource row's tag, as the rebuild reads it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… its mirror

Read-semantics fix: a joined finalizer's read of its thread's object
resource now follows the documented semantics (threads.md, Finalizers on
other threads): the value the thread holds, at the resource entity's row
in its mirror as replayed so far, its old rows before a rebuild, and the
start where no row holds it. Before b2e5e117 it could read undefined.

The rebuild branch reads the old rows rather than the resource row's tag,
so it reads nothing shared but the resource entity's index in the header,
and needs no structure-epoch check: the world gives the thread's object
resources only their start, so the old rows hold what the rebuild keeps.
Tests: the resource entity moved after it was last found, and moved twice
before the thread replayed. Tails' resource reads need no change, since
resources are added only once tails drained; threads.md says so. Adds the
string resource finding to followups.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… row of 100K

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a changed chunk's unchanged blocks

Every writer of a row tick raises its block's tick beside its chunk's, in shared memory with offsets in the layout
directory. Main-thread, worker and joined-thread filters skip chunks, blocks and rows each in a tight loop; WASM kernels
find runs of passing blocks in an outer loop and run their rows in an inner one. Index refreshes and `up` read blocks
too. Eight systems finding one changed row of 100K: 34.9 to 5.04 us in node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and no filters in an up kernel's WASM

Each writer stores a row's tick, then its block's, then its chunk's, the reverse of the order filters read them. New
tests fail with each remaining block stamp removed: WASM four-row stamps, including a group straddling two blocks from
an unaligned start; `added` blocks raised by a swap and stamped by a command's add; a JS `up` kernel's listed stamps.
A WASM `up` kernel given change filters now throws rather than stepping the wrong row. `stampAdded`, unused, is gone,
and perf.md states block memory for small archetypes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s rebuild

A thread whose log lapped rebuilds its mirror under the structure epoch. A rebuild
that found the epoch moved, odd when it began or changed by the end, gave up, and
the task retried only if the epoch was still odd when it checked. A sync point that
finished in between left the task never run, with nothing to wake the thread: the
world waited on the task forever (a joined tail's first task, under load).
`rebuild` now says "moved" for that, and the task retries once the epoch is even.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
krisnye and others added 30 commits October 23, 2026 23:27
…side an object, as DOM bindings make, is read

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… notes on lazy failures, list indexes and changes inside values

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ming packages

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arnings for rows taken as read

An observer's row the build can't follow is read whole, and a lookup's rows hold what its declaration says, so
neither warns; only a row taken as written whole does, at path:line:column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…zation, external utilities, docs, decisions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…'s readonly T.default, a reduction's identity, null or a zero value

The package's types have defaults of their own (F32.default, Vec3.default, Quat.default,
Entity.default ...). A T.default is shared, so the build takes it only when it's evidently
deeply readonly. A mutable Map or Set gets no implicit start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…schemas

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… explicit transactions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ties only when they're values, readonly defaults without mutable casts

Namespaces of yours outrank a reduction's identity at any depth; an identity outside
its type falls through to null and zero values; & is every bit of its column. A
namespace's default is checked against its type when the build can read it, and an
object start a joined thread knows by identity is never copied from a type-only import.
A schema's default is found only at keyword positions, and its error names the alias.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…name a module's package

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d object starts read in place

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e is readonly; failures name the part that isn't

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… narrowing what a system matches

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kernels, tails and joined observers leave them out for now

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quiring it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, the guide rendered with it, linked from the README

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the workflow names no package; @adobe/ecs publishes at /ecs/

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…; loops specialized for all present and all absent

JS and WASM: an archetype lacking an optional component gives its columns empty, so reads give undefined,
and a write fails the system as a body run as written does. WASM runs archetypes with every optional
component; JS runs the rest. GPU kernels still reject them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nents, rows of a snapshot bound by their archetype's

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… has them and never required

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tops still depend on with and not

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, tails and joined observers; GPU kernels still reject them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fused or reported, never a hang

The recorded hang was a test helper waiting for a joined status that a refused claim never reaches; the world
refuses the claim, or reports the finalizer once, as designed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…presence stamps per row; undefined assignment fails

Rows cached for one system were handed to another with the same keys but other
optional ones. A chunk stamped under presence tests of two components kept only
the last test, missing chunk and block stamps. Assigning undefined to a present
optional component now fails its system. Lookups on workers skip the extra
presence check where a column exists, kernels keep archetypes off WASM only for
optional components their body uses, and two doc comments are back on their
functions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ir package and the site

CI runs `pnpm -r run docs`, so typedoc warnings fail pull requests, not only the
deploy. collect-pages reads packages from pnpm-workspace.yaml, normalizes "./" to
".", and rejects a `from` leaving its package or a `to` leaving the site, which
check-workspace-conventions now checks too; site/ is ignored. The GPU samples
build for /data/gpu-samples/, where the site serves them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ponents

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant