From b057378b3ef6572dbc0a0bbe858c034ac43c79a8 Mon Sep 17 00:00:00 2001 From: Shreyan C Date: Mon, 28 Sep 2026 01:08:56 +0530 Subject: [PATCH] perf(tm): run long Turing machine runs through a table-driven fast lane A TM step cost about 300ns, and almost none of it was the machine: the generator handing over a step object, the cursor pulling it, a Map tape of strings, the loop fingerprint, three calls into the tape log and step columns, a transition lookup by name, and App.accepts asked through the reactive Set. The same steps as table lookups over a byte array take a few nanoseconds. Once the loop detector stops looking (after 5,000 configurations, or from step 0 when loop detection is off), streamTM hands the rest of the run to js/machines/fast-tm.js: states and symbols as small integers, delta as flat typed arrays filled the first time each pair is met, the tape log's own copy of the tape as the tape, and steps recorded straight into the same columns begin/noteWrite/step would have filled. Every reader of a run is untouched. The run cursor now tells a generator how many steps it wants (next(n)), and a producer can answer with batch(count, last, at). The drain still asks in 500-step slices, so block boundaries are found at the same granularity; playEagerly and for...of ask for nothing and get one step per next(). A single step skips the table and asks the lookup directly, so playback at Max does not pay for a re-check. Before each multi-step batch the table re-reads the transition fields and accepting marks the slow loop reads every step, so an edit to a paused run is obeyed the same way. BB(5) to its halt, Node 24: traceMachine 14.3s -> 1.43s; the drain's 500-step slices 13.6s -> 1.46s; with the space-time diagram built 15.5s -> 4.3s. One-step pulls 0.29 -> 0.25us. Memory unchanged at ~6 bytes a step. tests/fast-tm.test.js compares every step against the slow loop (setFastLane(false)) across BB(5), 160 random machines, pull sizes from 1 to a whole run, a run edited while paused, simTM, and an explicit blank stepped back over. Removing the undo exception, the re-check, or the live read on resume each fails it. --- .claude/skills/simulation/SKILL.md | 15 ++ CLAUDE.md | 3 + js/machines/columns.js | 48 +++++ js/machines/fast-tm.js | 307 +++++++++++++++++++++++++++++ js/machines/run.js | 81 ++++++-- js/machines/runtime.js | 6 +- js/machines/turing.js | 11 +- js/tape-log.js | 42 +++- tests/fast-tm.test.js | 225 +++++++++++++++++++++ tests/harness.js | 5 +- 10 files changed, 722 insertions(+), 21 deletions(-) create mode 100644 js/machines/fast-tm.js create mode 100644 tests/fast-tm.test.js diff --git a/.claude/skills/simulation/SKILL.md b/.claude/skills/simulation/SKILL.md index 284feac..1d6889e 100644 --- a/.claude/skills/simulation/SKILL.md +++ b/.claude/skills/simulation/SKILL.md @@ -195,6 +195,21 @@ Measured at the default `maxTmSteps` of 10,000: **921 MB**, on default settings. - **The window rule is duplicated from `Tape.snapshot()` and must not drift**, so [tests/tape-log.test.js](tests/tape-log.test.js) checks the rebuild against `snapshot()` itself across the bounded, two-way and right-bounded cases rather than against its own expectations. It also pins that read order does not matter (forward, backward and scattered access agree), and — the guard against the whole thing coming back — that `tape`, `head` and `view` are **not own properties** of a step. - The spread is gone from the window rule (`Math.min(head, ...keys)`): on a tape this module exists to make large, that is an argument list with an engine limit on it. +### The fast lane + +**A TM step cost ~300ns, and none of it was the machine.** Profiled on BB(5): the generator yielding a step object (35%), the cursor pulling it (15%), the Map tape and the loop fingerprint (12%), three calls into the log and the step columns (22%), the transition lookup by name (6%), and `App.accepts.has` through the reactive Set (3%). The same steps as table lookups over a byte array take ~8ns. [js/machines/fast-tm.js](js/machines/fast-tm.js) is that loop, recording into the *same* columns, so every reader of a run — the player, the scrubber, the trace log, the space-time diagram, StateMate — is untouched. + +- **It takes over when the loop detector stops looking**, which is what makes it exact rather than approximate: `makeLoopTracker` gives up after `LOOP_TRACK_MAX` (5,000) configurations, and is off when the reader turned it off, so from there the slow loop does nothing the fast one does not. `loop.verifying` is the hand-over test. Short runs, where loops are actually caught, never leave the slow loop. It is `TM`/`ITM` only (both go through `streamTM`); LBA and MTM could follow on the same terms. +- **The log's own copy of the tape is the tape.** `log.appender()` and `cols.appender()` expose the columns, the interners and `live` (the tape as codes) to a producer that records thousands of steps in one loop; it must leave exactly what `begin`/`noteWrite`/`step` would, including the checkpoints and the undo exception for an explicit blank. The step columns are filled from local buffers a block at a time (`pushMany`). +- **A batch is as many steps as the cursor asked for.** `iter.next(n)` says how many (a generator reads it as the value of `yield`), and a producer answers with `batch(count, last, at)` from [js/machines/run.js](js/machines/run.js). The drain asks for its 500-step slices, so a block boundary is found at the same granularity as before; `playEagerly` and `for…of` ask for nothing and get one step per `next()`. The last step of a batch is handed over *before* its write, as the slow loop yields before writing. +- **A single step skips the table.** Playback pulls one step at a time, and a per-step re-check of the table made playback at Max 2.8× slower before this path existed. A batch of one asks `fires` and `App.accepts` directly, as the slow loop does. +- **Edits to a paused run are obeyed, and only the ones the slow loop obeys.** `singleTapeLookup` fixes *which* transition fires per state for the life of a run, so the table fills an entry once; but the slow loop reads that transition's `to`/`write`/`dir` and `App.accepts` every step, so before every multi-step batch the table compares its copies against the objects and the accepting marks and starts over if anything moved. +- **`pull()` returns the step `take` added, not `steps[length - 1]`**: the columnar `steps` is a Proxy, and those two traps were a third of a one-step pull. And a batch is told apart by `instanceof Batch`, not a Symbol key — asking a dozen shapes of step object for a key they lack showed up on Moore. + +Measured on BB(5) to its halt (47,176,870 steps, Node 24, Ryzen 7 7435HS), against the slow loop: `traceMachine` **14.3 s → 1.43 s**; the drain's 500-step slices **13.6 s → 1.46 s**; with the space-time diagram built as well **15.5 s → 4.3 s** (the diagram's own index and overview, ~3 s, are unchanged). One-step pulls **0.29 → 0.25 µs**. Memory is unchanged at ~6 bytes a step — the same columns. `npm run bench -- player` shows the TM cases 84–90% faster and every other machine within its noise. + +[tests/fast-tm.test.js](tests/fast-tm.test.js) holds it to the slow loop (`setFastLane(false)`, reset by the harness) on every field a reader can ask — state, transition, note, verdict, every journal row with its symbol codes, the checkpoints, and the tape read forwards, backwards and scattered — across BB(5), 160 random machines (wildcard reads and writes, missing transitions, stay-put moves, both tape models, loop detection on and off, explicit blanks in the input) pulled one step, 500 steps and odd sizes at a time, a run edited while paused, `simTM`, and a trailing explicit blank stepped back over one write at a time. Removing the undo exception, the re-check, or reading the transition live on resume each fails it. + ### What a step remembers, when it is not a tape **The same fact was true of every machine that is not a tape machine, and it was the largest single cost in the app.** [js/machines/step-log.js](js/machines/step-log.js) is the companion to `tape-log.js`, and the shape of the bug was identical: a step held a copy of an array that grows with the run, so a run was quadratic in the word. diff --git a/CLAUDE.md b/CLAUDE.md index 14d38af..7ca578b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -236,6 +236,9 @@ js/machines/ what a checkpoint copies. zipper-tape.js the NDTM search's tape: two interned stacks and the cell under the head, so a fork is O(1). + fast-tm.js the rest of a TM run as a table and a loop, once the + loop detector has stopped looking — see the simulation + notes, *The fast lane*. batch.js the batch tester's deciding half, with no page attached. ``` diff --git a/js/machines/columns.js b/js/machines/columns.js index d7eeb00..8b72a00 100644 --- a/js/machines/columns.js +++ b/js/machines/columns.js @@ -69,6 +69,26 @@ export class CodeColumn { this.max = Type === Uint16Array ? 0xffff : 0xffffffff; } + /** + * Append `src[0..count)` — what `count` pushes would leave, a block copy at + * a time, for a producer that fills a buffer of its own first + * (js/machines/fast-tm.js). + */ + pushMany(src, count) { + let top = 0; + for (let k = 0; k < count; k++) if (src[k] > top) top = src[k]; + if (top > this.max) this.widen(top); + let k = 0; + while (k < count) { + const i = this.length; + const c = this.chunks[i >>> BITS] || newChunk(this.Type, this.chunks); + const n = Math.min(count - k, SIZE - (i & MASK)); + c.set(src.subarray(k, k + n), i & MASK); + this.length += n; + k += n; + } + } + /** What the column holds, in bytes — for the tests and the benchmark. */ bytes() { return this.chunks.length * SIZE * this.Type.BYTES_PER_ELEMENT; } } @@ -120,6 +140,34 @@ export class HeadColumn { this.last = v; } + /** Append `src[0..count)`, as `count` pushes would — the same anchors, the + * same jumps — with the fields kept in locals rather than read per row. */ + pushMany(src, count) { + let k = 0; + let last = this.last; + while (k < count) { + let i = this.length; + const dc = this.deltas[i >>> BITS] || newChunk(Int8Array, this.deltas); + const end = k + Math.min(count - k, SIZE - (i & MASK)); + for (; k < end; k++, i++) { + const v = src[k]; + if (i % SPAN === 0) { + const a = this.nAnchors++; + (this.anchors[a >>> BITS] || newChunk(Int32Array, this.anchors))[a & MASK] = v; + } + const d = i === 0 ? 0 : v - last; + if (d > JUMP && d <= 127) dc[i & MASK] = d; + else { + dc[i & MASK] = JUMP; + (this.jumps ??= new Map()).set(i, v); + } + last = v; + } + this.length = i; + } + this.last = last; + } + anchor(k) { const a = k / SPAN; return this.anchors[a >>> BITS][a & MASK]; diff --git a/js/machines/fast-tm.js b/js/machines/fast-tm.js new file mode 100644 index 0000000..023ea4d --- /dev/null +++ b/js/machines/fast-tm.js @@ -0,0 +1,307 @@ +// ══════════════════════════════════════════════════════════════════ +// THE FAST LANE — a deterministic one-tape run as a table and a loop +// ══════════════════════════════════════════════════════════════════ +// streamTM spends about 300ns a step, and none of it on the machine: a Map +// tape of strings, a transition lookup by state name and symbol, a reactive +// Set asked whether the state accepts, a fingerprint hashed for the loop +// detector, two calls into the tape log and one into the step columns, and a +// generator handing the step over. Measured on the five-state busy beaver, +// that is 14 seconds to its halt; the same steps as table lookups over a byte +// array take a fifth of a second. +// +// So once the loop detector has stopped looking (it gives up after +// LOOP_TRACK_MAX configurations, and is off when the reader turned it off), +// streamTM hands the rest of the run to this: states and symbols as small +// integers, δ as flat typed arrays filled in the first time each (state, +// symbol) pair is met, and the tape as the log's own copy of it in codes. +// Steps are recorded straight into the log's and the step columns' arrays — +// the same entries, in the same order, `begin`/`noteWrite`/`step` would have +// made — so nothing downstream can tell which loop produced a step. +// tests/fast-tm.test.js holds it to that, column for column. +// +// **A batch is as many steps as the cursor asked for, and no more.** The run +// cursor says how many it wants (see `batch` in js/machines/run.js); a drain +// asks for a slice, playback for one. The last step of a batch is handed over +// *before* its write, exactly as the slow loop yields a step before writing, +// so a run paused on it is in the same state either way. +// +// **The table is re-checked every time the cursor comes back.** The app does +// not reset a paused run when the machine is edited, and the slow loop carries +// on under the edit — though not under every edit. *Which* transition fires is +// fixed per state the first time the run is in it (`singleTapeLookup` builds a +// state's row once and keeps it for the run), so the table can fill an entry +// the first time it is met and never ask again. But the slow loop reads the +// chosen transition's `to`, `write` and `dir` off the object on every step, and +// asks `App.accepts` every step, so an edit to either shows up from the next +// step on. The table copies those, so before each batch it compares its copies +// with the objects and re-reads the accepting marks, and starts over if +// anything moved. An edit cannot land inside a batch: a batch is synchronous. +// +// **A single step skips the table altogether.** Playback asks for one step at +// a time, and a re-check per step would cost more than the step: it made +// playback at Max 2.8× slower before this path existed. So a batch of one is +// answered the way the slow loop answers it — the lookup and the accepting +// set asked directly, the transition's fields read on resume — with only the +// Map tape and the fingerprint gone. + +import { App } from '../state.js'; +import { DELETED, NO_WRITE } from '../tape-log.js'; +import { batch } from './run.js'; +import { markTimeoutStep } from './runtime.js'; + +// Whether streamTM may hand over at all. A seam for `setJumpRoute`'s reason: +// the fast lane is on for every long run, so the slow loop past step 5,000 is +// a path nothing would otherwise test — and the slow loop is the oracle +// tests/fast-tm.test.js holds this one to. +let enabled = true; + +export function setFastLane(on) { + enabled = on !== false; + return enabled; +} + +export const fastLaneEnabled = () => enabled; + +const UNRESOLVED = -2; +const NONE = -1; // no transition fires: the step rejects +const SAME = -1; // the write puts back what was read (no write, or the wildcard) + +const moveOf = dir => (dir === 'R' ? 1 : dir === 'L' ? -1 : 0); + +/** + * The rest of a streamTM run, from step `n`. + * + * @param ctx.log, ctx.cols the run's tape log and step columns + * @param ctx.fires the slow loop's transition lookup, (state, sym) → t + * @param ctx.blank the blank symbol + * @param ctx.twoWay whether the head may go left of cell 0 + * @param ctx.n the index of the next step + * @param ctx.state, ctx.via the state at step n and the transition into it + * @param ctx.head the head at step n + * @param ctx.last the step handed over most recently, which the + * budget running out is stamped on + * @param ctx.want how many steps the cursor last asked for + */ +export function* fastTM({ log, cols, fires, blank, twoWay, n, state, via, head, last, want }) { + const A = log.appender(); + const { live, heads, reads, writes, syms, blankCode } = A; + const { states, tids, stateCol, tidCol } = cols.appender(); + const vals = syms.values; + const tidCode = id => (id === null || id === undefined ? 0 : tids.code(id)); + + // ── states, by index ────────────────────────────────────────────── + // Interned into the step columns only when a step is first recorded in + // them, as the slow loop does, so the two agree on every code. + const ids = [], index = new Map(), scode = [], acc = []; + function idxOf(id) { + let k = index.get(id); + if (k === undefined) { + k = ids.length; + index.set(id, k); + ids.push(id); + scode.push(-1); + acc.push(App.accepts.has(id) ? 1 : 0); + } + return k; + } + + // ── δ, as flat arrays over (state << kbits) | symbol code ───────── + let kbits = 3, Q = 8; + let nxt, wr, mv, tc, tobj, tsnap, filled; + function allocate() { + const size = Q << kbits; + nxt = new Int32Array(size).fill(UNRESOLVED); + wr = new Int32Array(size); + mv = new Int8Array(size); + tc = new Int32Array(size); + tobj = new Array(size); + tsnap = new Array(size); + filled = []; + } + // Room for state index s and symbol code c. Growing empties the table; + // refilling it interns nothing new, since everything it names already is. + function ensure(s, c) { + if (s < Q && c < 1 << kbits) return; + while (s >= Q) Q *= 2; + while (c >= 1 << kbits) kbits++; + allocate(); + } + allocate(); + + let any = App.config.sym.any; + const snapshot = t => (t ? [t.to, t.write, t.dir, t.id] : null); + const same = (t, snap) => (t === null ? snap === null : snap !== null && + t.to === snap[0] && t.write === snap[1] && t.dir === snap[2] && t.id === snap[3]); + + // Fill the entry for state s reading code c. Returns the next state index, + // or NONE. The symbol a write names is interned here, which is the step that + // first writes it — where the slow loop's noteWrite interns it. + function resolve(s, c) { + const t = fires(ids[s], vals[c]) ?? null; + let to = NONE, w = SAME, d = 0, id = 0; + if (t) { + to = idxOf(t.to); + w = (!t.write || t.write === any) ? SAME : (t.write === blank ? 0 : syms.code(t.write)); + d = moveOf(t.dir); + id = tidCode(t.id); + } + ensure(Math.max(s, to), Math.max(c, w)); + const e = (s << kbits) | c; + nxt[e] = to; wr[e] = w; mv[e] = d; tc[e] = id; + tobj[e] = t; tsnap[e] = snapshot(t); + filled.push(e); + return to; + } + + // Everything the table copied, looked at again — see the header for why the + // lookup itself need not be. + function revalidate() { + for (let k = 0; k < ids.length; k++) acc[k] = App.accepts.has(ids[k]) ? 1 : 0; + let stale = App.config.sym.any !== any; + for (let k = 0; !stale && k < filled.length; k++) { + const e = filled[k]; + stale = !same(tobj[e], tsnap[e]); + } + if (stale) { any = App.config.sym.any; allocate(); } + } + + // ── the loop ────────────────────────────────────────────────────── + let s = idxOf(state), h = head, i = n, tid = tidCode(via); + // What the batch left for the resume: how it ended, and the last step's read. + let ended = null, lastC = 0, lastRc = 0, lastT = null; + + // A step's five numbers go into these first and into the columns a block at + // a time: five typed-array stores a step instead of five calls, each with a + // chunk lookup and a widening check. Nothing reads the columns while a batch + // runs — it is synchronous — so they are whole again before anyone could. + const BUF = 4096; + const hB = new Int32Array(BUF), rB = new Uint32Array(BUF), wB = new Uint32Array(BUF); + const sB = new Uint32Array(BUF), tB = new Uint32Array(BUF); + function flush(n) { + if (n === 0) return; + heads.pushMany(hB, n); + reads.pushMany(rB, n); + writes.pushMany(wB, n); + stateCol.pushMany(sB, n); + tidCol.pushMany(tB, n); + } + + // Record up to `lim` steps from step i. Stops early at an accepting state or + // a missing transition. Every step but the last is written and moved; the + // last is recorded with no write yet, which `finish` supplies on resume. + function run(lim) { + let S = s, H = h, I = i, T = tid; + let nextCp = A.nextCheckpoint; + let K = kbits, N = nxt, W = wr, M = mv, C = tc; + let count = 0, b = 0; + // Locals, not the closure's: a context-slot load per store adds up here. + const hb = hB, rb = rB, wb = wB, sb = sB, tb = tB, sc_ = scode, ac = acc, blk = blankCode, two = twoWay; + // The tape read straight out of its buffer; re-read after any write that + // may have grown it. + let tbuf = live.buf, tbase = live.base, tlen = tbuf.length; + ended = null; + for (;;) { + if (b === BUF) { flush(b); b = 0; } + if (I >= nextCp) nextCp = A.checkpoint(I); + const tk = H - tbase; + const c = tk >= 0 && tk < tlen ? tbuf[tk] : 0; + const rc = c === 0 ? blk : c; + hb[b] = H; + rb[b] = rc; + let sc = sc_[S]; + if (sc < 0) sc = sc_[S] = states.code(ids[S]); + sb[b] = sc; + tb[b] = T; + count++; + if (ac[S] === 1) { wb[b++] = NO_WRITE; ended = 'accept'; break; } + if (S >= Q || rc >= 1 << K) { ensure(S, rc); K = kbits; N = nxt; W = wr; M = mv; C = tc; } + let e = (S << K) | rc; + let to = N[e]; + if (to === UNRESOLVED) { + to = resolve(S, rc); + K = kbits; N = nxt; W = wr; M = mv; C = tc; + e = (S << K) | rc; + } + if (to === NONE) { wb[b++] = NO_WRITE; ended = 'reject'; break; } + if (count === lim) { + wb[b++] = NO_WRITE; + lastC = c; lastRc = rc; lastT = tobj[e]; + break; + } + const w = W[e]; + const now = w === SAME ? (rc === blk ? 0 : rc) : w; + wb[b++] = now === 0 ? DELETED : now + 1; + // The one cell whose read does not say what it held: an explicit blank + // from the input, where the blank is otherwise the absence of a cell. + if (c === blk) A.undoException(I, c); + // A cell left as it was needs no store — the log has the write either way. + if (now !== c) { live.set(H, now); tbuf = live.buf; tbase = live.base; tlen = tbuf.length; } + T = C[e]; + S = to; + const d = M[e]; + if (two || H + d >= 0) H += d; + I++; + } + flush(b); + s = S; h = H; i = I; tid = T; + return count; + } + + // A batch of one, asked the way the slow loop asks: see the header. + function one() { + const I = i; + if (I >= A.nextCheckpoint) A.checkpoint(I); + const c = live.get(h); + const rc = c === 0 ? blankCode : c; + heads.push(h); + reads.push(rc); + let sc = scode[s]; + if (sc < 0) sc = scode[s] = states.code(ids[s]); + stateCol.push(sc); + tidCol.push(tid); + writes.push(NO_WRITE); + ended = null; + if (App.accepts.has(ids[s])) { ended = 'accept'; return 1; } + const t = fires(ids[s], vals[rc]) ?? null; + if (!t) { ended = 'reject'; return 1; } + lastC = c; lastRc = rc; lastT = t; + return 1; + } + + // The last step of a batch writes and moves now, as the slow loop does after + // its yield — from the transition object as it stands, since that is what + // the slow loop reads on resume. + function finish() { + const t = lastT; + const sym = vals[lastRc]; + const put = (!t.write || t.write === App.config.sym.any) ? sym : t.write; + const now = put === blank ? 0 : syms.code(put); + writes.set(i, now === 0 ? DELETED : now + 1); + if (lastC === blankCode) A.undoException(i, lastC); + live.set(h, now); + tid = tidCode(t.id); + s = idxOf(t.to); + const d = moveOf(t.dir); + if (twoWay || h + d >= 0) h += d; + i++; + } + + for (;;) { + const max = App.config.maxTmSteps; + if (i >= max) { + if (last && !last.final) markTimeoutStep(last); + return; + } + const lim = Math.min(want >= 1 ? want : 1, max - i); + let count; + if (lim === 1) count = one(); + else { revalidate(); count = run(lim); } + const step = cols.at(i); + if (ended === 'accept') { step.final = 'accept'; step.note += ' — ACCEPT'; } + else if (ended === 'reject') { step.final = 'reject'; step.note += ' — REJECT'; } + last = step; + want = yield count === 1 ? step : batch(count, step, cols.at); + if (ended) return; + finish(); + } +} diff --git a/js/machines/run.js b/js/machines/run.js index 9e6dcf6..74c6ad1 100644 --- a/js/machines/run.js +++ b/js/machines/run.js @@ -32,6 +32,28 @@ // business knowing about App, the page, or which machine produced it — which // is why the one thing it needs from a machine, how to rebuild a step, is // handed in (`opts.columnsOf`) rather than imported. +// +// **A producer may answer with many steps at once.** The cursor says how many +// it wants — `iter.next(n)`, which a generator reads as the value of its +// `yield` — and a producer that can compute a stretch of steps faster than it +// can hand them over one by one (js/machines/fast-tm.js, which records into +// columns directly) yields `batch(count, last, at)`: `count` steps now exist, +// the newest is `last`, and `at(i)` builds any of the others. It never yields +// more than it was asked for, so a caller that pulls in slices — the drain, +// which scans each slice for a block boundary — sees exactly the granularity +// it asked for. A consumer that calls `next()` with no count, `playEagerly` and +// a plain `for…of`, gets one step per iteration as before. +// +// A batch is told apart by its class. A Symbol-keyed field was the first +// version, and asking every step object for a key it does not have — step +// objects come in a dozen shapes — cost Moore's one-step pulls a fifth of +// their time. +class Batch { + constructor(count, last, at) { this.count = count; this.last = last; this.at = at; } +} + +/** Several steps at once, for a producer the cursor asked for `count` or more. */ +export const batch = (count, last, at) => new Batch(count, last, at); /** * @param source a generator/iterator/iterable of steps, or a finished array. @@ -41,6 +63,11 @@ export function makeRun(source, opts = {}) { let steps; let append; + let appendMany = (count, last, at) => { + const from = steps.length; + for (let i = from; i < from + count - 1; i++) append(at(i)); + append(last); + }; let iter = null; let done = false; let result; @@ -54,20 +81,34 @@ export function makeRun(source, opts = {}) { if (source && typeof source.next === 'function') iter = source; else if (source && typeof source[Symbol.iterator] === 'function') iter = source[Symbol.iterator](); else done = true; - if (iter && opts.columnsOf) ({ steps, push: append } = columnarSteps(opts.columnsOf)); + if (iter && opts.columnsOf) ({ steps, push: append, pushMany: appendMany } = columnarSteps(opts.columnsOf)); else { steps = []; append = s => steps.push(s); } } - /** Materialize one more step. Returns it, or undefined at the end. */ - function pull() { - if (done) return undefined; - const r = iter.next(); + // The newest step `take` added — read back from here rather than from + // `steps`, which for a columnar run is a Proxy: two traps a pull, and on + // playback's one-step pulls that was a third of the time. + let newest; + + /** Ask for up to `want` more steps. Returns how many arrived: 0 at the end. */ + function take(want) { + if (done) return 0; + const r = iter.next(want); // A finished generator still holds its frame — the live tape it was // driving, which on a machine that walks off down its tape is as long as // the run — so the cursor lets go of it once there is nothing left to pull. - if (r.done) { done = true; result = r.value; iter = null; return undefined; } - append(r.value); - return r.value; + if (r.done) { done = true; result = r.value; iter = null; newest = undefined; return 0; } + const v = r.value; + if (!(v instanceof Batch)) { append(v); newest = v; return 1; } + appendMany(v.count, v.last, v.at); + newest = v.last; + return v.count; + } + + /** Materialize one more step. Returns it, or undefined at the end. */ + function pull() { + take(1); + return newest; } return { @@ -92,7 +133,7 @@ export function makeRun(source, opts = {}) { /** Materialize up to index i. Returns steps[i], or undefined past the end. */ at(i) { - while (!done && steps.length <= i) pull(); + while (!done && steps.length <= i) take(i + 1 - steps.length); return steps[i]; }, @@ -105,7 +146,7 @@ export function makeRun(source, opts = {}) { */ drain(cap = Infinity) { let n = 0; - while (!done && n < cap) { pull(); n++; } + while (!done && n < cap) n += take(cap - n); return done; } }; @@ -178,6 +219,24 @@ function columnarSteps(columnsOf) { n++; } + // `count` steps, the newest `last`. The ones between were recorded straight + // into the columns and never handed over as objects, so each is plain — a + // rebuild is exactly it — and nothing needs holding but the old tail. + function pushMany(count, last, at) { + if (count === 1) { push(last); return; } + const first = n; + push(at(first)); + if (!cols) { + // Not rebuildable after all: hold every one of them whole. + for (let i = first + 1; i < first + count - 1; i++) push(at(i)); + push(last); + return; + } + // Steps first..first+count-2 are plain; only the tail is special. + n = first + count; + tail = last; + } + function put(i, v) { if (i === n) { push(v); return true; } if (!(i >= 0 && i < n) || !Number.isInteger(i)) return false; @@ -225,6 +284,6 @@ function columnarSteps(columnsOf) { return keys.concat(Reflect.ownKeys(t)); } }); - return { steps, push }; + return { steps, push, pushMany }; } diff --git a/js/machines/runtime.js b/js/machines/runtime.js index 7c6d864..4085f70 100644 --- a/js/machines/runtime.js +++ b/js/machines/runtime.js @@ -391,11 +391,15 @@ export function makeLoopTracker() { // batch deciders (testTM3 and friends) keep their own repeat check, because // there the repeat *is* the answer — a word nobody is watching run gains // nothing from "no verdict" in place of a correct reject. - if (!detectsLoops()) return { seenAt: () => -1, seenAtVerified: () => -1 }; + if (!detectsLoops()) return { seenAt: () => -1, seenAtVerified: () => -1, verifying: false }; let seen = new Map(); let fps = makeFingerprintTable(); return { + // Whether seenAtVerified can still report a loop. Once it has given up + // it answers -1 for the rest of the run, which is what lets streamTM hand + // the rest to a loop that does not fingerprint (js/machines/fast-tm.js). + get verifying() { return fps !== null; }, // Step index where this configuration was first seen, or -1 if new. seenAt(key, idx) { if (!seen) return -1; diff --git a/js/machines/turing.js b/js/machines/turing.js index da5b851..4beb96b 100644 --- a/js/machines/turing.js +++ b/js/machines/turing.js @@ -32,6 +32,7 @@ import { } from '../state.js'; import { Tape, makeTapes, tapesKey } from '../tape.js'; import { makeStepColumns, makeTapeLog } from '../tape-log.js'; +import { fastLaneEnabled, fastTM } from './fast-tm.js'; import { buildMarkedInputTape, tapeTuplesOverlap } from './predicates.js'; import { ConfigSet, Fifo, firstOverlappingTransition, makeRepeatDetector, makeStateNumbering, formatTapeInstantaneousDescription, langStepBudget, makeLoopTracker, markLoopStep, markTimeoutStep, multiTapeLookup, nameOfState, parseWordInput, playEagerly, singleTapeLookup, tokenize, transitionsFrom } from './runtime.js'; import { defineFamily, machineDef } from './registry.js'; @@ -83,7 +84,15 @@ export function* streamTM(tokens) { const sameAs = j => replaySingleTapeKey(() => new Tape(tokens, blank, twoWay), j) === (now ??= `${state}|${tape.key()}`); let step = null; let n = 0; + // How many steps the cursor last asked for (see `batch` in run.js). + let want = 1; for (; n < App.config.maxTmSteps; n++) { + // Once no loop can be reported any more, the rest of the run is a table + // and a loop — see js/machines/fast-tm.js. From here the log's copy of the + // tape is the tape; `tape` is left where it stands and read no further. + if (!loop.verifying && fastLaneEnabled()) { + return yield* fastTM({ log, cols, fires, blank, twoWay, n, state, via, head: tape.head, last: step, want }); + } const sym = tape.read(); const i = log.begin(tape.head, sym); step = cols.step(i, state, via); @@ -94,7 +103,7 @@ export function* streamTM(tokens) { if (at >= 0) { markLoopStep(step, at); yield step; return; } const t = fires(state, sym); if (!t) { step.final = 'reject'; step.note += ' — REJECT'; yield step; return; } - yield step; + want = yield step; const cell = tape.head; tape.write((!t.write || t.write === App.config.sym.any) ? sym : t.write); log.noteWrite(i, cell, tape.cells); diff --git a/js/tape-log.js b/js/tape-log.js index 3f3214a..327b9a6 100644 --- a/js/tape-log.js +++ b/js/tape-log.js @@ -46,8 +46,8 @@ import { BITS, CodeColumn, DenseTape, HeadColumn, JUMP, MASK, makeInterner } from './machines/columns.js'; // A write column's codes: none, a deleted cell, or a symbol's code + 1. -const NO_WRITE = 0; -const DELETED = 1; +export const NO_WRITE = 0; +export const DELETED = 1; // The fewest steps between checkpoints; a wide tape spaces them further — see // `begin`. Each costs an object as well as its cells, so a floor this high @@ -115,6 +115,13 @@ export function makeTapeLog(tape) { const checkpoints = []; let nextCheckpoint = 0; + function checkpoint(i) { + const cp = live.snapshot(i); + checkpoints.push(cp); + nextCheckpoint = i + Math.max(CHECKPOINT_MIN, cp.codes.length); + return nextCheckpoint; + } + // ── the reader's copy ─────────────────────────────────────────── // `cur` is the tape as it stands before step `curI`: what step curI shows. // One, because one step is displayed at a time and every reader of a step @@ -249,11 +256,7 @@ export function makeTapeLog(tape) { */ begin(head, read) { const i = heads.length; - if (i >= nextCheckpoint) { - const cp = live.snapshot(i); - checkpoints.push(cp); - nextCheckpoint = i + Math.max(CHECKPOINT_MIN, cp.codes.length); - } + if (i >= nextCheckpoint) checkpoint(i); heads.push(head); reads.push(read === undefined ? 0 : syms.code(read)); writes.push(NO_WRITE); @@ -279,6 +282,27 @@ export function makeTapeLog(tape) { live.set(cell, now); }, + /** + * The columns themselves, for a producer that records thousands of steps + * in one loop (js/machines/fast-tm.js) and cannot afford two calls and a + * Map lookup per step. It must leave exactly what `begin` and `noteWrite` + * would have: the same entries in the same order, a checkpoint wherever + * `nextCheckpoint` falls due, and an undo exception wherever the read does + * not say what the cell held. tests/fast-tm.test.js holds it to that, + * column for column, against the loop that goes through them. + * + * `live` is the tape as codes — 0 for a cell holding nothing — and is + * the producer's to drive from here on. + */ + appender() { + return { + live, heads, reads, writes, syms, blankCode, + get nextCheckpoint() { return nextCheckpoint; }, + checkpoint, + undoException(i, prior) { (undoExceptions ??= new Map()).set(i, prior); } + }; + }, + frameAt, /** The absolute head position at step i. */ @@ -474,6 +498,10 @@ export function makeStepColumns(logs, tokens, noteAt, multi = false) { /** Step i, built again from the columns. */ at: build, + /** The state and transition columns and their interners, for the same + * bulk producer as the log's `appender()` and on the same terms. */ + appender: () => ({ states, tids, stateCol, tidCol }), + /** * Whether `step` holds nothing `at(i)` would not rebuild: its own fields * are its address and nothing else. diff --git a/tests/fast-tm.test.js b/tests/fast-tm.test.js new file mode 100644 index 0000000..8a8c601 --- /dev/null +++ b/tests/fast-tm.test.js @@ -0,0 +1,225 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { createHarness } from './harness.js'; + +// js/machines/fast-tm.js takes over a TM run once the loop detector has +// stopped looking, and records straight into the tape log's and the step +// columns' arrays. The claim it rests on: **nothing downstream can tell which +// loop produced a step.** So the oracle here is the slow loop itself +// (`setFastLane(false)`), and every run is compared on everything a reader can +// ask of it — each step's state, transition, note and verdict, each journal +// row including the symbol codes, the checkpoints, and the tape the tracker +// draws, read forwards, backwards and scattered, which is the undo path. + +const harness = createHarness(); +const { context } = harness; + +function rng(seed) { + let s = seed >>> 0 || 1; + return () => (s = (Math.imul(s, 1103515245) + 12345) & 0x7fffffff) / 0x7fffffff; +} + +function bb5() { + harness.resetApp(); + const { App, setMachine } = context; + setMachine('TM'); + const B = App.config.sym.blank; + for (const s of ['A', 'B', 'C', 'D', 'E', 'H']) App.states.push({ id: s, name: s, x: 0, y: 0 }); + App.startId = 'A'; + App.accepts.add('H'); + App.sigma = new Set(['1']); + App.tapeAlphabet = new Set(['1', B]); + App.config.twoWayTape = true; + [['A', B, '1', 'R', 'B'], ['A', '1', '1', 'L', 'C'], ['B', B, '1', 'R', 'C'], ['B', '1', '1', 'R', 'B'], + ['C', B, '1', 'R', 'D'], ['C', '1', B, 'L', 'E'], ['D', B, '1', 'L', 'A'], ['D', '1', '1', 'L', 'D'], + ['E', B, '1', 'R', 'H'], ['E', '1', B, 'L', 'A']] + .forEach(([from, symbol, write, dir, to], i) => App.transitions.push({ id: 't' + i, from, to, symbol, write, dir })); +} + +/** + * A random deterministic TM over {a, b, x}: some pairs with no transition + * (a reject), a wildcard read on some states, writes that are omitted or the + * wildcard (put back what was read), a stay-put direction, an accepting state + * or none. Returns the input, which may hold an explicit blank. + */ +function randomTM(seed) { + harness.resetApp(); + const { App, setMachine } = context; + setMachine('TM'); + const r = rng(seed); + const B = App.config.sym.blank, any = App.config.sym.any; + const Q = 2 + Math.floor(r() * 5); + const syms = ['a', 'b', 'x', B]; + for (let q = 0; q < Q; q++) App.states.push({ id: 'q' + q, name: 'q' + q, x: 0, y: 0 }); + App.startId = 'q0'; + if (r() < 0.6) App.accepts.add('q' + Math.floor(r() * Q)); + App.sigma = new Set(['a', 'b']); + App.tapeAlphabet = new Set(syms); + App.config.twoWayTape = r() < 0.5; + App.config.detectLoops = r() < 0.5; + let id = 0; + const dirs = ['L', 'R', 'R', 'L', 'S']; + for (let q = 0; q < Q; q++) { + const wild = r() < 0.2; + for (const sym of wild ? [any] : syms) { + if (r() < 0.06) continue; + const w = r(); + App.transitions.push({ + id: 't' + id++, from: 'q' + q, to: 'q' + Math.floor(r() * Q), symbol: sym, + write: w < 0.15 ? '' : w < 0.3 ? any : syms[Math.floor(r() * syms.length)], + dir: dirs[Math.floor(r() * dirs.length)] + }); + } + } + const len = Math.floor(r() * 12); + return Array.from({ length: len }, () => (r() < 0.1 ? B : r() < 0.5 ? 'a' : 'b')); +} + +/** Pull the run the way a reader would: `slices` is the size of each ask. */ +function play(tokens, fast, slices) { + context.setFastLane(fast); + const run = context.streamMachine('TM', tokens); + if (!slices) run.drain(); + else for (let k = 0; !run.done; k++) run.drain(slices(k)); + return run; +} + +function sameRun(slow, fast, label, r = rng(99)) { + const a = slow.steps, b = fast.steps; + assert.equal(b.length, a.length, `${label}: length`); + const n = a.length; + const ja = context.stepJournals(a[n - 1])[0], jb = context.stepJournals(b[n - 1])[0]; + for (let i = 0; i < n; i++) { + const x = a[i], y = b[i]; + assert.equal(y.state, x.state, `${label}: state at ${i}`); + assert.equal(y.tid, x.tid, `${label}: transition at ${i}`); + assert.equal(y.final, x.final, `${label}: verdict at ${i}`); + assert.equal(y.loopFrom, x.loopFrom, `${label}: loop at ${i}`); + assert.equal(y.note, x.note, `${label}: note at ${i}`); + const ra = { ...ja.into(i) }, rb = { ...jb.into(i) }; + assert.deepEqual(rb, ra, `${label}: journal row ${i}`); + } + assert.deepEqual(jb.checkpoints.map(c => [c.row, c.lo, [...c.codes]]), ja.checkpoints.map(c => [c.row, c.lo, [...c.codes]]), `${label}: checkpoints`); + // The tape as the tracker draws it: forwards, backwards, scattered. + const at = []; + for (let i = 0; i < n; i += Math.max(1, n >> 7)) at.push(i); + for (let i = n - 1; i >= 0; i -= Math.max(1, n >> 7)) at.push(i); + for (let k = 0; k < 64; k++) at.push(Math.floor(r() * n)); + for (const i of at) { + assert.deepEqual([a[i].tape.join('|'), a[i].head, a[i].view.origin], [b[i].tape.join('|'), b[i].head, b[i].view.origin], `${label}: tape at ${i}`); + } +} + +test('BB(5): the fast lane records what the slow loop does, from the hand-over at ~5,000 steps', () => { + bb5(); + context.App.config.maxTmSteps = 300000; + const slow = play([], false); + const fast = play([], true); + assert.equal(fast.steps[fast.steps.length - 1].final, 'timeout'); + sameRun(slow, fast, 'bb5'); +}); + +test('BB(5) with loop detection off: the fast lane runs from step 0', () => { + bb5(); + context.App.config.detectLoops = false; + context.App.config.maxTmSteps = 100000; + sameRun(play([], false), play([], true), 'bb5 no loops'); +}); + +test('BB(5) runs to its halt the same either way, however it is pulled', () => { + bb5(); + context.App.config.detectLoops = false; + context.App.config.maxTmSteps = 47176871; + const run = play([], true); + const last = run.steps[run.steps.length - 1]; + assert.equal(run.steps.length, 47176871); + assert.equal(last.final, 'accept'); + assert.equal(last.tape.filter(s => s === '1').length, 4098); +}); + +test('random machines agree on every step, pulled in slices of every size', () => { + let handed = 0; + for (let seed = 1; seed <= 160; seed++) { + const tokens = randomTM(seed); + context.App.config.maxTmSteps = 3000 + (seed % 7) * 2000; + const slow = play(tokens, false); + if (slow.steps.length > 5001 || !context.App.config.detectLoops) handed++; + const r = rng(seed); + // One step at a time, the drain's slices, and odd sizes between. + const pick = seed % 3 === 0 ? () => 1 : seed % 3 === 1 ? () => 500 : () => 1 + Math.floor(r() * 900); + sameRun(slow, play(tokens, true, pick), `seed ${seed}`); + } + // The comparison is only worth something if the fast lane actually ran. + assert.ok(handed > 60, `the fast lane ran on ${handed} machines`); +}); + +test('an edit made while a run is paused is obeyed from the next step, in both loops', () => { + const edited = fast => { + bb5(); + context.App.config.detectLoops = false; + context.App.config.maxTmSteps = 60000; + context.setFastLane(fast); + const run = context.streamMachine('TM', []); + run.drain(20000); + // In place, as the transition dialog edits: a write, a target and a + // direction changed, and an accepting mark moved. + const t = context.App.transitions; + t[3].write = context.App.config.sym.blank; + t[5].to = 'D'; + t[6].dir = 'R'; + context.App.accepts.delete('H'); + context.App.accepts.add('E'); + // One at a time for a while — the path that asks δ directly — then in + // slices, which re-check the table. + for (let k = 0; k < 50; k++) run.pull(); + run.drain(15000); + t[5].to = 'E'; + context.App.accepts.delete('E'); + for (let k = 0; k < 50; k++) run.pull(); + run.drain(); + return run; + }; + sameRun(edited(false), edited(true), 'edited'); +}); + +test('simTM, which pulls one step per next(), gets the same steps', () => { + bb5(); + context.App.config.detectLoops = false; + context.App.config.maxTmSteps = 20000; + context.setFastLane(false); + context.simTM([]); + const slow = { steps: context.App.simSteps }; + context.setFastLane(true); + context.simTM([]); + sameRun(slow, { steps: context.App.simSteps }, 'simTM'); +}); + +test('an explicit blank in the input is put back when stepping backwards', () => { + // A blank and an explicit blank draw the same, so the difference only shows + // at an end: a trailing blank in the input is part of step 0's window. The + // machine writes over it at once and runs on; walking back to step 0 one + // write at a time — no checkpoint to lean on — must put it back. + const trailing = fast => { + harness.resetApp(); + const { App, setMachine } = context; + setMachine('TM'); + const B = App.config.sym.blank; + App.states.push({ id: 'r', name: 'r', x: 0, y: 0 }); + App.startId = 'r'; + App.sigma = new Set(['1']); + App.tapeAlphabet = new Set(['1', 'x', B]); + App.transitions.push({ id: 'w', from: 'r', to: 'r', symbol: App.config.sym.any, write: 'x', dir: 'R' }); + App.config.detectLoops = false; + App.config.maxTmSteps = 3000; + const run = play(['1', '1', B], fast); + context.setJumpRoute('near'); + const n = run.steps.length; + run.steps[n - 1].tape; + const first = run.steps[0].tape; + context.setJumpRoute('auto'); + return first; + }; + const slow = trailing(false); + assert.equal(slow.length, 3, 'the slow loop keeps the trailing blank in step 0'); + assert.deepEqual(trailing(true), slow); +}); diff --git a/tests/harness.js b/tests/harness.js index 2dfaf67..70ffe65 100644 --- a/tests/harness.js +++ b/tests/harness.js @@ -114,6 +114,7 @@ import * as machinePaint from '../js/machines/paint.js'; import * as machineRun from '../js/machines/run.js'; import * as machineColumns from '../js/machines/columns.js'; import * as machineBranchTree from '../js/machines/branch-tree.js'; +import * as machineFastTm from '../js/machines/fast-tm.js'; import * as parallelPool from '../js/parallel/pool.js'; import * as parallelSnapshot from '../js/parallel/snapshot.js'; import * as parallelCore from '../js/parallel/decide-core.js'; @@ -146,7 +147,7 @@ import * as wizardUi from '../js/wizard-ui.js'; const NAMESPACES = [ state, store, themes, exportRegistry, dropdown, modal, utils, anim, viewport, geometry, statesTransitions, blocks, blocksUi, viewGraph, graphThumb, scope, runScope, canvas, render, panelState, panelSections, traceFormat, panelSectionsUi, panelFloat, sectionStatus, machineOptionsUi, deltaTable, panelShake, panelList, mobile, notes, dividers, - machineRegistry, machineRuntime, machineBranchTree, machineFinite, machineWeighted, machineOmega, + machineRegistry, machineRuntime, machineBranchTree, machineFastTm, machineFinite, machineWeighted, machineOmega, machinePushdown, machineEmbedded, machineTuring, machineTransducer, machineTwoWay, machines, machinePredicates, machineBatch, machinePaint, machineRun, machineColumns, parallelPool, parallelSnapshot, parallelCore, simulation, speedControl, tape, tapeLog, tapeView, spacetime, spacetimeUi, branchTreeUi, branchTokens, complexity, complexityUi, suggest, language, alphabet, markdown, @@ -219,6 +220,8 @@ function resetModuleState() { geometry.setLabelKernel('auto'); // Likewise a test that forced one way of reaching a step in a tape log. tapeLog.setJumpRoute('auto'); + // And one that ran a TM on the slow loop alone. + machineFastTm.setFastLane(true); viewport.invalidateCull(); // The block index validates itself the way the state index does, and a test // can replace App.blocks with an equal-looking array the validator coincides