Repository navigation
Move bench onto the site's Bun, Biome, Next.js 16 and TypeScript 7 - #128
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
bench/scripts/parity/run.sh serves a base git ref and the working tree side by side with `next start`, against one Postgres restored from a production dump, and compares them. compare-api.ts requests every group and chart route from both servers and compares status, content-type, cache-control and body, printing the route and first differing JSON path with both values. screenshots.spec.ts compares the landing page and five chart pages pixel by pixel at desktop and phone widths, and Playwright keeps the diff images. The database runs from a local Postgres 16 data directory, since the integration suite's container is not required. Group-cache fills run one at a time because they share a database-wide advisory lock, and each server starts with an empty Data Cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Every URL the API check requests comes from the base's own /api/groups, so a 404, a redirect or an empty body on either side now fails with "nothing was compared for this route", even when both servers agree on it. Slugs are percent-encoded, and the summary line tallies the statuses and body bytes that were compared. run.sh deletes the whole .next/cache of each side before it starts a server, so a Next.js version that keeps Data Cache payloads somewhere other than fetch-cache cannot serve payloads made by older code. The README and config comment now say what the screenshot comparison does: Playwright's comparator ignores pixels it classes as anti-aliasing, and any other differing pixel, or a layout shift of 1px or more, fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
GitHub's dependency graph does not read bun.lock, so dependency-review sees none of the site's transitive packages. scripts/dependency-snapshot.ts turns the lock into a snapshot for the Dependency Submission API, with one manifest per workspace. A package is runtime when a workspace's dependencies or optionalDependencies reach it, and development otherwise. Optional peers add no edge: Next.js lists Playwright as one, and following it would mark Playwright runtime. Every package in the current lock is in the snapshot, and every open high advisory in `bun audit` is development scope. The dependency-snapshot workflow runs it on pushes to main and on pull requests from this repository, with contents: write on that job only and no bun install. dependency-review now retries for up to 300 seconds while the snapshot arrives. fail-on-scopes stays at the action's default, runtime. CLAUDE.md layer 5 says so. The tests use node:test, so the root tsc needs no bun types. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
The submission script now waits 5 seconds and tries once more after a network error or a 5xx answer. Any other status, and a second failure, still exits 1. The wait can be set with SNAPSHOT_RETRY_DELAY_MS, which the tests use. The token is never logged. The workflow groups a push to main by commit, so a burst of pushes cannot cancel or replace a pending run and leave a commit without a snapshot. Pull requests keep one group per ref and cancel the older run. New tests run the script against a local mock of the API: 503 then 201, 503 twice, 422 with no retry, and a refused connection. Another test covers a workspace whose path (bench/web) differs from its package name (benchmarks-web), with a workspace-keyed copy of a package, a hoisted copy, and a decoy keyed by path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
bench/web joins the root Bun workspace. One bun.lock replaces bench/web/pnpm-lock.yaml, and bench/web/pnpm-workspace.yaml goes: - The zod override (^4.6.7) moves to the root package.json. It also moves Lighthouse CI's chromium-bidi to zod 4, as decided, and clears the site's accepted zod advisory. The sharp override was already there. The postcss override is not carried over: Next.js 16.4.1 depends on postcss 8.5.23, which is patched, so until bench moves to Next.js 16 its next 15.5.28 brings postcss 8.4.31. CLAUDE.md lists those advisories under "bench advisories". - undici 7.30.1 is still inside the 14-day window, so bunfig.toml exempts undici until 2026-10-29 14:18 UTC, with the reason from the pnpm file. Every other version pnpm had locked is past the window. - The lockfile keeps pnpm's versions for every package only bench uses. The rest are the site's copies, which Bun dedupes onto. The site's resolution changes only in zod 3.25.76 -> 4.6.7. bunfig.toml sets linker = "isolated", Bun's default for a workspace. A hoisted tree hands bench's typescript-eslint the site's TypeScript 7, which it rejects. publicHoistPattern puts ESLint and Prettier packages in the root node_modules, where bench's ESLint config finds its plugins, as pnpm's default did. next.config.js traces from the repository root, where the packages now live. Every route traces the same packages at the same versions as the pnpm build on main. The bench-checks action, bench-ci.yml and bench-deploy.yml install with `bun install --frozen-lockfile` at the root and run bench's scripts with `bun run --cwd bench/web`. bench/web/vercel.json sets the same install command for `vercel build`, so the deploy does not depend on the project's settings. bench-deploy.yml also runs on changes to the root package.json, bun.lock and bunfig.toml. changed-paths counts the workspace's root files for both apps. CLAUDE.md and bench's docs use the Bun commands. trustedDependencies stays ["esbuild", "sharp"]. pnpm also let unrs-resolver build; its postinstall only checks for the native binding, which Bun installs as an optional dependency. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
bench moves to next 16.4.1 and react/react-dom ^19.3.1, the site's
versions, so both apps now share one copy. Next.js 16.4.1 depends on
postcss 8.5.23, which clears the four postcss advisories that next
15.5.28's postcss 8.4.31 carried; no postcss override is needed.
- POST /api/revalidate calls revalidateTag(BENCH_DATA_TAG, { expire: 0 }).
That is the documented equivalent of the old one-argument call; 'max'
would serve stale data once after an ingest. A new test pins it.
- Other bumps, each the newest version at least 14 days old: pg and
@types/pg 8.23.1, jsdom 30.1.2, vitest 5.0.4 (clears the two
@vitest/mocker advisories), eslint-config-next 16.4.1, @types/node
^24.19.2, @types/react(-dom) ^19.3.0, @vercel/functions ^3.10.1
(already locked). chart.js stays at 4.5.1.
- jsdom 30 uses undici 8. The newest 8.x past the window, 8.12.1, has
four high advisories fixed in 8.12.2, so the existing undici exemption
now covers 8.12.2 as well; the lock pins 8.12.2, and the exemption
list's removal time moves to 2026-10-29 15:06 UTC.
- The lock keeps @types/pg on the shared @types/node 24.19.2, as the
workspace move did.
- eslint-config-next 16 ships flat config, so eslint.config.mjs imports
it directly instead of through FlatCompat. Two new React Compiler rules
from eslint-plugin-react-hooks 7 are off, so the upgrade needs no code
change.
- next build (Turbopack, now the default) set tsconfig's jsx to
react-jsx and added .next/dev/types; committed so builds stay clean.
- Docs: bench/web/README.md, the architecture pages that describe
revalidateTag, the postcss.config.mjs and vitest.config.ts comments,
bunfig.toml, and CLAUDE.md (toolchain, audit counts 17 -> 11, the
bench advisories list).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Will Manning <will@willmanning.io>
These two files were outside Prettier's glob and ran past 100 columns. Biome's check now covers them, so this is the one-time mechanical reformat, kept apart from the Biome switch. No logic changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
The root biome.json now covers bench. A bench/** override keeps its Prettier style (single quotes, trailing commas, 100 columns) and turns useExhaustiveDependencies on, in place of ESLint's react-hooks rule. It turns off five rules that fire on deliberate markup and CSS, and import sorting. bench/AGENTS.md gives the reasons, because biome.json cannot hold comments. Lint fixes keep behavior: a block around a switch case, an explicit type on one let, braces on void forEach callbacks, an optional chain, a template string, an import type. Two ESLint disable comments for exhaustive-deps become biome-ignore, and a third effect gets one. Stale ESLint disable comments are gone. Remove eslint, eslint-config-next, @eslint/eslintrc and prettier with their config files. Their plugins were the only reason for publicHoistPattern in bunfig.toml, so it goes too; a clean install, the suite and both builds pass without it. Fix the stale Next.js, TypeScript and ESLint statements in bunfig.toml and CLAUDE.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
bench's format and lint steps covered only bench/web, and ci.yml skips the root check:ci on bench-only changes, so bench/scripts went unchecked until the next site PR. One step, bun run biome ci bench, replaces both: it uses the workspace's pinned Biome with the root config and covers web, scripts and the parity harness. The job name does not change. Fix two notes in bench/AGENTS.md: organizeImports would rewrite 27 files (25 in bench/web, 2 in bench/scripts), and the site config also turns off noExplicitAny, so bench's old no-explicit-any check is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Move bench/web to typescript ^7.0.2, as at the root, so both apps run the Go-based compiler. The tsconfig options the two apps shared now live in a root tsconfig.base.json (strict, noEmit, esModuleInterop, module, moduleResolution, resolveJsonModule, isolatedModules, skipLibCheck, allowJs, incremental, noUnusedLocals, noUnusedParameters, lib, jsx and the next plugin). Each tsconfig.json extends it and keeps its own target, paths and extra flags: the site its noUncheckedIndexedAccess and exactOptionalPropertyTypes, bench its noFallthroughCasesInSwitch and noUncheckedSideEffectImports. `tsc --showConfig` gives the same effective options for both apps as before. TypeScript 7 reports no errors in bench under the existing flags, so no source changes. bench-deploy.yml now also runs when tsconfig.base.json changes, since bench's build extends it. bunfig.toml and CLAUDE.md no longer say bench keeps its own TypeScript 5.9. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
The bench override in biome.json turned four a11y rules off for all of bench. New UI would then skip them unseen. Turn them back on and put a biome-ignore comment with its reason at each of the seven existing places. The markup does not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
…heck The parity check compared only the read API and light-theme screenshots. It now also requests the landing page and the five chart pages the screenshots use, and compares their status, content-type, cache-control and title. It probes an unknown chart slug (404) and a malformed slug (400 from the API, 404 from the page), and accepts only those expected statuses. Every other URL must still answer 200. The screenshots run in the dark theme too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
bunfig.toml gave a reason for the isolated linker that no longer holds, now that both apps share TypeScript and ESLint is gone. State the reason that does hold. vitest.config.ts said Next.js owns tsconfig's jsx setting; it now comes from tsconfig.base.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
bench-checks lost its dependency cache when it moved from pnpm to Bun. Restore it with the same cache step as ci.yml. Run the parity check's own tests there too, by explicit path, so its planning and comparison logic is checked on every bench change. Correct a comment that said Biome checks bench's imports; import sorting is off for bench. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Dependency review passed without bun.lock's packages when the snapshot job failed, was skipped for a fork, or finished after the action's retry window. A guard now runs before the action. For a pull request from this repository, it waits up to 10 minutes for the snapshot job on the head commit and fails unless that job succeeded. For a fork, which gets no snapshot, it fails when the pull request changes a bun.lock. The job gains checks: read, and its checkout no longer keeps the token. The snapshot job now checks out the pull request's head commit, so the graph it files under that commit holds the head's own bun.lock. CLAUDE.md describes the guard, says that development-scope advisories (bench's included) no longer block merge and show in bun audit, and gives the real fallback for the zod override. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
Give the publish times behind the statement that next 16.4.1 and react 19.3.1 were past the cooldown when bench moved to them, and say how the "13 days old" for the site differs. Rewrite the vite note, which still described pnpm. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🔦 Lighthouse audit
Run |
|
Bench preview for |
lwwmanning
marked this pull request as ready for review
October 29, 2026 05:32
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
lwwmanning
added a commit
that referenced
this pull request
Oct 29, 2026
Resolves CLAUDE.md and bun.lock against #128, which moved bench onto the root Bun workspace, Biome, Next.js 16 and TypeScript 7. - CLAUDE.md keeps main's toolchain, CI and deploy notes and this branch's architecture text; the audit state is 10 advisories (8 in the site, all via @lhci/cli, and bench's 2), since this branch removed @tailwindcss/typography. - bun.lock is main's, re-resolved with this branch's dependencies. - bench's vortex-skin.css takes Biome's formatting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bench now uses the same toolchain as the site: one Bun workspace and lockfile, Biome, Next.js 16.4.1, React 19.3 and TypeScript 7. The design and plan are in
bench/docs/plans/; #113 tracks the move.A new parity check (
bench/scripts/parity/) servesmainand this branch side by side on a copy of the production database. It found no difference in 881 responses (API, pages, error probes, titles) and 28 screenshots in light and dark themes.Two changes need your attention. Dependency review now reads
bun.lockthrough a submitted snapshot and fails if the snapshot is missing, but it blocks only on runtime packages; dev-only advisories show inbun audit. bench's Next.js build moves from postcss 8.5.30 to the patched 8.5.23.🤖 Generated with Claude Code