Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Keep them in lockstep when bumping, and move to a new major only after Vercel of

`bench/web` is a member of the root Bun workspace, so the layers above cover it: the cooldown in `bunfig.toml`, `trustedDependencies` and `overrides` in the root `package.json`, `bun audit`, and the `bun.lock (bench/web)` manifest of the dependency snapshot. What is particular to bench:

- **Install at the root.** `bun install --frozen-lockfile` at the root installs both apps, and `bun run --cwd bench/web <script>` runs bench's scripts; CI and `bench-deploy.yml` do the same, and `bench/web/vercel.json` gives `vercel build` the same install command. A `bun install` in `bench/web` also finds the root's `bunfig.toml`.
- **Install at the root.** `bun install --frozen-lockfile` at the root installs both apps, and `bun run --cwd bench/web <script>` runs bench's scripts; CI and `bench-deploy.yml` do the same, and `bench/web/vercel.json` gives `vercel build` the same install command. A `bun install` in `bench/web` also finds the root's `bunfig.toml`. `bench/web/package.json` pins the root's `packageManager` (`bun@…`), so pnpm refuses to install there instead of installing around the cooldown; a `renovate.json` rule updates both pins in one branch, and `bench/web/lib/package-manager.test.ts` fails if they differ.
- **Linker.** `bunfig.toml` sets `linker = "isolated"`, so every package gets the dependencies it resolved, as pnpm gave bench. That layout keeps one app's tools from loading the other's copy of a package; both apps now resolve the same TypeScript, so that no longer matters for it.
- **Lifecycle scripts.** pnpm's `allowBuilds` let `sharp` build (and `unrs-resolver`, which left with ESLint). `trustedDependencies` already trusts `sharp`, and stays `["esbuild", "sharp"]`. Bun blocks three bench scripts, `protobufjs`, `ssh2` and `cpu-features`, which pnpm blocked too. With the isolated linker, `bun pm untrusted` lists only packages linked at the top of `node_modules`, so it reports none of them; to audit, install once with `bun install --linker hoisted` in a scratch copy and run it there.
- **Deploy CLI:** `bench-deploy.yml` installs the Vercel CLI from `.github/vercel-cli` with `npm ci --ignore-scripts`. Its `package.json` pins the version, and its `package-lock.json` locks the whole tree. That job holds `VERCEL_TOKEN`. The lockfile is npm's, not Bun's, so GitHub's dependency graph, Dependabot and dependency review read it natively. To change it, run `npm install --package-lock-only --ignore-scripts --before=<7 days ago>` in that directory (Renovate passes the same `--before`), then `npm audit`. Its `overrides` patch advisories in dependencies the CLI pins; see "bench advisories".
Expand Down
16 changes: 10 additions & 6 deletions bench/docs/architecture/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,9 @@ nor by the scroll to a link's first group. Scroll is
restored by element, not by pixel
([`lib/scroll-restore.ts`](../../web/lib/scroll-restore.ts)): the open bodies
mount only after hydration, so a saved `scrollY` would land against a shorter
page.
page. While no group is open the page leaves restoration to the browser
(`history.scrollRestoration` stays `auto`), because the collapsed list is as
tall before hydration as after; it switches to `manual` while a group is open.

Measured against the previous build, the head of `claude/bench-cleanup`
(b4caf7c): a local restore of the production dump, desktop (1440×900), no
Expand All @@ -293,7 +295,7 @@ with one-off scripts that are not in this repository.

Each request to `/` decides how to send the groups
([`app/page.tsx`](../../web/app/page.tsx)). It starts reading them and waits up
to 150 ms (`LOADING_VIEW_AFTER_MS`). A Data Cache hit answers well inside that,
to 300 ms (`LOADING_VIEW_AFTER_MS`). A Data Cache hit answers well inside that,
so the page renders the groups directly, with no Suspense boundary: plain,
visible HTML that a reader with JavaScript off can read in full. A miss gets a
Suspense boundary instead: the "Loading benchmark summaries…" view streams at
Expand Down Expand Up @@ -328,10 +330,12 @@ keep misses rare, and an ingest causes none: it marks the entries stale rather
than dropping them (see "Refresh on ingest"), so the next render of `/` takes
the plain-HTML path with the previous data.

The 150 ms threshold was measured against the local Data Cache, where a hit
answers in a few milliseconds. Check it against Vercel's function logs on a
preview deployment before relying on it: a hit there reads the cache over the
network.
The threshold was first 150 ms, measured against the local Data Cache, where a
hit answers in a few milliseconds. On Vercel a hit reads the cache over the
network: on 2026-10-30, 1 of 12 cache-busted production renders of `/` missed
150 ms and streamed the loading view, which the CDN would then keep for five
minutes. At 300 ms a hit has room, and a miss still shows the loading view
early.

### Visual downsampling (LTTB)

Expand Down
6 changes: 6 additions & 0 deletions bench/docs/architecture/read-path.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,12 @@ the fills never ask for more connections at once than the pool holds. The pool
is never smaller than 2 (`resolvePoolMax`), because a fill queries on a second
connection while the first holds its lock.

A background refresh (a stale read after an ingest, see above) has no retry: if
another invocation holds the entry's lock, it fails at once and Next logs the
`GroupCacheFillInProgressError`. That is expected. The lock holder's fill
refreshes the shared entry, so a few of these errors after each ingest cost
nothing; waiting and filling again would repeat the holder's query.

`bench_read` uses a **static password** rather than RDS IAM auth because the
Vercel runtime has no AWS credentials to mint an IAM token, and RDS `rds_iam`
membership disables password auth. The CI roles (`migrator`, `bench_ingest`) do
Expand Down
12 changes: 6 additions & 6 deletions bench/web/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -93,13 +93,13 @@ export default async function Home({

/**
* How long the landing page waits for its data before it streams the loading
* view instead. A Data Cache hit reads in a few milliseconds, well inside it,
* so the common case stays plain HTML; a miss reads the database, about half a
* second locally and seconds in production after an idle gap. Long enough that
* a busy server's hit still lands inside it, short enough that a reader on a
* miss sees the loading view almost at once.
* view instead. A Data Cache hit reads in a few milliseconds locally, but on
* Vercel it is a network read, and at 150 ms about one hit in twelve missed the
* wait (2026-10-30). The CDN then kept that loading-view copy for five minutes.
* A miss reads the database, about half a second locally and seconds in
* production after an idle gap, so 300 ms still shows the loading view early.
*/
const LOADING_VIEW_AFTER_MS = 150;
const LOADING_VIEW_AFTER_MS = 300;

type SearchParams = Record<string, string | string[] | undefined>;
type LandingData = [SearchParams, Group[], FilterUniverse];
Expand Down
38 changes: 38 additions & 0 deletions bench/web/components/Chart.lazy-hydration.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { Chart } from '@/components/Chart';
import { GroupBody } from '@/components/GroupBody';
import { FETCH_TIMEOUT_MS } from '@/lib/chart-format';
import { PAGE_LEAVE_GRACE_MS } from '@/lib/page-leave';
import { bundleQueue, hydrationQueue, resetPayloadCache } from '@/lib/chart-store';

vi.mock('@/lib/chart-js', () => ({
Expand Down Expand Up @@ -512,6 +513,43 @@ describe('PR-5.0.97 group-bundle hydration', () => {
},
);

// Leaving the page cancels the bundle fetch. WebKit rejects it with a `TypeError` ("Load
// failed") before the page's `pagehide`, the same error as a real network failure.
it('drops a bundle failure quietly when the page is being left', async () => {
vi.useFakeTimers();
let reject: (e: unknown) => void = () => {};
vi.stubGlobal('fetch', (url: string | URL) => {
fetchCalls.push(String(url));
return new Promise<Response>((_resolve, rej) => {
reject = rej;
});
});
await renderGroup(2);
await act(async () => {
MockIO.instances[0]!.fire();
MockIO.instances[1]!.fire();
await vi.advanceTimersByTimeAsync(0);
});
expect(bundleFetchCount()).toBe(1);
await act(async () => {
reject(new TypeError('Load failed'));
await vi.advanceTimersByTimeAsync(0);
window.dispatchEvent(new Event('pagehide'));
await vi.advanceTimersByTimeAsync(PAGE_LEAVE_GRACE_MS * 5);
});
expect(container.querySelectorAll('[data-role="fetch-retry"]')).toHaveLength(0);
expect(container.querySelectorAll('.chart-error')).toHaveLength(0);
expect(chartFetchCount()).toBe(0);

// Back from the back/forward cache: the group did lose its charts, so the retry shows.
await act(async () => {
window.dispatchEvent(new Event('pageshow'));
await vi.advanceTimersByTimeAsync(0);
});
expect(container.querySelectorAll('[data-role="fetch-retry"]')).toHaveLength(2);
expect(chartFetchCount()).toBe(0);
});

it('closing the group aborts the in-flight bundle (its fetch signal aborts)', async () => {
const signals: AbortSignal[] = [];
vi.stubGlobal('fetch', (url: string | URL, init?: { signal?: AbortSignal }) => {
Expand Down
81 changes: 81 additions & 0 deletions bench/web/components/Chart.loading.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';

import { Chart } from '@/components/Chart';
import { FETCH_TIMEOUT_MS } from '@/lib/chart-format';
import { PAGE_LEAVE_GRACE_MS } from '@/lib/page-leave';
import { fullHistoryQueue } from '@/lib/chart-store';
import { loadChartJs } from '@/lib/chart-js';

Expand Down Expand Up @@ -557,6 +558,86 @@ describe('Chart opt-in full-history loading', () => {
});
});

// Leaving the page cancels its fetches. WebKit rejects them with a `TypeError` ("Load failed")
// before the page's `pagehide`, the same error as a real network failure.
describe('a fetch cancelled by leaving the page', () => {
function stubRejectableFetch(urlPart: string): { reject: (e: unknown) => void } {
let rejecter: (e: unknown) => void = () => {};
responders.push({
match: (u) => u.includes(urlPart),
respond: () =>
new Promise<Response>((_res, rej) => {
rejecter = rej;
}),
});
return { reject: (e) => rejecter(e) };
}

async function leaveThenShow(shown: () => void): Promise<void> {
await act(async () => {
window.dispatchEvent(new Event('pagehide'));
await vi.advanceTimersByTimeAsync(PAGE_LEAVE_GRACE_MS * 5);
});
shown();
// Back from the back/forward cache: the page did lose the data, so now it reports.
await act(async () => {
window.dispatchEvent(new Event('pageshow'));
await vi.advanceTimersByTimeAsync(0);
});
}

it('drops the full-history failure quietly', async () => {
const full = stubRejectableFetch('n=all');
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
const chip = await renderOpenGroup();
vi.useFakeTimers();
await act(async () => {
chip?.click();
await vi.advanceTimersByTimeAsync(0);
});
await act(async () => {
full.reject(new TypeError('Load failed'));
await vi.advanceTimersByTimeAsync(0);
});
await leaveThenShow(() => {
expect(warn).not.toHaveBeenCalled();
expect(chip?.dataset.state).not.toBe('error');
});
expect(chip?.dataset.state).toBe('error');
});

it('drops the initial fetch failure quietly', async () => {
vi.useFakeTimers();
const initial = stubRejectableFetch('n=100');
await renderOpenGroup();
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
initial.reject(new TypeError('Load failed'));
await vi.advanceTimersByTimeAsync(0);
});
await leaveThenShow(() => {
expect(container.querySelector('.chart-error')).toBeNull();
});
expect(container.querySelector('.chart-error [data-role="fetch-retry"]')).not.toBeNull();
});

it('still reports a network failure while the page stays', async () => {
vi.useFakeTimers();
const initial = stubRejectableFetch('n=100');
await renderOpenGroup();
await act(async () => {
await vi.advanceTimersByTimeAsync(0);
initial.reject(new TypeError('Failed to fetch'));
await vi.advanceTimersByTimeAsync(PAGE_LEAVE_GRACE_MS - 1);
});
expect(container.querySelector('.chart-error')).toBeNull();
await act(async () => {
await vi.advanceTimersByTimeAsync(1);
});
expect(container.querySelector('.chart-error')?.textContent).toContain('Failed to fetch');
});
});

describe('PR-5.0.95 loading spinner', () => {
it('renders an animated spinner element (not bare text) while loading', async () => {
// A never-resolving window fetch keeps the card in the loading state.
Expand Down
99 changes: 95 additions & 4 deletions bench/web/components/FilterBar.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,17 @@

import { act } from 'react';
import { createRoot, type Root } from 'react-dom/client';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';

import { FilterBar } from '@/components/FilterBar';
import { parseFilterCsv, seriesPassesFilter } from '@/lib/chart-format';
import { getGlobalFilterSnapshot } from '@/lib/chart-store';
import { resetUrlWrites, URL_WRITE_DELAY_MS } from '@/lib/url-writes';

/** Fake the write scheduler's timers and its clock, `performance.now()`. */
function useFakeClock(): void {
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'Date', 'performance'] });
}

const universe = { engines: ['duckdb', 'datafusion'], formats: ['vortex', 'parquet', 'lance'] };

Expand All @@ -19,13 +25,16 @@ describe('global filter URL round trips', () => {

beforeEach(() => {
(globalThis as { IS_REACT_ACT_ENVIRONMENT?: boolean }).IS_REACT_ACT_ENVIRONMENT = true;
resetUrlWrites(window);
container = document.createElement('div');
document.body.appendChild(container);
});

afterEach(async () => {
await act(async () => root.unmount());
container.remove();
vi.useRealTimers();
resetUrlWrites(window);
});

async function mount(): Promise<void> {
Expand All @@ -50,7 +59,16 @@ describe('global filter URL round trips', () => {
});
}

/** The URL once a pending write has landed: chip clicks inside the write delay wait for it. */
async function settledUrl(): Promise<URL> {
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, URL_WRITE_DELAY_MS));
});
return new URL(window.location.href);
}

async function reload(): Promise<void> {
await settledUrl();
await act(async () => root.unmount());
await mount();
}
Expand All @@ -68,15 +86,15 @@ describe('global filter URL round trips', () => {

for (const engine of universe.engines) await click('engine', engine);
for (const format of ['vortex', 'parquet']) await click('format', format);
let url = new URL(window.location.href);
let url = await settledUrl();
expect(url.searchParams.get('engine')).toBe('');
expect(url.searchParams.get('format')).toBe('');
await reload();
expect(getGlobalFilterSnapshot().active).toEqual({ engines: [], formats: [] });

await click('engine', '*');
await click('format', '*');
url = new URL(window.location.href);
url = await settledUrl();
expect(url.searchParams.has('engine')).toBe(false);
expect(url.searchParams.get('format')).toBe('vortex,parquet,lance');
expect(url.searchParams.get('n')).toBe('all');
Expand All @@ -100,8 +118,81 @@ describe('global filter URL round trips', () => {
expect(seriesPassesFilter({ engine: 'duckdb', format: 'lance' }, active, universe)).toBe(false);
expect(container.querySelector('[data-role="filter-badge"]')?.textContent).toBe('3');
await click('engine', '*');
expect(new URL(window.location.href).searchParams.get('format')).toBe('vortex,old,older');
expect((await settledUrl()).searchParams.get('format')).toBe('vortex,old,older');
await reload();
expect(getGlobalFilterSnapshot().active.formats).toEqual(['vortex', 'old', 'older']);
});

// Safari refuses more than 100 `history.replaceState` calls in 10 seconds, and Next.js adds one
// of its own to each of ours, so the chips share the open groups' coalesced writes.
it('writes the URL a few times for a burst of chip clicks, and ends current', async () => {
window.history.replaceState(null, '', '/?n=all#tpc-h-nvme-sf-1');
await mount();
useFakeClock();
const replaceState = vi.spyOn(window.history, 'replaceState');
const writes: number[] = [];
replaceState.mockImplementation(function (this: History, ...args) {
writes.push(Date.now());
return History.prototype.replaceState.apply(this, args);
});

// 200 clicks 25 ms apart: 5 seconds, twice Safari's budget of calls if each one wrote.
for (let i = 0; i < 200; i += 1) {
await click('engine', 'duckdb');
await act(async () => {
await vi.advanceTimersByTimeAsync(25);
});
}
expect(writes.length).toBeLessThanOrEqual(3);
await act(async () => {
await vi.advanceTimersByTimeAsync(URL_WRITE_DELAY_MS);
});
expect(writes.length).toBeLessThanOrEqual(4);

// 200 toggles leave duckdb as it started: on.
const url = new URL(window.location.href);
expect(url.searchParams.has('engine')).toBe(false);
expect(url.searchParams.get('n')).toBe('all');
expect(url.hash).toBe('#tpc-h-nvme-sf-1');

// One more click after a quiet spell writes at once.
await act(async () => {
await vi.advanceTimersByTimeAsync(URL_WRITE_DELAY_MS);
});
await click('engine', 'duckdb');
expect(new URL(window.location.href).searchParams.get('engine')).toBe('datafusion');
});

it('writes a pending chip change when the page is hidden', async () => {
window.history.replaceState(null, '', '/chart/qm.example');
await mount();
useFakeClock();
await click('format', 'vortex');
await click('format', 'parquet');
expect(new URL(window.location.href).searchParams.get('format')).toBe('parquet');

await act(async () => {
window.dispatchEvent(new Event('pagehide'));
});
expect(new URL(window.location.href).searchParams.get('format')).toBe('');
});

it('writes a pending chip change before a same-origin link is followed', async () => {
window.history.replaceState(null, '', '/chart/qm.example');
await mount();
useFakeClock();
await click('format', 'vortex');
await click('format', 'parquet');
expect(new URL(window.location.href).searchParams.get('format')).toBe('parquet');

const link = document.createElement('a');
link.href = '/';
link.addEventListener('click', (event) => event.preventDefault());
document.body.appendChild(link);
await act(async () => {
link.click();
});
link.remove();
expect(new URL(window.location.href).searchParams.get('format')).toBe('');
});
});
Loading
Loading