Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 29 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -459,8 +459,11 @@ While a run goes on, the benchmarks page shows it live. Every run, published con
- **ngrok.** With `NGROK_AUTHTOKEN` set, the server is tunnelled through the static domain in
`live.domain` (or `NGROK_DOMAIN`) with the `@ngrok/ngrok` package, and the public URL is logged.
Without it there is no tunnel. The tunnel and the server close when the run ends.
- **The page.** `project.live.url` (default `https://<live.domain>/live.json`) goes into
report.json, so the latest page (the site root) knows where to look. It polls every 15
- **The page.** The live URL (`project.live.url`, else `live.url`, else
`https://<live.domain>/live.json`) goes into report.json and into the site's `data/site.json`,
so the latest page (the site root) knows where to look, even before the first run is
published (`perfkit render --config`, below). It reads `data/site.json` first, then the latest
report. It polls every 15
seconds; while a run is in progress it shows it live, marked preliminary. When the run ends it
says the report is being published and shows it as soon as `data/runs.json` lists the run.
With no run in progress the page is the latest report, unchanged. Run pages never go live.
Expand Down Expand Up @@ -500,15 +503,31 @@ stopped, 1 for a usage error and 2 when it can't listen (the port is taken).

```sh
perfkit render --report <report.json> [--history <data/runs.json|https URL>] --out <dir>
perfkit render --config <perfkit.yaml> [--history <data/runs.json|https URL>] --out <dir>
```

Renders one run's `report.json` into the project's benchmarks site, a static site the caller
publishes (for example under `/benchmarks` on GitHub Pages). It validates the report first, with
the same error lines as `perfkit validate`.

With `--config` instead of `--report` it adds no run: it writes the site from the project's
`perfkit.yaml` (logo and favicon inlined) and the history as it is. Without `--history` that is
a site with no runs yet: the site root shows the project's title and lead and "No published runs
yet. When a run is in progress it appears here live.", and goes live as soon as a run starts. Use
it to publish (or clear) a project's site before its first published run:

```console
$ npx perfkit render --config perf/perfkit.yaml --out out/benchmarks
wrote out/benchmarks/app/0.1.12/app.js (perfkit 0.1.12)
wrote out/benchmarks/data/runs.json (0 runs)
wrote out/benchmarks/data/site.json (live: https://perfkit.ngrok.app/live.json)
wrote out/benchmarks/index.html (no runs yet)
```

| Flag | Meaning |
| :--- | :--- |
| `--report <file>` | The run's `report.json`. Required. |
| `--report <file>` | The run's `report.json`. This or `--config` is required, not both. |
| `--config <perfkit.yaml>` | Instead of `--report`: write the site from the project's perfkit.yaml with no run added. |
| `--history <file\|url>` | The published `data/runs.json`, as a file or an `https://` URL. A URL that answers 404 is treated as no history. Without it, this is the first run: the page has no deltas and perfkit prints `perfkit: no history: first run, deltas omitted`. |
| `--out <dir>` | The directory to write the site into. Required. |

Expand All @@ -519,6 +538,7 @@ wrote out/benchmarks/app/0.2.0/app.js (perfkit 0.2.0)
wrote out/benchmarks/data/runs/20261103T141210Z-bd3254a/report.json
wrote out/benchmarks/runs/20261103T141210Z-bd3254a/report.json
wrote out/benchmarks/data/runs.json (15 runs)
wrote out/benchmarks/data/site.json (live: https://perfkit.ngrok.app/live.json)
wrote out/benchmarks/index.html (latest: 20261103T141210Z-bd3254a)
wrote out/benchmarks/runs/20261103T141210Z-bd3254a/index.html
wrote out/benchmarks/runs/20261027T093000Z-36bf8f8/index.html
Expand All @@ -532,6 +552,10 @@ files on the same site. The output layout:
position. A run already listed (same id) is replaced, so rendering twice is safe. Point
`--history` at the published copy of this file (a `runs.json` of the old layout, at the site
root, also works).
- `data/site.json` (`schema: perfkit/site/v1`, JSON Schema in `schemas/site-v1.schema.json`):
the project, as report.json carries it: name, repo, site URL, accent, logo and favicon SVGs,
title, lead and live URL. Every render rewrites it, from the report or from `--config`; the
app draws the no-runs page from it and reads the live URL from it first.
- `data/runs/<id>/report.json`: this run's report, as read; the app draws the page from it. A
copy also goes to `runs/<id>/report.json`, where the page's `report.json` link points.
- `app/<version>/app.js`: this perfkit's browser app, the same renderer `perfkit` has always used,
Expand All @@ -552,8 +576,8 @@ compare with, is in [docs/report-page.md](docs/report-page.md).
| Exit code | Meaning |
| :--- | :--- |
| 0 | The site was written. |
| 1 | Usage error, such as a missing `--out`. |
| 2 | The report or the history is invalid, unreadable or of another schema. |
| 1 | Usage error, such as a missing `--out`, or neither or both of `--report` and `--config`. |
| 2 | The report, the config or the history is invalid, unreadable or of another schema. |
| 3 | The history URL could not be fetched (any error but a 404). |

## Development
Expand Down
3 changes: 3 additions & 0 deletions docs/report-page.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ the history shows the runs before it.

## Pages

- A site with no published runs yet (`perfkit render --config`) has only `/benchmarks/`: the
project's title and lead and "No published runs yet. When a run is in progress it appears
here live.", drawn from `data/site.json`. It goes live like the latest page.
- `/benchmarks/` is the latest run, the one with the newest `startedAt`. Its run picker marks it
`latest`, and "Link to this run" points to its own page.
- `/benchmarks/runs/<id>/` is one run's page. It starts with "You're viewing the run of …" and a
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "perfkit",
"version": "0.1.11",
"version": "0.1.12",
"description": "Benchmark harness for Fonoster projects: DigitalOcean runs, one report.json schema, one branded report page",
"license": "MIT",
"author": "Pedro Sanders <psanders@fonoster.com>",
Expand Down
76 changes: 76 additions & 0 deletions schemas/site-v1.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"schema": {
"type": "string",
"const": "perfkit/site/v1"
},
"project": {
"type": "object",
"properties": {
"name": {
"type": "string",
"minLength": 1
},
"repo": {
"type": "string",
"pattern": "^[\\w.-]+\\/[\\w.-]+$"
},
"siteUrl": {
"type": "string",
"format": "uri"
},
"accent": {
"type": "string",
"pattern": "^#[0-9A-Fa-f]{6}$"
},
"logoSvg": {
"type": "string",
"pattern": "<svg[\\s>]"
},
"faviconSvg": {
"type": "string",
"pattern": "<svg[\\s>]"
},
"title": {
"type": "string",
"minLength": 1
},
"lead": {
"type": "string",
"minLength": 1
},
"live": {
"type": "object",
"properties": {
"url": {
"type": "string",
"format": "uri"
}
},
"required": [
"url"
],
"additionalProperties": false
}
},
"required": [
"name",
"repo",
"siteUrl",
"accent",
"title",
"lead"
],
"additionalProperties": false
}
},
"required": [
"schema",
"project"
],
"additionalProperties": false,
"title": "site.json v1",
"description": "The project of a perfkit benchmarks site: its name, branding and live.json URL (schema: perfkit/site/v1)."
}
31 changes: 27 additions & 4 deletions src/cli/runCli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -100,15 +100,22 @@ Writes to --out:
runs/<id>/report.json this run's report, for the page's link
data/runs.json the history, newest first, with this run in it
data/runs/<id>/report.json this run's report, which the app draws
data/site.json the project: name, branding, live.json URL
app/<version>/app.js this perfkit's app, which draws every page

With --config instead of --report, no run is added: the project comes from
perfkit.yaml and the history is kept as it is. Without --history that is a
site with no runs yet, which says so and shows a run in progress live:

perfkit render --config perf/perfkit.yaml --out out/benchmarks

The pages are drawn in the browser from data/, so older reports are never
read or rewritten, and publishing with a newer perfkit redraws every run.

Exit codes:
0 the site was written
1 usage error
2 the report or the history is invalid, unreadable or of another schema
2 the report, the config or the history is invalid, unreadable or of another schema
3 the history URL could not be fetched (a 404 means no history)`;

const SERVE_HELP = `
Expand Down Expand Up @@ -305,17 +312,33 @@ export async function runCli(argv: string[], deps: CliDeps): Promise<number> {
program
.command("render")
.description(
"Add a report.json to the benchmarks site: its data, the page shells and the app that draws them."
"Add a report.json to the benchmarks site (or, with --config alone, write the site with no new run): its data, the page shells and the app that draws them."
)
.option("--report <file>", "the run's report.json")
.option(
"--config <perfkit.yaml>",
"instead of --report: write the site from the project's perfkit.yaml and the history as it is, with no run added (no runs yet without --history)"
)
.requiredOption("--report <file>", "the run's report.json")
.option(
"--history <file|url>",
"the published data/runs.json (or a runs.json of the old layout), as a file or an https URL; without it, this is the first run"
)
.requiredOption("--out <dir>", "the directory to write the site into")
.addHelpText("after", RENDER_HELP)
.action(
async (options: { report: string; history?: string; out: string }) => {
async (options: {
report?: string;
config?: string;
history?: string;
out: string;
}) => {
if ((options.report === undefined) === (options.config === undefined)) {
deps.stderr(
"perfkit: render needs --report <file> or --config <perfkit.yaml>, not both\n"
);
exitCode = EXIT_USAGE;
return;
}
const result = await renderSite(options);
if (!result.ok) {
result.errors.forEach((line) => deps.stderr(`${line}\n`));
Expand Down
19 changes: 4 additions & 15 deletions src/orchestrate/toReportProject.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/**
* Copyright (C) 2026 by Fonoster Inc (https://fonoster.com). MIT License.
*/
import { projectFromConfig } from "../schemas/projectFromConfig.js";
import type { Report } from "../schemas/index.js";
import type { RunState } from "./runState.schema.js";

Expand All @@ -10,20 +11,8 @@ import type { RunState } from "./runState.schema.js";
* domain in `live.domain`, else there is none.
*/
export function toReportProject(state: RunState): Report["project"] {
const { project, live } = state.config;
const liveUrl =
project.live?.url ??
live?.url ??
(live?.domain ? `https://${live.domain}/live.json` : null);
return {
name: project.name,
repo: project.repo,
siteUrl: project.siteUrl,
accent: project.accent,
return projectFromConfig(state.config, {
...(state.logoSvg ? { logoSvg: state.logoSvg } : {}),
...(state.faviconSvg ? { faviconSvg: state.faviconSvg } : {}),
title: project.title,
lead: project.lead,
...(liveUrl ? { live: { url: liveUrl } } : {})
};
...(state.faviconSvg ? { faviconSvg: state.faviconSvg } : {})
});
}
32 changes: 26 additions & 6 deletions src/render/app/createDrawRun.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,26 @@
import type { z } from "zod/v4";
import { toFieldErrors } from "../../errors/toFieldErrors.js";
import { withErrorHandlingAndValidation } from "../../errors/withErrorHandlingAndValidation.js";
import { reportSchema, runsSchema } from "../../schemas/index.js";
import { reportSchema, runsSchema, siteSchema } from "../../schemas/index.js";
import { renderEmptyPage } from "../renderEmptyPage.js";
import { renderRunPage } from "../renderRunPage.js";
import { toPageContext } from "../toPageContext.js";
import { type DrawRunInput, drawRunInputSchema } from "./drawRun.schema.js";
import { resolveRunPath } from "./resolveRunPath.js";
import { SITE_PATHS } from "./sitePaths.js";
import type { DrawRunDeps, DrawRunResult } from "./types.js";

type Fetched<T> = { ok: true; value: T } | { ok: false; errors: string[] };
type Fetched<T> =
{ ok: true; value: T } | { ok: false; errors: string[]; missing?: true };

/**
* Creates the browser app's draw step: from the page's path it picks the run,
* fetches `data/runs.json` and the run's report.json, validates both with
* the same schemas as `perfkit validate`, and returns the page HTML that
* `perfkit render` used to write, from the same renderer. A report missing
* from `data/runs/<id>/` is read from `runs/<id>/report.json`, where
* sites rendered before the app kept it.
* sites rendered before the app kept it. The latest page of a site with no
* runs yet is drawn from `data/site.json`, the project alone.
*
* @param deps - The app's version and a fetch relative to the site root
* @returns A validated function that returns the page HTML, or error lines
Expand All @@ -32,11 +35,27 @@ export function createDrawRun(deps: DrawRunDeps) {
return { ok: false, errors: [`perfkit: no run page at ${path}`] };
}
const runs = await fetchValid(SITE_PATHS.runs, runsSchema);
if (!runs.ok) return runs;
if (!runs.ok) return { ok: false, errors: runs.errors };
const current =
target.page === "latest"
? runs.value.runs[0]
: runs.value.runs.find((run) => run.id === target.id);
if (current === undefined && target.page === "latest") {
const site = await fetchValid(SITE_PATHS.site, siteSchema);
if (!site.ok && site.missing) {
return { ok: false, errors: [`${SITE_PATHS.runs}: no runs yet`] };
}
if (!site.ok) return { ok: false, errors: site.errors };
return {
ok: true,
runId: null,
html: renderEmptyPage({
project: site.value.project,
perfkitVersion: deps.version
}),
liveUrl: site.value.project.live?.url ?? null
};
}
if (current === undefined) {
return {
ok: false,
Expand All @@ -52,7 +71,7 @@ export function createDrawRun(deps: DrawRunDeps) {
reportSchema,
`${current.path}report.json`
);
if (!report.ok) return report;
if (!report.ok) return { ok: false, errors: report.errors };
const html = renderRunPage(
toPageContext({
report: report.value,
Expand Down Expand Up @@ -92,7 +111,8 @@ export function createDrawRun(deps: DrawRunDeps) {
if (response.status < 200 || response.status > 299) {
return {
ok: false,
errors: [`${path}: cannot fetch: HTTP ${response.status}`]
errors: [`${path}: cannot fetch: HTTP ${response.status}`],
...(response.status === 404 ? { missing: true as const } : {})
};
}
let document: unknown;
Expand Down
28 changes: 28 additions & 0 deletions src/render/app/createReadSiteLiveUrl.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/**
* Copyright (C) 2026 by Fonoster Inc (https://fonoster.com). MIT License.
*/
import { siteSchema } from "../../schemas/index.js";
import { SITE_PATHS } from "./sitePaths.js";
import type { DrawRunDeps } from "./types.js";

/**
* Creates the app's read of the project's live.json URL from
* `data/site.json`. A site rendered before site.json existed, or an invalid
* or unreachable one, has none: the app then falls back to the latest
* report's.
*
* @param deps - A fetch relative to the site root
* @returns A function that returns the URL or null; it never rejects
*/
export function createReadSiteLiveUrl(deps: Pick<DrawRunDeps, "fetchText">) {
return async (): Promise<string | null> => {
try {
const response = await deps.fetchText(SITE_PATHS.site);
if (response.status !== 200) return null;
const parsed = siteSchema.safeParse(JSON.parse(response.text));
return parsed.success ? (parsed.data.project.live?.url ?? null) : null;
} catch {
return null;
}
};
}
Loading
Loading