Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions crm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,13 @@ One shared password, per-login sessions, one Railway service, no database.
| Overview | PostHog, blob, Prometheus | visitors, pageviews, sessions (7 d vs previous 7 d), **AI-referred visitors** (ChatGPT, Perplexity, Claude, Gemini, Copilot, …), search-referred visitors, 28 d daily series, 12 w weekly series with the AI share, channels, AI domains, sections |
| Pages | PostHog | sections week over week, biggest gains and losses, top 100 pages (filter by section), entry pages |
| Audience | PostHog | new vs returning, bounce, countries, devices, UTM sources, referring domains with their channel |
| Actions | PostHog custom events | outbound clicks by destination host (visitors sent to providers), copies (endpoint, API URL, MCP, embed) per bench, search queries with the result picked |
| Data health | index blob, Prometheus, Dune | live / stale (> 24 h) / expired (> 7 d) benches per category, benches needing attention, scrape targets down, Dune credits and period end, the daily history kept on the volume |

The site captures `$pageview` and `$pageleave` only (autocapture off, nobody
identified); every query reads `$pageview`, so every traffic number is a
pageview aggregate and a visitor is a device cookie. Events from staging and
The site captures `$pageview`, `$pageleave` and three custom events
(`outbound_click`, `copy`, `search`, see `src/lib/analytics.ts`; autocapture
off, nobody identified). Traffic pages read `$pageview`, the Actions page the
custom events; a visitor is a device cookie. Events from staging and
localhost are excluded (`properties.$host`).

Bench health reads `aggregate/index.json` (every bench with its status and
Expand All @@ -27,10 +29,10 @@ from the sitemap before publishing, which is exactly what this page must show.
PostHog allows **2400 query requests per hour per organisation**, shared by
every key and every team member. This app never queries in the request path:

- a refresh runs a **fixed list of 11 HogQL queries**, one at a time
- a refresh runs a **fixed list of 15 HogQL queries**, one at a time
(`lib/traffic.ts`), and writes a snapshot; pages read the snapshot;
- the scheduler (`instrumentation.ts`) refreshes every `REFRESH_MINUTES`
(default 60): **11 queries per hour, about 0.5 % of the organisation's
(default 60): **15 queries per hour, about 0.6 % of the organisation's
budget**;
- the Refresh button is refused for 10 minutes after any refresh;
- a local budget (`POSTHOG_HOURLY_BUDGET`, default 300 per rolling hour) is a
Expand Down
148 changes: 148 additions & 0 deletions crm/app/actions/page.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
import { Shell } from "@/components/shell";
import { Delta, Empty, fmtInt, Kpi } from "@/components/ui";
import { readSnapshot } from "@/lib/snapshot";

export const dynamic = "force-dynamic";
const SITE = `https://${process.env.SITE_HOST ?? "openchainbench.com"}`;

const ACTION_LABEL: Record<string, string> = {
outbound_click: "Outbound clicks",
copy: "Copies",
search: "Searches",
};

export default async function ActionsPage({ searchParams }: { searchParams: Promise<{ refresh?: string }> }) {
const [snap, sp] = await Promise.all([readSnapshot(), searchParams]);
const t = snap.traffic;
const actions = t.actions ?? [];
const byName = (n: string) => actions.find((a) => a.name === n);
const outbound = (t.outbound ?? []).filter((o) => o.clicks > 0 || o.prevClicks > 0);
const copies = t.copies ?? [];
const searches = t.searches ?? [];

return (
<Shell current="/actions" snapshot={snap} refreshFlag={sp.refresh}>
<section className="grid grid-cols-2 gap-3 md:grid-cols-3">
{(["outbound_click", "copy", "search"] as const).map((n) => {
const a = byName(n);
return (
<Kpi
key={n}
label={`${ACTION_LABEL[n]}, 7 d`}
value={fmtInt(a?.count)}
delta={a && { now: a.count, prev: a.prevCount }}
sub={a ? `${fmtInt(a.visitors)} visitors` : "no event yet"}
/>
);
})}
</section>
<p className="mt-3 text-xs" style={{ color: "var(--faint)" }}>
Events the site sends on top of pageviews (src/lib/analytics.ts). Nothing here before the site deploy that added them; the
Actions numbers are what the traffic turns into.
</p>

<section className="mt-6 grid gap-3 md:grid-cols-2">
<div className="panel p-4">
<p className="label">Where visitors go, 7 d (outbound clicks by host)</p>
{outbound.length > 0 ? (
<table className="data mt-2">
<thead>
<tr>
<th>Host</th>
<th className="num">Clicks</th>
<th className="num">w/w</th>
<th className="num">Visitors</th>
<th>From</th>
</tr>
</thead>
<tbody>
{outbound.map((o) => (
<tr key={o.host}>
<td className="mono">{o.host}</td>
<td className="num mono">{fmtInt(o.clicks)}</td>
<td className="num mono">
<Delta now={o.clicks} prev={o.prevClicks} />
</td>
<td className="num mono">{fmtInt(o.visitors)}</td>
<td className="mono truncate" style={{ maxWidth: 200, color: "var(--muted)" }} title={o.topPage}>
{o.topPage}
</td>
</tr>
))}
</tbody>
</table>
) : (
<Empty text="No outbound click recorded yet." />
)}
</div>
<div className="panel p-4">
<p className="label">What gets copied, 7 d</p>
{copies.length > 0 ? (
<table className="data mt-2">
<thead>
<tr>
<th>Kind</th>
<th>Value</th>
<th>Bench</th>
<th className="num">Copies</th>
</tr>
</thead>
<tbody>
{copies.map((c, i) => (
<tr key={`${c.kind}/${c.value}/${c.bench}/${i}`}>
<td>{c.kind}</td>
<td className="mono truncate" style={{ maxWidth: 260 }} title={c.value}>
{c.value || "–"}
</td>
<td className="mono" style={{ color: "var(--muted)" }}>
{c.bench ? (
<a href={`${SITE}/benchmarks/${c.bench}`} target="_blank" rel="noreferrer" className="underline-offset-2 hover:underline">
{c.bench}
</a>
) : (
"–"
)}
</td>
<td className="num mono">{fmtInt(c.count)}</td>
</tr>
))}
</tbody>
</table>
) : (
<Empty text="No copy recorded yet." />
)}
</div>
</section>

<section className="panel mt-6 p-4">
<p className="label">What people search for, 7 d (a result was picked)</p>
{searches.length > 0 ? (
<table className="data mt-2">
<thead>
<tr>
<th>Query</th>
<th className="num">Times</th>
<th>Kind</th>
<th>Landed on</th>
</tr>
</thead>
<tbody>
{searches.map((s) => (
<tr key={s.query}>
<td className="mono">{s.query}</td>
<td className="num mono">{fmtInt(s.count)}</td>
<td style={{ color: "var(--muted)" }}>{s.kind || "–"}</td>
<td className="mono" style={{ color: "var(--muted)" }}>
{s.url}
</td>
</tr>
))}
</tbody>
</table>
) : (
<Empty text="No search recorded yet." />
)}
</section>
</Shell>
);
}
1 change: 1 addition & 0 deletions crm/components/shell.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ const NAV = [
["/", "Overview"],
["/pages", "Pages"],
["/audience", "Audience"],
["/actions", "Actions"],
["/health", "Data health"],
] as const;

Expand Down
43 changes: 41 additions & 2 deletions crm/lib/traffic.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
/**
* The PostHog side of the snapshot: one fixed list of HogQL queries per
* refresh (eleven today), each mapped to a plain JSON section. Every query is
* refresh (fifteen today), each mapped to a plain JSON section. Every query is
* scoped to the production host, so staging and localhost never count, and
* to `$pageview`, the only event the site captures today (autocapture is off).
* to one named event: `$pageview` for the traffic sections, the three custom
* events of src/lib/analytics.ts for the Actions sections (autocapture is off).
*
* Distinct id, not person id: the site runs `person_profiles: identified_only`
* and never identifies anyone, so a visitor is a device cookie.
Expand All @@ -13,13 +14,19 @@ import { num, queryHogQL, str } from "@/lib/posthog";
const SITE_HOST = process.env.SITE_HOST ?? "openchainbench.com";
const HOST_FILTER = `properties.$host = '${SITE_HOST}'`;
const PV = `event = '$pageview' AND ${HOST_FILTER}`;
// The site's custom events (src/lib/analytics.ts): outbound_click, copy, search.
const CUSTOM = `event IN ('outbound_click', 'copy', 'search') AND ${HOST_FILTER}`;

export type DailyPoint = { day: string; pageviews: number; visitors: number; sessions: number };
export type WeeklyPoint = { week: string; visitors: number; ai: number; search: number; pageviews: number };
export type PageRow = { path: string; section: Section; visitors: number; prevVisitors: number; pageviews: number };
export type ReferrerRow = { domain: string; channel: Channel; visitors: number; prevVisitors: number; pageviews: number };
export type NamedCount = { name: string; visitors: number; share: number };
export type EntryRow = { path: string; section: Section; sessions: number };
export type ActionRow = { name: string; count: number; prevCount: number; visitors: number };
export type OutboundRow = { host: string; clicks: number; prevClicks: number; visitors: number; topPage: string };
export type SearchRow = { query: string; count: number; kind: string; url: string };
export type CopyRow = { kind: string; value: string; bench: string; count: number };

export type Traffic = {
daily: DailyPoint[];
Expand All @@ -45,6 +52,10 @@ export type Traffic = {
prevSearchVisitors: number;
};
engagement: { pagesPerSession: number; bounceRate: number; sessions: number };
actions: ActionRow[];
outbound: OutboundRow[];
searches: SearchRow[];
copies: CopyRow[];
};

export const QUERIES = {
Expand Down Expand Up @@ -119,6 +130,24 @@ export const QUERIES = {
SELECT properties.$session_id AS s, count() AS n
FROM events WHERE ${PV} AND timestamp >= now() - INTERVAL 7 DAY AND s IS NOT NULL GROUP BY s
)`,
actions: () => `
SELECT event, countIf(timestamp >= now() - INTERVAL 7 DAY) AS n, countIf(timestamp < now() - INTERVAL 7 DAY) AS prev_n,
uniqIf(distinct_id, timestamp >= now() - INTERVAL 7 DAY) AS visitors
FROM events WHERE ${CUSTOM} AND timestamp >= now() - INTERVAL 14 DAY
GROUP BY event ORDER BY n DESC`,
outbound: () => `
SELECT properties.host AS host, countIf(timestamp >= now() - INTERVAL 7 DAY) AS clicks, countIf(timestamp < now() - INTERVAL 7 DAY) AS prev_clicks,
uniqIf(distinct_id, timestamp >= now() - INTERVAL 7 DAY) AS visitors, topK(1)(properties.page) AS top_page
FROM events WHERE event = 'outbound_click' AND ${HOST_FILTER} AND timestamp >= now() - INTERVAL 14 DAY
GROUP BY host ORDER BY greatest(clicks, prev_clicks) DESC LIMIT 40`,
searches: () => `
SELECT lower(properties.query) AS q, count() AS n, topK(1)(properties.kind) AS kind, topK(1)(properties.url) AS url
FROM events WHERE event = 'search' AND ${HOST_FILTER} AND timestamp >= now() - INTERVAL 7 DAY AND q != ''
GROUP BY q ORDER BY n DESC LIMIT 40`,
copies: () => `
SELECT properties.kind AS kind, properties.value AS value, properties.bench AS bench, count() AS n
FROM events WHERE event = 'copy' AND ${HOST_FILTER} AND timestamp >= now() - INTERVAL 7 DAY
GROUP BY kind, value, bench ORDER BY n DESC LIMIT 40`,
} as const;

export type TrafficSection = keyof typeof QUERIES;
Expand Down Expand Up @@ -165,6 +194,16 @@ export async function loadTrafficSection(section: TrafficSection): Promise<Parti
};
case "audience":
return { audience: { newVisitors: num(rows[0]?.[0]), returningVisitors: num(rows[0]?.[1]) } };
case "actions":
return { actions: rows.map((r) => ({ name: str(r[0]), count: num(r[1]), prevCount: num(r[2]), visitors: num(r[3]) })) };
case "outbound":
return {
outbound: rows.map((r) => ({ host: str(r[0]) || "?", clicks: num(r[1]), prevClicks: num(r[2]), visitors: num(r[3]), topPage: str(Array.isArray(r[4]) ? r[4][0] : r[4]) })),
};
case "searches":
return { searches: rows.map((r) => ({ query: str(r[0]), count: num(r[1]), kind: str(Array.isArray(r[2]) ? r[2][0] : r[2]), url: str(Array.isArray(r[3]) ? r[3][0] : r[3]) })) };
case "copies":
return { copies: rows.map((r) => ({ kind: str(r[0]) || "other", value: str(r[1]), bench: str(r[2]), count: num(r[3]) })) };
case "engagement":
return { engagement: { pagesPerSession: num(rows[0]?.[0]), bounceRate: num(rows[0]?.[1]), sessions: num(rows[0]?.[2]) } };
}
Expand Down
6 changes: 3 additions & 3 deletions crm/test/traffic.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@ import { describe, expect, test } from "bun:test";
import { channelTotals, QUERIES, sectionTotals, sumWindow, TRAFFIC_SECTIONS } from "../lib/traffic";

describe("queries", () => {
test("every query is scoped to pageviews on the production host", () => {
test("every query is scoped to the production host and to a named event", () => {
for (const name of TRAFFIC_SECTIONS) {
const q = QUERIES[name]();
expect(q).toContain("event = '$pageview'");
expect(q).toContain("properties.$host = 'openchainbench.com'");
expect(/event (= '\$pageview'|= '(outbound_click|search|copy)'|IN \('outbound_click', 'copy', 'search'\))/.test(q)).toBe(true);
}
});
test("the refresh spends a bounded number of queries", () => {
expect(TRAFFIC_SECTIONS.length).toBeLessThanOrEqual(12);
expect(TRAFFIC_SECTIONS.length).toBeLessThanOrEqual(16);
});
test("the weekly series and the totals embed the AI domain list", () => {
expect(QUERIES.weekly()).toContain("'chatgpt.com'");
Expand Down
12 changes: 6 additions & 6 deletions src/app/mcp/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ export default async function McpPage() {
<code className="font-mono text-sm text-ink break-all">
{MCP_URL}
</code>
<CopyButton value={MCP_URL} label="Copy URL" />
<CopyButton value={MCP_URL} label="Copy URL" event={{ kind: "mcp_url" }} />
</div>
</div>

Expand All @@ -177,7 +177,7 @@ export default async function McpPage() {
</code>{" "}
(macOS) or the equivalent on your OS, then restart the app.
</p>
<CodeBlock value={CLAUDE_DESKTOP_CONFIG} />
<CodeBlock value={CLAUDE_DESKTOP_CONFIG} name="claude_desktop" />
<p className="mt-3 text-xs text-ink-muted leading-relaxed">
Once connected, the three tools appear under the 🔌 icon in the
chat input. Ask Claude{" "}
Expand Down Expand Up @@ -205,7 +205,7 @@ export default async function McpPage() {
</code>
:
</p>
<CodeBlock value={CURSOR_CONFIG} />
<CodeBlock value={CURSOR_CONFIG} name="cursor" />
</section>

{/* Other clients */}
Expand All @@ -226,7 +226,7 @@ export default async function McpPage() {
: all accept the same URL with the streamable-HTTP transport. SSE
is intentionally disabled. Anything else, raw curl works:
</p>
<CodeBlock value={CURL_EXAMPLE} />
<CodeBlock value={CURL_EXAMPLE} name="curl" />
</section>

{/* What's exposed */}
Expand Down Expand Up @@ -337,14 +337,14 @@ export default async function McpPage() {
);
}

function CodeBlock({ value }: { value: string }) {
function CodeBlock({ value, name }: { value: string; name: string }) {
return (
<div className="mt-4 relative">
<pre className="overflow-x-auto border border-ink/20 bg-ink/5 px-4 py-3 font-mono text-[11px] sm:text-xs text-ink leading-relaxed">
<code>{value}</code>
</pre>
<div className="mt-2 flex justify-end">
<CopyButton value={value} label="Copy" />
<CopyButton value={value} label="Copy" event={{ kind: "mcp_config", value: name }} />
</div>
</div>
);
Expand Down
4 changes: 2 additions & 2 deletions src/components/ai-brief-block.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ export function AiBriefBlock() {
tag="any LLM"
title="Web brief"
desc="Plain markdown. Paste into ChatGPT, Claude, Cursor, Aider, Codex, Continue or any chat-based agent."
action={<CopyButton value="https://openchainbench.com/contribute/ai-brief.md" label="Copy brief URL" />}
action={<CopyButton value="https://openchainbench.com/contribute/ai-brief.md" label="Copy brief URL" event={{ kind: "brief", value: "url" }} />}
link={{ label: "View raw", href: "/contribute/ai-brief.md" }}
/>
<Tile
tag="Claude Code"
title="ClawHub skill"
desc="One-line install with the ClawHub CLI. The skill auto-loads as a slash command and applies the conventions on every reply."
code="openclaw skills install openchainbench-contributor"
action={<CopyButton value="openclaw skills install openchainbench-contributor" label="Copy install" mono />}
action={<CopyButton value="openclaw skills install openchainbench-contributor" label="Copy install" mono event={{ kind: "brief", value: "install" }} />}
link={{ label: "ClawHub listing ↗", href: "https://clawhub.ai/skills/openchainbench-contributor", external: true }}
/>
<p className="text-xs text-ink-muted leading-relaxed">
Expand Down
1 change: 1 addition & 0 deletions src/components/badges-catalog.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,7 @@ function EmbedModal({
<CopyButton
value={currentSnippet}
label={`Copy ${active}`}
event={{ kind: "embed", value: active, bench: pair.benchSlug }}
/>
</div>
</div>
Expand Down
2 changes: 2 additions & 0 deletions src/components/citation-bar.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import { useState } from "react";
import { Check, Copy } from "lucide-react";
import { track } from "@/lib/analytics";

const ORIGIN = "https://openchainbench.com";

Expand All @@ -21,6 +22,7 @@ export function CitationBar({ slug }: { slug: string }) {
async function onCopy() {
try {
await navigator.clipboard.writeText(apiUrl);
track("copy", { kind: "api_url", bench: slug });
setCopied(true);
window.setTimeout(() => setCopied(false), 1400);
} catch {
Expand Down
Loading
Loading