Nightly Lighthouse (PageSpeed Insights), weekly Chrome UX Report field evidence, and agent-readiness monitoring for a watchlist of priority Webflow.com pages. For each page it tracks per-strategy (mobile + desktop) scores over time, classifies status, surfaces recommendations, lets you triage them into tasks, log change markers, and posts drop alerts and 2/7/30-day follow-up comparisons to Slack.
Built with Next.js (App Router) + React. TypeScript throughout.
npm install
npm run dev # http://localhost:3000 (the bundled launch config uses 3100)Other scripts:
npm run build # production build
npm start # serve the production build
npm run lint # eslint
npm test # vitest (unit + integration-focused concurrency tests)
npm run collector:check # type-check the durable Workflow worker
npm run collector:dry-run # bundle/validate the Workflow without deployingAll are optional for local development — the app runs without them.
| Variable | Purpose |
|---|---|
PAGESPEED_API_KEY |
PSI credential. Configure it on the collector Worker; it is also used by the local runner. |
CRUX_API_KEY |
Google Cloud key with the Chrome UX Report API enabled; required by the collector Worker. |
CRON_SECRET |
Existing shared bearer secret for FDE state/report access, Workflow dispatch, and manual nightly requests. |
COLLECTOR_URL |
Production Workflow endpoint, ending in /jobs. |
FDE_DATA_URL |
Optional FDE Worker base URL. If omitted, it is derived from COLLECTOR_URL. |
STORAGE_DRIVER |
Set to remote only after the one-time FDE copy verifies; unset keeps Webflow storage as the source. |
DATASET_MODE |
demo uses the existing sample namespace; live uses an isolated, initially empty namespace. |
BASE_URL |
Webflow Cloud mount path (for example /page-watch); client routes and APIs are prefixed automatically. |
SLACK_WEBHOOK_URL |
Incoming webhook for drop alerts and follow-up reports. |
PSI_MOCK |
Local-only deterministic scores instead of PSI. |
PSI_RUNS |
Samples per strategy (1–5, default 5). |
ANTHROPIC_API_KEY |
Enables post-commit recommendation explanations and Watcher narratives on the app. |
ANTHROPIC_MOCK |
Uses deterministic placeholder AI text without making Anthropic requests. |
WEBFLOW_TOKEN_ENCRYPTION_KEY |
Collector-only base64 AES-256 key used to encrypt tenant Webflow site tokens before D1 persistence. |
Put these in .env.local.
- Collection — baseline, on-demand, and nightly actions reserve a durable
job in FDE D1 and return
202. A Cloudflare Workflow performs up to five staggered PSI samples per strategy and stores every successful raw response, Lighthouse warning, and normalized failing audit in FDE R2. Cached/replayed responses are retained for diagnosis but do not count as independent samples. Warned runs are also excluded from trusted medians. A collection needs at least three unique, warning-free measurements before it can commit; otherwise the Workflow sleeps durably and retries up to twice at one-hour intervals. Recommendations require a strict-majority Lighthouse finding across those eligible runs. The Workflow also scans agent readiness and commits the completed result directly into FDE storage. The SSO-protected Webflow app only makes authenticated outbound requests; the Worker never has to call into the Webflow Access tenant. Retries are durable, duplicates coalesce, stale jobs become visible failures, and a run ID can append history only once. - Baselines — ordinary on-demand/nightly runs may store snapshots, history, recommendations, and scan results, but a page stays Pending until the user explicitly captures a baseline. Zero placeholders are never treated or shown as real baselines.
- Scheduled collection — every 15 minutes, the FDE Worker reads the live
watchlist from its own D1 and dispatches only pages due in the workspace's
saved local-time/timezone window. Active pages receive stable 15-minute
offsets, and individual PSI samples are spaced one minute apart.
POST /nightlystill provides a force-all operation for authenticated manual tests. There is no Webflow callback and no Cloudflare Access service token. - Weekly data audit — each Monday at 05:30 UTC, the Worker reconciles the prior seven days of stored scores against their raw PSI responses. It checks raw-report coverage, usable sample counts, Lighthouse warnings/runtime errors, score medians/ranges, and finding quorum. The saved health report has hashed page references and no customer names, URLs, raw payloads, or raw errors. See audits/README.md.
- Weekly field evidence — each Tuesday at 06:15 UTC, the Worker queries the
CrUX History API for phone and desktop evidence on every active page. It
prefers exact-URL data, falls back to origin data only when URL evidence is
unavailable, stores rolling 28-day p75 metrics and histograms in dedicated
D1 tables, and retains the complete provider response in R2. An authenticated
POST /crux/collectruns the same collection manually. - Storage — the production source of truth is the FDE-owned
page-watcher-fdeD1 database plus thepage-watcher-reportsR2 bucket. The Webflow app uses a tenant-scoped remoteDataStore; D1 state updates use version-guarded compare-and-swap retries. Local development continues to use.data/. The Webflow-provisioned D1/R2 bindings remain intact as a reversible migration source and are not deleted by the migration. - State mutations go through targeted server-side domain endpoints
(
/api/pages,/api/recs,/api/pages/[id]/*) and the store-level atomic update primitive. External PSI/Slack work happens outside that critical section; result commits re-read authoritative state. - Background execution — Cloudflare Workflows own production execution.
The local runner uses Next.js
after()only for development. - Post-commit enrichment — after the scores/raw reports are safely stored, optional Anthropic recommendation explanations, Watcher narrative refreshes, Slack alerts, and due follow-ups cannot roll back or mislabel a successful collection.
- Webflow Enterprise activity — Settings can connect one Enterprise site
per tenant with a site token scoped to
sites:read,site_activity:read,pages:read,assets:read, andcms:read. The collector validates every capability, encrypts the token with AES-GCM before D1 persistence, imports normalized actor/resource activity into D1, retains bounded raw events in R2, and refreshes activity on the existing 15-minute scheduler. Webflow failures are recorded on the connection and never block scheduled PSI. See docs/webflow-activity.md for the product contract and remaining publish-verification phases.
The interactive app relies on the enclosing site's SSO. There is no second
HTTP Basic layer and no FDE_ACCESS_* configuration. Non-interactive nightly
and collector result endpoints remain protected by CRON_SECRET.
-
Apply
migrations/to the FDE-ownedpage-watcher-fdedatabase, then deploycollector-worker/wrangler.jsonc. The Worker needsPAGESPEED_API_KEY,CRUX_API_KEY,CRON_SECRET, and a base64-encoded 32-byteWEBFLOW_TOKEN_ENCRYPTION_KEYgenerated withopenssl rand -base64 32. Its D1, R2, Workflow, 15-minute due-page/Webflow-activity scheduler, Monday 05:30 UTC audit, and Tuesday 06:15 UTC CrUX Cron bindings are declared in that config. -
Deploy the Webflow app code with
STORAGE_DRIVERstill unset. The app keeps reading and writing its existing Webflow-provisioned D1/R2 bindings at this stage. -
While signed into the app through SSO, make this same-origin request from the browser console:
fetch("/api/admin/migrate-fde", { method: "POST", headers: { "content-type": "application/json", "x-page-watcher-migration": "copy-to-fde", }, body: "{}", }).then((response) => response.json()).then(console.log)
The endpoint copies the selected
DATASET_MODEtenant and all of its raw reports outward, verifies a SHA-256 state checksum, and never mutates the source. It refuses to overwrite differing destination state unless the body explicitly contains{ "replace": true }. -
Only after the response reports
ok: true, setSTORAGE_DRIVER=remoteon the Webflow app and redeploy.FDE_DATA_URLis optional whenCOLLECTOR_URLalready points to the same Worker and ends in/jobs. -
Call
/api/health;storage.drivershould beremote. Run one page or send an authenticatedPOST /nightlyto the FDE Worker, then verify the new history entry before relying on the next scheduled run. After the first weekly audit, an authenticatedGET /audits/weekly/lateston the collector returns its privacy-safe health summary; the public collector/healthresponse exposes only its status, audit ID, and update time. Before relying on the CrUX Cron, send one authenticatedPOST /crux/collectand verify thecrux_snapshotsandcrux_statusrows plus the public/healthsummary.
Rollback is just as deliberate: remove STORAGE_DRIVER=remote and redeploy to
return to the preserved Webflow bindings. Do not write to both stores after
cutover and expect automatic bidirectional merging.
Use DATASET_MODE=demo for the existing sample state and DATASET_MODE=live
for real URLs. The two modes use separate tenant keys in the same D1 database,
so switching modes is reversible and never overwrites the demo dataset.
- Per-user identity and role-based authorization
- Automated page remediation
The four fixed scoring/scheduling choices for this phase are documented in DECISIONS.md.