Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

69 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Page Watch · Brand Studio

Nightly Lighthouse (PageSpeed Insights), weekly Chrome UX Report field evidence, and agent-readiness monitoring for a watchlist of priority Webflow.com pages. For each page it tracks per-strategy (mobile + desktop) scores over time, classifies status, surfaces recommendations, lets you triage them into tasks, log change markers, and posts drop alerts and 2/7/30-day follow-up comparisons to Slack.

Built with Next.js (App Router) + React. TypeScript throughout.

Getting started

npm install
npm run dev      # http://localhost:3000  (the bundled launch config uses 3100)

Other scripts:

npm run build    # production build
npm start        # serve the production build
npm run lint     # eslint
npm test         # vitest (unit + integration-focused concurrency tests)
npm run collector:check    # type-check the durable Workflow worker
npm run collector:dry-run  # bundle/validate the Workflow without deploying

Environment

All are optional for local development — the app runs without them.

Variable Purpose
PAGESPEED_API_KEY PSI credential. Configure it on the collector Worker; it is also used by the local runner.
CRUX_API_KEY Google Cloud key with the Chrome UX Report API enabled; required by the collector Worker.
CRON_SECRET Existing shared bearer secret for FDE state/report access, Workflow dispatch, and manual nightly requests.
COLLECTOR_URL Production Workflow endpoint, ending in /jobs.
FDE_DATA_URL Optional FDE Worker base URL. If omitted, it is derived from COLLECTOR_URL.
STORAGE_DRIVER Set to remote only after the one-time FDE copy verifies; unset keeps Webflow storage as the source.
DATASET_MODE demo uses the existing sample namespace; live uses an isolated, initially empty namespace.
BASE_URL Webflow Cloud mount path (for example /page-watch); client routes and APIs are prefixed automatically.
SLACK_WEBHOOK_URL Incoming webhook for drop alerts and follow-up reports.
PSI_MOCK Local-only deterministic scores instead of PSI.
PSI_RUNS Samples per strategy (1–5, default 5).
ANTHROPIC_API_KEY Enables post-commit recommendation explanations and Watcher narratives on the app.
ANTHROPIC_MOCK Uses deterministic placeholder AI text without making Anthropic requests.
WEBFLOW_TOKEN_ENCRYPTION_KEY Collector-only base64 AES-256 key used to encrypt tenant Webflow site tokens before D1 persistence.

Put these in .env.local.

How it works

  • Collection — baseline, on-demand, and nightly actions reserve a durable job in FDE D1 and return 202. A Cloudflare Workflow performs up to five staggered PSI samples per strategy and stores every successful raw response, Lighthouse warning, and normalized failing audit in FDE R2. Cached/replayed responses are retained for diagnosis but do not count as independent samples. Warned runs are also excluded from trusted medians. A collection needs at least three unique, warning-free measurements before it can commit; otherwise the Workflow sleeps durably and retries up to twice at one-hour intervals. Recommendations require a strict-majority Lighthouse finding across those eligible runs. The Workflow also scans agent readiness and commits the completed result directly into FDE storage. The SSO-protected Webflow app only makes authenticated outbound requests; the Worker never has to call into the Webflow Access tenant. Retries are durable, duplicates coalesce, stale jobs become visible failures, and a run ID can append history only once.
  • Baselines — ordinary on-demand/nightly runs may store snapshots, history, recommendations, and scan results, but a page stays Pending until the user explicitly captures a baseline. Zero placeholders are never treated or shown as real baselines.
  • Scheduled collection — every 15 minutes, the FDE Worker reads the live watchlist from its own D1 and dispatches only pages due in the workspace's saved local-time/timezone window. Active pages receive stable 15-minute offsets, and individual PSI samples are spaced one minute apart. POST /nightly still provides a force-all operation for authenticated manual tests. There is no Webflow callback and no Cloudflare Access service token.
  • Weekly data audit — each Monday at 05:30 UTC, the Worker reconciles the prior seven days of stored scores against their raw PSI responses. It checks raw-report coverage, usable sample counts, Lighthouse warnings/runtime errors, score medians/ranges, and finding quorum. The saved health report has hashed page references and no customer names, URLs, raw payloads, or raw errors. See audits/README.md.
  • Weekly field evidence — each Tuesday at 06:15 UTC, the Worker queries the CrUX History API for phone and desktop evidence on every active page. It prefers exact-URL data, falls back to origin data only when URL evidence is unavailable, stores rolling 28-day p75 metrics and histograms in dedicated D1 tables, and retains the complete provider response in R2. An authenticated POST /crux/collect runs the same collection manually.
  • Storage — the production source of truth is the FDE-owned page-watcher-fde D1 database plus the page-watcher-reports R2 bucket. The Webflow app uses a tenant-scoped remote DataStore; D1 state updates use version-guarded compare-and-swap retries. Local development continues to use .data/. The Webflow-provisioned D1/R2 bindings remain intact as a reversible migration source and are not deleted by the migration.
  • State mutations go through targeted server-side domain endpoints (/api/pages, /api/recs, /api/pages/[id]/*) and the store-level atomic update primitive. External PSI/Slack work happens outside that critical section; result commits re-read authoritative state.
  • Background execution — Cloudflare Workflows own production execution. The local runner uses Next.js after() only for development.
  • Post-commit enrichment — after the scores/raw reports are safely stored, optional Anthropic recommendation explanations, Watcher narrative refreshes, Slack alerts, and due follow-ups cannot roll back or mislabel a successful collection.
  • Webflow Enterprise activity — Settings can connect one Enterprise site per tenant with a site token scoped to sites:read, site_activity:read, pages:read, assets:read, and cms:read. The collector validates every capability, encrypts the token with AES-GCM before D1 persistence, imports normalized actor/resource activity into D1, retains bounded raw events in R2, and refreshes activity on the existing 15-minute scheduler. Webflow failures are recorded on the connection and never block scheduled PSI. See docs/webflow-activity.md for the product contract and remaining publish-verification phases.

Production setup

The interactive app relies on the enclosing site's SSO. There is no second HTTP Basic layer and no FDE_ACCESS_* configuration. Non-interactive nightly and collector result endpoints remain protected by CRON_SECRET.

  1. Apply migrations/ to the FDE-owned page-watcher-fde database, then deploy collector-worker/wrangler.jsonc. The Worker needs PAGESPEED_API_KEY, CRUX_API_KEY, CRON_SECRET, and a base64-encoded 32-byte WEBFLOW_TOKEN_ENCRYPTION_KEY generated with openssl rand -base64 32. Its D1, R2, Workflow, 15-minute due-page/Webflow-activity scheduler, Monday 05:30 UTC audit, and Tuesday 06:15 UTC CrUX Cron bindings are declared in that config.

  2. Deploy the Webflow app code with STORAGE_DRIVER still unset. The app keeps reading and writing its existing Webflow-provisioned D1/R2 bindings at this stage.

  3. While signed into the app through SSO, make this same-origin request from the browser console:

    fetch("/api/admin/migrate-fde", {
      method: "POST",
      headers: {
        "content-type": "application/json",
        "x-page-watcher-migration": "copy-to-fde",
      },
      body: "{}",
    }).then((response) => response.json()).then(console.log)

    The endpoint copies the selected DATASET_MODE tenant and all of its raw reports outward, verifies a SHA-256 state checksum, and never mutates the source. It refuses to overwrite differing destination state unless the body explicitly contains { "replace": true }.

  4. Only after the response reports ok: true, set STORAGE_DRIVER=remote on the Webflow app and redeploy. FDE_DATA_URL is optional when COLLECTOR_URL already points to the same Worker and ends in /jobs.

  5. Call /api/health; storage.driver should be remote. Run one page or send an authenticated POST /nightly to the FDE Worker, then verify the new history entry before relying on the next scheduled run. After the first weekly audit, an authenticated GET /audits/weekly/latest on the collector returns its privacy-safe health summary; the public collector /health response exposes only its status, audit ID, and update time. Before relying on the CrUX Cron, send one authenticated POST /crux/collect and verify the crux_snapshots and crux_status rows plus the public /health summary.

Rollback is just as deliberate: remove STORAGE_DRIVER=remote and redeploy to return to the preserved Webflow bindings. Do not write to both stores after cutover and expect automatic bidirectional merging.

Use DATASET_MODE=demo for the existing sample state and DATASET_MODE=live for real URLs. The two modes use separate tenant keys in the same D1 database, so switching modes is reversible and never overwrites the demo dataset.

Deferred integrations

  • Per-user identity and role-based authorization
  • Automated page remediation

Product decisions

The four fixed scoring/scheduling choices for this phase are documented in DECISIONS.md.

About

Next.js page watcher app

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages