[ci] Serve the docs from Cloudflare Workers with per-PR previews - #6287
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
dosubot
Bot
added
size:L
📘 Docs preview
This comment updates in place on every push. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (13)
Included review availability: Your plan provides up to 8 included reviews per hour; 6 remain after this review. 📝 WalkthroughSummary by CodeRabbit
WalkthroughAdded production and pull-request preview workflows for Cloudflare Workers documentation deployments. Added asset staging, cache and redirect rules, preview noindex handling, changed-page links, sitemap parity checks, and deployment documentation. ChangesDocumentation deployment
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: 🔵 Low · up to The PR adds Cloudflare-based docs previews and deployment, while the validation script can accept redirected URLs with changed query strings and does not verify cache headers or missing-page responses. This is a bounded merge-readiness risk that is mergeable with explicit owner follow-up. Sequence Diagram(s)sequenceDiagram
participant PullRequest
participant GitHubActions
participant CloudflareWorkers
participant PullRequestComment
PullRequest->>GitHubActions: trigger preview workflow
GitHubActions->>GitHubActions: resolve and check out PR head
GitHubActions->>CloudflareWorkers: deploy PR-specific docs alias
CloudflareWorkers-->>GitHubActions: return deployment URL
GitHubActions->>CloudflareWorkers: verify /docs returns HTTP 200
GitHubActions->>PullRequestComment: publish ready or failure status
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 4 files. (9 skipped: 9 unsupported.) ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
e4753ff to
8d913dc
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 500780d2-57b2-4c07-8e00-12e0dfc967c5
📒 Files selected for processing (13)
.github/workflows/18-docs-preview.yml.github/workflows/19-docs-production.ymldocs/.gitignoredocs/README.mddocs/docusaurus.config.tsdocs/package.jsondocs/scripts/changed-pages.mjsdocs/scripts/check-parity.mjsdocs/scripts/stage-worker-assets.mjsdocs/worker-assets/_headersdocs/worker-assets/_redirectsdocs/wrangler.jsoncdocs/wrangler.production.jsonc
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.
8d913dc to
7d68c48
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: f49fee6d-24ce-4dd1-aa4e-78aa8ea41656
📒 Files selected for processing (1)
docs/scripts/changed-pages.mjs
Included review availability: Your plan provides up to 8 included reviews per hour; 5 remain after this review.
7d68c48 to
8719644
Compare
|
Addressed every review comment. Pushed as CodeQL, incomplete escaping in A manual dispatch could publish
No The rule ends in the literal Unquoted README implied this was the cutover. Reworded. The table now says Re-verified after the changes: 950 of 950 sitemap URLs return 200 on the preview, and the preview now also carries @coderabbitai review |
There was a problem hiding this comment.
Actionable comments posted: 2
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 439c7a89-a625-4ce4-b816-b689028c296e
📒 Files selected for processing (5)
.github/workflows/18-docs-preview.yml.github/workflows/19-docs-production.ymldocs/README.mddocs/scripts/check-parity.mjsdocs/worker-assets/_headers
🚧 Files skipped from review as they are similar to previous changes (1)
- docs/README.md
Included review availability: Your plan provides up to 8 included reviews per hour; 6 remain after this review.
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== PR diff summary ==='
git diff --stat HEAD^ HEAD || true
echo '=== Changed files ==='
git diff --name-only HEAD^ HEAD || true
echo '=== Workflow outlines ==='
ast-grep outline .github/workflows/18-docs-preview.yml --items all 2>/dev/null || true
ast-grep outline .github/workflows/19-docs-production.yml --items all 2>/dev/null || true
echo '=== Relevant files with line numbers ==='
for f in \
.github/workflows/18-docs-preview.yml \
.github/workflows/19-docs-production.yml \
docs/scripts/changed-pages.mjs \
docs/scripts/check-parity.mjs \
docs/scripts/stage-worker-assets.mjs \
docs/worker-assets/_headers \
docs/worker-assets/_redirects \
docs/README.md \
docs/wrangler.jsonc \
docs/wrangler.production.jsonc \
docs/docusaurus.config.ts \
docs/package.json
do
if [ -f "$f" ]; then
echo "--- $f ---"
cat -n "$f"
fi
doneLength of output: 74126 🤖 web_search executed:
Length of output: 14624
The workflows now resolve and check out the exact same-repository PR head for manual dispatches. The parity script now validates options, rejects empty sitemaps, uses only the target origin, and applies request deadlines. The changed-page script escapes backslashes and brackets. The changed-file collection preserves filenames with spaces. The I will run a new review.
|
8719644 to
7a957ad
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 0990b122-1bb7-49a3-99d7-96e22497b67c
📒 Files selected for processing (3)
.github/workflows/18-docs-preview.yml.github/workflows/19-docs-production.ymldocs/scripts/check-parity.mjs
Included review availability: Your plan provides up to 8 included reviews per hour; 5 remain after this review.
7a957ad to
6d96f33
Compare
|
Second round addressed. Pushed as A same-origin redirect could still swallow every deep link. Right, and the previous fix only closed the cross-origin half. A production deploy could ship with analytics off. Agreed, and the fallback made it silent. The build step now fails with a clear error when SC2034 on the retry loops. Changed to For the record, the Also re-ran the full gate after these changes: 950 of 950 sitemap URLs return 200 on the preview, with the only two redirects being the two version roots. @coderabbitai review |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: eb7e8920-3f31-47d6-b9ef-34c0d203d96e
📒 Files selected for processing (3)
.github/workflows/18-docs-preview.yml.github/workflows/19-docs-production.ymldocs/scripts/check-parity.mjs
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '=== Commit metadata and changed files ==='
git show --no-ext-diff --format=fuller --stat 6d96f33aa5
git diff-tree --no-commit-id --name-only -r 6d96f33aa5
echo '=== Commit patch ==='
git show --no-ext-diff --format= --find-renames 6d96f33aa5 -- \
docs/scripts/check-parity.mjs \
.github/workflows/18-docs-preview.yml \
.github/workflows/19-docs-production.yml
echo '=== JavaScript outline ==='
ast-grep outline docs/scripts/check-parity.mjs --items all --view expanded 2>/dev/null || true
echo '=== Current relevant source ==='
for file in \
docs/scripts/check-parity.mjs \
.github/workflows/18-docs-preview.yml \
.github/workflows/19-docs-production.yml
do
echo "--- ${file} ---"
cat -n "$file"
doneLength of output: 43624
I will run a full review of the PR changeset. If you found this review helpful, would you consider giving us a shout-out on X? Thank you for using CodeRabbit! ✅ Action performedFull review finished. |
The docs site is the last thing we host on Vercel, deployed by Vercel's GitHub integration and configured in a dashboard we cannot review. Docs pull requests also get no preview link that anyone can check before merging. Move it onto the same Cloudflare Workers Static Assets setup the marketing site already uses: a preview worker that takes one version per pull request, a production worker deployed on merge, and a sticky pull request comment that carries the preview URL and deep links to the pages the change touches. 'pnpm run build:worker' writes the site into dist/docs so the file tree matches the public /docs/ URL prefix, which is what removes the Vercel rewrite. Nothing routes to the new production worker yet: cutover is a separate, deliberate swap of the agenta.ai/docs* route.
6d96f33 to
251b5ba
Compare
|
Cache policy and 404 handling are now part of the gate. Pushed as Fair point, and it is the last hole in this check. Every URL returning 200 is necessary but not sufficient: a deployment that answered everything with the home page would also score 950 out of 950, and one that lost After the sitemap sweep the script now probes a path that cannot exist and requires a 404, reads the home page to find a real content-hashed asset, and asserts the two cache policies that Against the Cloudflare preview: Against the current production docs on Vercel, the same check fails, correctly: That is a real difference, not a false alarm. The router worker in front of Vercel rewrites |
3 participants
Context
The docs site is the last thing we host on Vercel. Vercel's GitHub integration builds it, and the whole setup lives in a dashboard, so the only trace of it in this repo is a four-line
docs/vercel.json. Nobody can review the deploy, reproduce it, or change it from a pull request.Docs pull requests also get no preview that a reviewer can open. You approve a wording change by reading the diff and hoping the page still builds.
The marketing site moved to Cloudflare Workers in July and that setup works well. This puts the docs on the same one.
Changes
Two new workers, both plain static-asset workers with no worker code:
agenta-docsdocs/wrangler.production.jsoncagenta-docs-previewdocs/wrangler.jsoncTwo workflows drive them, cloned from the marketing site's
15/16:18-docs-preview.ymlbuilds every pull request that touchesdocs/**and uploads it as a preview version, and19-docs-production.ymlbuilds and deploys on merge tomain.The one real change to the site itself is where the build lands. Docusaurus writes the site to the root of
build/while every link is prefixed/docs/, which is exactly what the Vercel rewrite existed to paper over:pnpm run build:workerwrites todist/docsinstead, so the file tree matches the URL tree and Cloudflare serves/docs/anythingwith no rule at all.scripts/stage-worker-assets.mjsthen copies_headersand_redirectsup todist/, where Cloudflare reads them.The preview comment updates in place on every push and deep-links the pages the branch touched:
The links are not guessed from file paths. Numeric prefixes,
slug:frontmatter and versioned docs all break that.scripts/changed-pages.mjsreads the metadata Docusaurus writes during the build, which carries the real source path and permalink, and prints nothing at all rather than a wrong link.Preview builds set
DOCS_NOINDEX=true, which turns on DocusaurusnoIndexand writes a disallow-allrobots.txt, so a publicworkers.devpreview can never rank against the real docs.Nothing routes to the new production worker yet.
agenta.ai/docs/*still goes through the existingnew-docs-routerworker and on to Vercel. Merging this deploysagenta-docsto itsworkers.devURL and changes no live traffic. Cutover is a separate step (below).Tests
Verified against a real preview deploy of this branch,
agenta-docs-preview:scripts/check-parity.mjsrequested all 950 URLs from the live docs sitemap against the preview. All 950 returned 200. The only two redirects are the two version roots,/docs/and/docs/1.0/, which now 307 to the no-slash form.index.htmllookup, which would have broken/docs/1.0and/docs/self-host/upgrades/v0.100.3-migration. Both return 200.dist/docs.max-age=31536000, immutableon/docs/assets/*, one week on images, 60 s withstale-while-revalidateon HTML./docs/changelog/main.<meta name="robots" content="noindex, nofollow">is present on the preview and/robots.txtdisallows everything.wrangler versions upload --preview-alias pr-0produced the alias URL the workflow greps for. The first upload of the full 378 MB build took 35 seconds; the second took 6, because wrangler only sends changed files.Before merging
POSTHOG_API_KEYcurrently lives in the Vercel project settings. It needs to exist as a GitHub repo secret of the same name, or docs analytics goes quiet when we cut over. PostHog project keys are ingest-only and public by design, so this is about not losing the value, not about secrecy.Cutover, after this merges
node docs/scripts/check-parity.mjs https://agenta-docs.<subdomain>.workers.devagainst the deployed production worker.agenta.ai/docs*route fromnew-docs-routerand addagenta.ai/docsandagenta.ai/docs/*toagenta-docs. Two workers cannot hold the same route, so this is a single swap. The commented-outroutesblock inwrangler.production.jsonchas the exact patterns.https://agenta.ai, and confirmdocs.agenta.aistill redirects and/sitemap_index.xmlstill lists both sitemaps.new-docs-routerkeeps owning both; only its Vercel proxy rule goes away.docs/vercel.jsonand the Vercel project.Not in this pull request
Fork pull requests get no preview, because they cannot read the deploy secrets. Two of the last sixty docs pull requests came from forks. The safe pattern (build without secrets, upload an artifact, deploy from a
workflow_runjob) is a follow-up if we decide we want it.