Skip to content

[codex] fix site crawler checks#95

Open
zack-dev-cm wants to merge 1 commit into
openclaw:mainfrom
zack-dev-cm:codex/debug-clawpatch-site
Open

[codex] fix site crawler checks#95
zack-dev-cm wants to merge 1 commit into
openclaw:mainfrom
zack-dev-cm:codex/debug-clawpatch-site

Conversation

@zack-dev-cm
Copy link
Copy Markdown

Summary

Fixes the clawpatch.ai static site issues found during the debug pass:

  • Adds robots.txt and sitemap.xml so crawler routes no longer 404 after deployment.
  • Fixes the hero H1 spacing so extracted text reads Code review with explicit fixes.
  • Adds a Node-based website:smoke check for static site metadata, crawler files, anchor targets, security-header config, and social-card dimensions.
  • Adds _headers for hosts that support static header files.
  • Widens the SIGTERM-resistant descendant timeout test timing so the full Vitest suite is stable under load.

Root Cause

The deployed site was missing crawler artifacts, and the H1 line break did not leave a space for text extraction. The existing test suite also had a timing-sensitive process test that could fail under full-suite load before its IPC-ready marker was written.

Validation

  • pnpm test — 9 files passed, 623 tests passed, 1 skipped
  • pnpm typecheck
  • pnpm lint
  • pnpm format:check
  • pnpm website:smoke
  • Local HTTP/browser check: homepage 200, /robots.txt 200, /sitemap.xml 200, normalized H1 correct, no console errors, no broken hash links

Deployment Note

Merging to main should publish the static site through the existing Pages flow. _headers only takes effect on hosts or CDNs that support header files; GitHub Pages itself does not apply it as response headers.

@zack-dev-cm zack-dev-cm reopened this May 21, 2026
@zack-dev-cm zack-dev-cm marked this pull request as ready for review May 21, 2026 06:55
@zack-dev-cm zack-dev-cm requested a review from a team as a code owner May 21, 2026 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant