Skip to content

fix(cli): bound website capture and reject blocked pages - #2927

Closed
miguel-heygen wants to merge 7 commits into
mainfrom
fix/studio-5419-capture-timeout
Closed

fix(cli): bound website capture and reject blocked pages#2927
miguel-heygen wants to merge 7 commits into
mainfrom
fix/studio-5419-capture-timeout

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

Website capture no longer reports success for a minimal protection page or waits indefinitely on optional enrichment. Strong, structurally minimal 401/403/429/challenge responses now fail immediately, while vision, font, asset, media, screenshot, and contact-sheet work share a cooperative post-navigation budget with explicit degraded outcomes.

This addresses two related capture-path failures without conflating them:

  • STUDIO-5419: AWS/data-center blocking produced an 18-character page with no usable structure; local published 0.7.80 reproduced the same class as 403 - Forbidden, 50 characters, zero sections. The CLI previously exited 0 and deferred the failure to EF's frame builder.
  • Andy's timeout: external vision calls and response bodies could remain unsettled until the caller's 180/300-second watchdog. Requests are now bounded even when an SDK ignores AbortSignal, successful siblings are retained, and phase breadcrumbs identify the last stage.

Design decisions

  • --capture-budget is distinct from the navigation --timeout; it is cooperative and intentionally does not claim to interrupt already-started native Sharp/Puppeteer work.
  • --skip-vision gives callers a deterministic way to omit optional captions.
  • Provider timeouts and ordinary provider errors are sanitized and reported as distinct degraded outcomes; secrets and provider response bodies are not surfaced.
  • Low-text/image-led pages remain allowed. Hard failure requires a minimal DOM plus a strong blocked status, exact protection-page title, or structural challenge marker.
  • Lottie preview paths are published only after the screenshot exists, so budget degradation cannot leave stale manifest entries.

Verification

  • Full CLI suite: 2,323 passed, 2 skipped.
  • TypeScript typecheck, production build, oxlint, oxfmt, diff check, and pre-commit hooks passed.
  • Fresh built-CLI replay of https://fujicosmeticink.com/#about: HTTP 403 / 403 - Forbidden / 50 chars, actionable JSON error, exit 1 in 4.6 seconds before extraction.
  • Deterministic regressions cover never-settling provider requests and response bodies, mixed provider success/failure, exact 18/50-character blocked shapes, legitimate low-text titles, budget expiry between Lottie readiness and preview write, and bounded font/media/contact-sheet work.

Linear: STUDIO-5419

Experiment Framework boundary/diagnostics: experiment-framework#44091


Compound Engineering
Codex

Comment on lines +302 to +317
body: JSON.stringify({
model,
messages: [
{
role: "user",
content: [
{ type: "text", text: prompt },
{
type: "image_url",
image_url: { url: `data:${mimeType};base64,${base64}` },
},
],
},
],
max_tokens: maxTokens,
}),
@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Superseded by the review-sized stack #2928 -> #2929 -> #2930 -> #2931. Each PR is below 700 changed LOC, and #2931 is tree-identical to this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants