Skip to content

feat(extract): replace the autoparse fallback with zenrows extract prepare - #47

Draft
zenrows-dev-pool[bot] wants to merge 3 commits into
mainfrom
feat/extract-prepare-domain
Draft

zenrows-dev-pool[bot] wants to merge 3 commits into
mainfrom
feat/extract-prepare-domain

Conversation

@zenrows-dev-pool

@zenrows-dev-pool zenrows-dev-pool Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Summary

zenrows extract no longer retries with the deprecated Autoparse when Extract has not prepared a domain. It reports the error with the exact command to prepare the domain, and this PR adds that command, so users can prepare a domain themselves and extract again.

Flow: zenrows extract <url> → REQS007 → zenrows extract prepare <domain> --url <url> --wait → zenrows extract <url>.

Behaviour

Before After
403 REQS007 / legacy 402 AUTH010 on extract=auto Silent second request with autoparse=true; fellBackToAutoparse: true; Autoparse charged One request. Error EXTRACT_DOMAIN_NOT_ENABLED with the message below; nothing retried
Preparing a domain Not possible from the CLI zenrows extract prepare <domain> --url <u> [--url <u>...] [--wait [--timeout <s>]] and zenrows extract prepare status <domain>
--autoparse General-purpose extraction Same, plus a warning that Autoparse is deprecated
Adapter fallbackAutoparse Default true Default false (explicit opt-in kept)
Result whose fields are all null / "" / [] (for example page scaffolding) empty: false empty: true, so the usual warning (or --validate failure) applies

The REQS007 message, short so agents can copy it:

Extract hasn't prepared example.com yet. Prepare it with zenrows extract prepare example.com --url https://example.com/p/1 (a few example URLs, a few minutes, no credits), then retry; or run with --autoparse.

suggested_commands are the same command with --wait, zenrows extract prepare status example.com, and the original extract. The error code stays EXTRACT_DOMAIN_NOT_ENABLED, so callers that match on it keep working.

zenrows extract prepare

  • Calls POST /v1/extract/prepared-domains (X-API-Key) with {"domain", "example_urls"}. --url takes 1 to 5 full http(s) URLs; anything else is rejected before any request. A URL or www. host is normalized to the domain.
  • Returns at once by default. --wait polls GET /v1/extract/prepared-domains/{domain} every 15 seconds until ready or failed (default timeout 900 seconds).
  • prepare status shows status, state, per-page-kind roles, the failure, the monthly allowance (GET .../allowance: used / limit / remaining) and the next step. If the allowance call fails, the status is still shown.
Answer Result
202 Status record, next step (prepare status ..., or the extract once ready)
409 in_progress Follows the running preparation (current status); EXTRACT_PREPARE_IN_PROGRESS only when this account cannot read it
400 invalid_argument / 409 idempotency_conflict EXTRACT_PREPARE_INVALID (with reason)
404 not_found (status) EXTRACT_PREPARE_NOT_FOUND, with the prepare command
429 out_of_allowance EXTRACT_PREPARE_OUT_OF_ALLOWANCE (with used / limit); do not retry this month, use --css
503 overloaded / disabled EXTRACT_PREPARE_UNAVAILABLE; retry in a few minutes, do not loop
status failed (with --wait) EXTRACT_PREPARE_FAILED, exit 1, with the next step for each documented failure code
--wait runs out EXTRACT_PREPARE_TIMEOUT, with the status command

Docs and skills

skills/extract, skills/zenrows, agent-plugin/skills/*, README.md, docs/capabilities.md, the competitor-intelligence workflow, the pricing template and registry/*.json now teach the extract → REQS007 → prepare → poll → extract flow. They mark Autoparse deprecated, drop "falls back to autoparse", and say "any prepared domain (you can prepare one yourself)". The agent-plugin extract skill describes the MCP server's prepare_domain / prepared_domain_status tools and new defaults, which ship in the matching zenrows-mcp PR.

Release

No version change here. Per docs/contributing.md, release-please sets the version from the feat(extract) title after merge.

Tests

  • npm run typecheck: clean. The repo has no linter beyond tsc.
  • npm test: 384 passed, 0 failed. New: tests/extract-prepare.test.ts covers submit, --wait to ready, --wait to failed, --wait timeout, 409 followed and 409 unreadable, 400, 429, 503 overloaded and disabled, --url validation, status with and without the allowance, status 404, every failure code, and routing from zenrows extract. Updated: REQS007 and AUTH010 print the prepare command and are not retried; the fallback runs only with fallbackAutoparse: true; all-null nested payloads are empty; the capability notes name the prepare commands.
  • Live against a local API: prepare status on a ready domain, prepare quotes.toscrape.com --url ... --wait (preparing → ready, then printed the extract command), and prepare status on a domain never submitted (404 mapped).

…g back to autoparse

On 403 REQS007 (domain not prepared yet) or the legacy 402 AUTH010,
`zenrows extract` used to retry with the deprecated autoparse. That hid
the prepare step, charged for an Autoparse call and often returned
unusable data. The error is now reported, not retried, and names the
next step:

  Extract hasn't prepared <domain> yet. Prepare it with POST
  /v1/extract/prepared-domains (a few example URLs, a few minutes, no
  credits), then retry; or run with --autoparse.

`--autoparse` stays as the explicit opt-in and now warns that Autoparse
is deprecated. The adapter's `fallbackAutoparse` option defaults to
false.

The empty-result check now treats a payload whose fields are all null
or empty (recursively) as empty, so page scaffolding no longer reads as
data.

Skills, docs and the capability matrix describe the prepare step, mark
autoparse deprecated and drop the fallback wording.
Removing the autoparse fallback leaves REQS007 as the answer on a domain
Extract has not prepared yet, so CLI users need a way to prepare one.

- `zenrows extract prepare <domain> --url <u> [--url <u>...]` submits
  the domain with 1 to 5 example URLs to
  POST /v1/extract/prepared-domains and returns at once. `--wait`
  (off by default) polls until ready or failed, up to `--timeout`.
  On 409 in_progress it follows the running preparation.
- `zenrows extract prepare status <domain>` shows progress, the failure
  code and detail, the account's monthly allowance and the next step.
- Errors name the next step: EXTRACT_PREPARE_INVALID (400),
  EXTRACT_PREPARE_NOT_FOUND (404), EXTRACT_PREPARE_IN_PROGRESS (409),
  EXTRACT_PREPARE_OUT_OF_ALLOWANCE (429), EXTRACT_PREPARE_UNAVAILABLE
  (503 overloaded / disabled), EXTRACT_PREPARE_FAILED (failed status,
  with the step for its failure code), EXTRACT_PREPARE_TIMEOUT.
- The REQS007 error now prints the exact command:
  zenrows extract prepare <domain> --url <url>.

Skills, README and the capability matrix teach the flow: extract,
REQS007, prepare, poll, extract again.
@zenrows-dev-pool zenrows-dev-pool Bot changed the title feat(extract): report REQS007 with the prepare step instead of falling back to autoparse feat(extract): replace the autoparse fallback with zenrows extract prepare Oct 15, 2026
@jonathan-zenrows jonathan-zenrows self-assigned this Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant