feat(crawl): return each page as Markdown, and choose how a crawl finds URLs

[zenrows-dev-pool]

Summary

crawl create also takes --discovery links, and refuses any other value
locally.

crawl create --output-format markdown stores each kept page converted to
Markdown, read back with crawl content and crawl download as
--output-format html already was. A crawl also reports discovery, so
crawl get --json carries it instead of dropping it.

What the task asked for

The command had drifted from the API it wraps. It refused every output format
but html, and its result type dropped the discovery field a crawl reports.

How the check confirmed it

npm run typecheck and npm test both pass, 364 tests. This change adds two
checks to the Crawl acceptance suite. Check 22 covers a create with
--output-format markdown. Check 23 covers a value that is neither, which the
command still refuses locally before any network call.

A running Crawl service took output_format: "markdown" and came back with
markdown.

Encoded by: acceptance checks 22, 23 and 24, and tests/crawl-command.test.ts.

The test that listed --discovery among unknown flags now checks it sends
links and refuses every other value before any network call.

How to try it

zenrows crawl create https://example.com/ --output-format markdown --max-pages 20 --follow
zenrows crawl results <id> --json        # fetched rows carry content_url
zenrows crawl content <id> <content_id>  # the page, as Markdown
zenrows crawl create https://example.com/ --output-format pdf   # refused locally

Provenance

The command checks the format itself and names what it takes, which is how it
already handled html. A reader of --help sees the choice without a round
trip to the API.

The lock records the API description this surface follows. It lists
GET /healthz and POST /internal/v1/crawls/{crawl_id}/stop as pending. Both
are in the description and callers cannot reach either. Listing them keeps a
change to either from reading as drift here.

Review

Severity	File and line	Summary
Low	—	No finding survived the pass.

One pass, by the author. Human must verify: nothing outstanding.

Deviations

None.

Record owed

None.