[DanielDarrenJones]

static:warm finds out a URL is paginated from the X-Statamic-Pagination header on its first page, then warms the rest of its pages. Two things about how it does that cost time.

The other URLs wait while a paginated URL is warmed

The pages are warmed from inside the callback that handles the first page's response, with a second pool that is waited on there and then. Until that finishes, the main pool isn't topped up, so its requests drain away and nothing else is sent. A site with a lot of paginated listings spends much of the run at a fraction of warm_concurrency.

Paginated pages are now collected as responses come in, and warmed together once the main pool has finished, at the same concurrency.

Pages that can't be cached separately are requested anyway

With ignore_query_strings on, /blog?page=2 is cached under the same URL as /blog, which has just been warmed. Requesting it does nothing but serve the cached page back. The same is true of a page parameter that isn't in allowed_query_strings, or is in disallowed_query_strings.

A page is now only requested when the cacher would store it under a different URL from the one that was just warmed. This uses the cacher's own getUrl(), so it follows whatever the query string settings are. It applies to --queue too, where each of those pages was a job.

Also

A URL that already has a query string had its page added with a second ?, as in /blog?filter=x?page=2. It's now added with &, as the queued job already did. This only affected URLs added through the additional hook.

What changes for someone running it

• Paginated pages are listed at the end of the output, rather than after the page they belong to.
• With query strings ignored, the ?page=N lines are gone, and so are the requests.
• Which pages end up cached is the same as before.

Benchmark

A generated site of 2,888 URLs, of which 2,000 are articles and the paginated listings add another 1,049 pages. Half measure, warm_concurrency at its default of 25, served by php -S with 8 workers on an Apple M1 Pro. Each figure is one full static:warm from an empty cache, run twice. The machine was busy with other work, so the times are rough; the request counts are exact.

	Before	After	Requests
ignore_query_strings on	279 s, 262 s	184 s, 189 s	3,937 → 2,888
ignore_query_strings off	391 s, 415 s	362 s, 345 s	3,937 → 3,937

With query strings ignored, about a third of the time goes: a quarter of the requests are no longer made, and the rest aren't held up. With them kept, every page still has to be rendered, so the gain is only from not stalling the pool, and is closer to 10%.

The full strategy wasn't measured.

Tests

Five tests added, four for the command and one for the job. Each fails on 6.x.