[Bug]: Docker(0.9.x): Compose file invalid on Compose v5, silent API removals, misleading auth failure modes

[SohamKukreti]

Summary

A batch of deployment/server issues found while walking through a fresh 0.9.2 Docker setup, reproduced first-hand on 2026-07-21 against Docker Hub unclecode/crawl4ai:latest (v0.9.2) and develop @ 2d8f673. A companion issue covers the documentation side.

Environment: Linux (Fedora 41), Docker 29.3.0, Docker Compose v5.1.0.

1. docker-compose.yml is rejected by Compose v5 — nothing builds, and compose down breaks too

services.crawl4ai: can't set distinct values on 'pids_limit' and
'deploy.resources.limits.pids': invalid compose project

The file sets pids_limit: 512 alongside a deploy.resources.limits block (memory). Compose v5 normalizes pids_limit into deploy.resources.limits.pids and, when the limits block exists without a pids key, wrongly reports "512 vs. unset" as a conflict. Minimal repro (Compose v5.1.0):

• pids_limit: 512 alone → valid
• pids_limit: 512 + limits block with only memory → error
• both keys set to 512 → valid

Users who upgrade Compose with a running stack also cannot docker compose down (same error) and must docker rm -f.

Fix: replace pids_limit: 512 with pids: 512 under deploy.resources.limits. This is compose-spec-compliant and supported by all Compose v2.x+ (only the EOL Python docker-compose v1 lacks it, and v1 ignores deploy: anyway). Also drop the obsolete version: '3.8' attribute (warns on every command). The underlying normalization behavior may be worth reporting to docker/compose separately.

2. Compose: .llm.env is hard-required and the API token can't be passed from the host

• env_file: .llm.env has no required: false → fresh clones fail with "env file .llm.env not found", though docs frame the file as LLM-only.
• The environment: block is fully commented out, so MIGRATION.md's export CRAWL4AI_API_TOKEN=... silently never reaches the container → loopback-only server.

Fix: mark the env file required: false; add CRAWL4AI_API_TOKEN=${CRAWL4AI_API_TOKEN:-} passthrough under environment:; add a commented CRAWL4AI_API_TOKEN= line to deploy/docker/.llm.env.example.

3. Tokenless server: healthy container, dead published port, misleading log

Without a token the entrypoint binds 127.0.0.1 inside the container. docker ps shows the port published and the container healthy, but every host request gets Connection reset by peer. The only explanation is in docker logs:

entrypoint: no CRAWL4AI_API_TOKEN set; binding loopback only (127.0.0.1:11235).

A user reads this as "available at 127.0.0.1:11235" — but that's the container's loopback, unreachable from the host even with -p.

Fix (entrypoint.sh): make the message explicit, e.g. "published ports will NOT work; set CRAWL4AI_API_TOKEN to expose the server." Consider whether the healthcheck should report healthy while the published port is dead.

4. output_path on /screenshot is silently ignored (silent breaking change)

A request including output_path (the 0.8.x API) returns success: true + artifact_id with no error or warning — upgrading users' code "works" but never writes the file it asked for. Contrast with the js_code rejection, which is loud and clear:

{"detail":"Rejected config: field 'js_code' is not permitted on CrawlerRunConfig from an untrusted request"}

Fix: reject (or warn on) output_path in /screenshot and /pdf the same way.

5. Legacy hooks.code fails silently or with a misleading hint (silent breaking change)

Server state	Legacy hooks.code request result
Hooks disabled (default)	403 "Set CRAWL4AI_HOOKS_ENABLED=true" — misleading: enabling won't make code hooks work
Hooks enabled	HTTP 200 with "hooks":{"status":"success","attached":[]} — inline code silently dropped, reported as success

An upgrading 0.8.x user's auth/scroll/header hooks silently stop executing.

Fix: return an explicit 400 in both states: "inline hook code was removed in 0.9.0; use declarative actions (see MIGRATION.md)."

6. Playground shows "● Success" on a 401 response

Running a crawl in /playground without setting a token shows a green "● Success · Time: 8ms" banner above {"detail": "Authentication required"}. The UI treats any HTTP response as success, and nothing on the page says a token is required or where to find it.

Fix: status banner should reflect non-2xx responses; add a "token required" hint when unset.

7. Root / and /monitor are dead ends in the browser

• GET / returns bare {"detail": "Authentication required"} — no pointer to /playground, /dashboard, or auth instructions.
• GET /monitor (the URL the self-hosting guide gives for the dashboard) returns the same bare JSON; the actual UI is at /dashboard.

Fix: a minimal landing response at / linking the UIs; serve or redirect the dashboard at /monitor for compatibility with existing docs/links.