[Docs]: Self-hosting guide & migration guide are not updated for 0.9.x Docker deployment

[SohamKukreti]

Summary

Following the current documentation (docs/md_v2/core/self-hosting.md and deploy/docker/MIGRATION.md) as written, a fresh user cannot reach a working, authenticated Docker server. Every quickstart path ends in an unreachable server or an undocumented step. All findings below were reproduced first-hand on 2026-07-21 against Docker Hub unclecode/crawl4ai:latest (reports v0.9.2) and develop @ 2d8f673.

Environment: Linux (Fedora 41), Docker 29.3.0, Docker Compose v5.1.0.

Self-hosting guide (docs/md_v2/core/self-hosting.md)

1. The quickstart commands produce an unreachable server. Option 1's basic run:

   docker run -d -p 11235:11235 --name crawl4ai --shm-size=1g unclecode/crawl4ai:latest

   sets no CRAWL4AI_API_TOKEN, so the entrypoint binds 127.0.0.1 inside the container. The container reports healthy, but every host request gets curl: (56) Recv failure: Connection reset by peer. The guide then says "The server will be available at http://localhost:11235" — it is not. Options 2 and 3 have the same gap. Every quickstart needs the token added, plus a callout explaining the token-or-loopback bind behavior.

2. Stale version claim. The guide says "Our latest release is 0.8.0"; Docker Hub latest currently reports 0.9.2.

3. ~600 lines documenting removed features. The inline-Python hooks API and output_path on /screenshot//pdf are documented at length with only a top banner disclaiming them. These sections need replacing with the artifact flow and declarative hooks.

4. Wrong dashboard URL. The guide points to http://localhost:11235/monitor, which returns bare {"detail": "Authentication required"} JSON. The actual dashboard UI is at /dashboard.

5. Env-var pitfalls worth documenting: -e CRAWL4AI_API_TOKEN (no value) silently forwards empty from a shell where the var isn't set — the container starts in loopback mode with no host-side indication. Docs should show the explicit -e CRAWL4AI_API_TOKEN="$CRAWL4AI_API_TOKEN" form.

6. Startup window. For ~5–10s after start, the published port answers with connection reset — the same symptom as the missing-token failure. Worth one sentence so users don't misdiagnose a working deployment.

7. CRAWL4AI_HOOKS_ENABLED is undocumented (see migration-guide section below) — the hooks feature docs should mention it.

Migration guide (deploy/docker/MIGRATION.md)

8. Step 1 ("Everyone") doesn't work for compose users. The guide says:

   export CRAWL4AI_API_TOKEN="$(openssl rand -hex 32)"

   but docker-compose.yml never forwards host env vars (its environment: block is fully commented out). The exported token silently never reaches the container and the server stays loopback-only. The only working path today is putting CRAWL4AI_API_TOKEN=... inside .llm.env — mentioned nowhere, and deploy/docker/.llm.env.example has no such line.

9. The guide's own declarative-hooks example returns 403 on a default deployment:

   {"detail":"Hooks are disabled. Set CRAWL4AI_HOOKS_ENABLED=true to enable."}
   

   CRAWL4AI_HOOKS_ENABLED appears nowhere in MIGRATION.md, the release blogs, or the self-hosting guide. With the flag set, the documented JSON shape works exactly as described. The hooks section needs the flag documented.

10. Interim warnings for silent failures. Until server-side fixes land (see companion issue), the guide should warn that two removed features fail silently rather than with errors:

    • output_path on /screenshot is accepted and ignored (returns success: true + artifact_id, no warning).
    • Legacy hooks.code with hooks enabled returns HTTP 200 with "hooks":{"status":"success","attached":[]} — the inline code is dropped but reported as success.

What good looks like

A minimal working recipe the docs could carry:

# docker run
export CRAWL4AI_API_TOKEN="$(openssl rand -hex 32)"
docker run -d -p 11235:11235 --name crawl4ai --shm-size=1g \
  -e CRAWL4AI_API_TOKEN="$CRAWL4AI_API_TOKEN" \
  unclecode/crawl4ai:latest
# all endpoints except GET /health need: Authorization: Bearer $CRAWL4AI_API_TOKEN
# UIs: /playground and /dashboard — paste token into the "API token" bar

# docker compose
cp deploy/docker/.llm.env.example .llm.env
echo "CRAWL4AI_API_TOKEN=$(openssl rand -hex 32)" >> .llm.env   # NOT `export` — compose ignores host env
docker compose up --build -d