[lilyshen0722]

We hit this while evaluating LayerFS for our agents' workspaces. On a fresh Ubuntu 24.04 VM, with the Docker 29.1.3 it came with (Engine API 1.52), every sandbox.create() failed with a 400 before any container started. #459 made the reason visible; this makes the client speak the Engine's version.

Fixes #457. The docs said the SDK does not negotiate on purpose, and you invited a fix, so here it is.

What changes

• The first request of a Docker client reads the Engine's version once with an unversioned GET /version. That client and its clones then use /v1.<minor>/ at the Engine's ApiVersion, up to 1.54. Call sites pass unversioned paths; Docker::send adds the prefix and transmit is the old send.
• An Engine below 1.48 is refused before any versioned request, with attempted: false and a message naming both versions.
• If GET /version can't be read (not 200, not JSON, no ApiVersion), the client uses 1.54 as before. Nothing that works today changes, and the Engine's own refusal still reaches the caller through Surface Docker error details in sandbox failures #459.
• The probe is a separate read. A failed request is never sent again at another version.
• api/README.md, api/docs/agent.md and the 71- architecture note describe the new range.

Why 1.48: Docker's version history shows nothing this client sends changing in 1.53 or 1.54. But on Engines 23.0 to 27.5 (API 1.42 to 1.47) the daemon upload fails in PUT /containers/{id}/archive with getent unable to find entry "0:0" in passwd database (the container's User under copyUIDGID). Engine 28.0 (API 1.48) and every later one tested pass. We didn't test 1.53 to 1.55, because no Engine release reports those versions.

One choice you may want differently: RuntimeError::Unsupported carries only a &'static str, so the refusal puts its version text in the failure's error_body diagnostic. If you'd rather not reuse that field for a client-side message, I can add a variant to RuntimeError instead.

Before and after

Real SDK walk: LayerFS.init, open, sandbox.create, sandbox.close, with debian:bookworm-slim@sha256:7ff6ba31527b9e336162fa061746c366c4b2032f4d2cff2078bea43f78efe3f8, in docker:<version>-dind.

Before, on main with Docker 29.1.3 (API 1.52): sandbox create stopped at Create, phase: Volume, status: Some(400), with

error_body: Some(ErrorDiagnostic { prefix: "{\"message\":\"client version 1.54 is too new. Maximum supported API version is 1.52\"}\n", prefix_bytes: 84, observed_bytes: 84, complete: true, read_error: None })

After:

Docker 29.1.3 (API 1.52)
sandbox.create ok {"sandbox":"1","image":"sha256:7ff6ba31527b9e336162fa061746c366c4b2032f4d2cff2078bea43f78efe3f8"}
sandbox.close ok

Docker 29.9.2 (API 1.56, addressed as /v1.54)
sandbox.create ok {"sandbox":"1","image":"sha256:7ff6ba31527b9e336162fa061746c366c4b2032f4d2cff2078bea43f78efe3f8"}
sandbox.close ok

Engines 28.0.4 (1.48) to 28.3.3 (1.51) pass. Engine 27.5.1 (1.47) is refused with Engine API 1.47 is older than the 1.48 to 1.54 this client speaks.

Tests

• engine_protocol.rs gains four cases on a scripted Engine: a 1.52 Engine, a 1.56 Engine, a 1.47 Engine (refused, with /version the only request sent) and a 404 on /version. All four fail on main.
• The peers in support/engine.rs and engine_keepalive.rs answer the version request themselves, so existing assertions are unchanged.
• One test is renamed: ..._without_negotiating becomes ..._without_retrying, which is what it still proves.
• No tests deleted.
• cargo test -p layerfs-sandbox --locked gives 36 passed. Each test command ran under a 120 s timeout.
• cargo fmt --all -- --check and Clippy with -D warnings (workspace, without layerfs-node) are clean, as is tools/check_product_boundary.py --skip-sdk.

Production LOC: 102200 -> 102288 (delta +88), all in layerfs-sandbox (3329 -> 3417); the method is in the commit message.

I work on Commonly (commonly.me), and we ran into this ourselves; the fix was written with AI assistance and the repro and tests were run as described.