Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,10 @@ place, avoiding stale macOS executable-page signature state. `CARGO_TARGET_DIR`
must stay inside this worktree; `LAYERFS_NATIVE_OUTPUT` can select a separate
platform artifact. Keep Linux addons separate from the host addon.

At runtime, the Docker Engine server selected by `LAYERFS_DOCKER` must accept
Engine API v1.54 requests. The SDK always uses explicit `/v1.54/...` paths and
does not negotiate from the server's maximum API version; see the [agent settings](how-to-use/agent.md#settings).

New Store schema versions are 16–18 (pack layouts) and 22–24 (with acquisition),
with history identity format 3. Old Disposable Stores are rejected without
migration. Select a fresh project/Store for the new format.
Expand Down
5 changes: 5 additions & 0 deletions api/how-to-use/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,11 @@ Keep each agent's operations on a Workspace sequential.
| `LAYERFS_TRACE_STEP` | Optional step label put on each trace line as `step`; read at every call. |
| `LAYERFS_TRACE_START` | `1` adds a start line before each traced call; read at every call. |

The Docker Engine server selected by `LAYERFS_DOCKER` must accept Engine API
v1.54 requests. The SDK always uses explicit `/v1.54/...` paths and does not
negotiate from the server's maximum API version. Check the server's API range
with `docker version`.

The SDK and the `layerfs` command read only these `LAYERFS_` names. An
integration that can set only names with its own prefix maps them to these in
its own wrapper.
Expand Down
5 changes: 4 additions & 1 deletion core/crates/layerfs-sandbox/src/types/failure.rs
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ impl WireFailure {
}
}

/// Original bounded HTTP error diagnostic; bytes beyond this window are explicitly unobserved.
/// Original bounded HTTP error diagnostic; Debug shows only the retained prefix as lossy UTF-8,
/// with controls escaped, while bytes beyond this window remain explicitly unobserved.
pub struct ErrorDiagnostic {
pub prefix: Vec<u8>,
pub observed_bytes: u64,
Expand All @@ -84,7 +85,9 @@ pub struct ErrorDiagnostic {
}
impl fmt::Debug for ErrorDiagnostic {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let prefix = String::from_utf8_lossy(&self.prefix);
f.debug_struct("ErrorDiagnostic")
.field("prefix", &prefix)
.field("prefix_bytes", &self.prefix.len())
.field("observed_bytes", &self.observed_bytes)
.field("complete", &self.complete)
Expand Down
77 changes: 76 additions & 1 deletion core/crates/layerfs-sandbox/tests/engine_protocol.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#[path = "support/engine.rs"]
mod peer;
use layerfs_sandbox::{
backend::docker::{Docker, ExecRequest},
backend::docker::{CreatePhase, Docker, ExecRequest, SandboxRequest},
CommandIdentity, ContainerId, ExecId, RuntimeError,
};
use std::io::{self, Write};
Expand All @@ -22,6 +22,14 @@ fn request() -> ExecRequest {
stdin: false,
}
}
fn sandbox_request() -> SandboxRequest {
SandboxRequest {
image: format!("sha256:{}", "d".repeat(64)),
store_volume: "project-store".into(),
port: 30421,
labels: vec![("layerfs.role".into(), "sandbox".into())],
}
}
fn created() -> Vec<u8> {
peer::fixed(201, format!("{{\"Id\":\"{}\"}}", exec()).as_bytes())
}
Expand Down Expand Up @@ -214,12 +222,79 @@ fn actual_mismatched_inspection_and_http_error_body_remain_distinct_from_selecto
let failed = docker.inspect_exec(container(), exec()).unwrap_err();
assert_eq!(failed.observed_exec, None);
assert_eq!(failed.requested_exec, Some(exec()));
let rendered = format!("{failed:?}");
assert!(rendered.contains("original remote error"));
let diagnostic = failed.error_body.unwrap();
assert!(diagnostic.complete);
assert_eq!(diagnostic.prefix, body);
server.finish();
}
#[test]
fn sandbox_create_debug_shows_docker_api_version_refusal_without_negotiating() {
let message = "client version 1.54 is too new. Maximum supported API version is 1.52";
let body = format!("{{\"message\":\"{message}\"}}");
let volume = br#"{"Name":"project-store","Driver":"local","Scope":"local","Options":null}"#;
let (docker, server) = peer::Peer::new(
vec![peer::fixed(200, volume), peer::fixed(400, body.as_bytes())],
8192,
);
let failed = docker.create_sandbox(sandbox_request()).unwrap_err();
assert_eq!(failed.phase, CreatePhase::Create);
assert_eq!(failed.transfer.status, Some(400));
let rendered = format!("{failed:?}");
assert!(rendered.contains(message));
let requests = server.finish();
assert_eq!(requests.len(), 2);
assert!(String::from_utf8(requests[0].clone())
.unwrap()
.starts_with("GET /v1.54/volumes/project-store HTTP/1.1"));
assert!(String::from_utf8(requests[1].clone())
.unwrap()
.starts_with("POST /v1.54/containers/create HTTP/1.1"));
}
#[test]
fn sandbox_volume_api_version_refusal_stops_before_create() {
let message = "client version 1.54 is too new. Maximum supported API version is 1.52";
let body = format!("{{\"message\":\"{message}\"}}");
let (docker, server) = peer::Peer::new(vec![peer::fixed(400, body.as_bytes())], 8192);
let failed = docker.create_sandbox(sandbox_request()).unwrap_err();
assert_eq!(failed.phase, CreatePhase::Volume);
assert_eq!(failed.transfer.status, Some(400));
assert!(format!("{failed:?}").contains(message));
let requests = server.finish();
assert_eq!(requests.len(), 1);
let request = String::from_utf8(requests[0].clone()).unwrap();
assert!(request.starts_with("GET /v1.54/volumes/project-store HTTP/1.1"));
assert!(!request.contains("POST /v1.54/containers/create"));
}
#[test]
fn http_error_debug_shows_only_the_bounded_prefix() {
let mut body = vec![b'x'; 1024];
body.extend_from_slice(b"TAIL_SECRET_MARKER");
let (docker, server) = peer::Peer::new(vec![peer::fixed(500, &body)], 8192);
let failed = docker.inspect_exec(container(), exec()).unwrap_err();
let rendered = format!("{failed:?}");
let diagnostic = failed.error_body.unwrap();
assert_eq!(diagnostic.prefix, vec![b'x'; 1024]);
assert_eq!(diagnostic.observed_bytes, 1025);
assert!(!diagnostic.complete);
assert!(rendered.contains("prefix_bytes: 1024"));
assert!(rendered.contains("observed_bytes: 1025"));
assert!(rendered.contains("x".repeat(64).as_str()));
assert!(!rendered.contains("TAIL_SECRET_MARKER"));
server.finish();
}
#[test]
fn http_error_debug_escapes_controls_and_replaces_invalid_utf8() {
let body = b"message:\nnext\t\0\xff";
let (docker, server) = peer::Peer::new(vec![peer::fixed(400, body)], 8192);
let failed = docker.inspect_exec(container(), exec()).unwrap_err();
let rendered = format!("{failed:?}");
assert!(rendered.contains("prefix: \"message:\\nnext\\t\\0\u{fffd}\""));
assert!(!rendered.contains("message:\nnext"));
server.finish();
}
#[test]
fn wrong_upgrade_media_and_malformed_status_trailers_extensions_refuse() {
let valid = String::from_utf8(peer::upgrade(&[])).unwrap();
for wrong in [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ remaining counters, never payload allocations. Unknown JSON strings/Arguments
stream without collection. Selected-key duplicates, escapes/UTF-8/surrogates,
integer types/width, malformed/truncated framing and conflicting headers refuse
with original transfer and failed-fence knowledge. Error HTTP statuses retain a
separate bounded prefix/count/completion/error diagnostic with redacted Debug.
separate bounded prefix/count/completion/error diagnostic. Debug shows only the
retained prefix (at most 1024 bytes) as lossy UTF-8 with controls escaped, alongside
the byte count, completion state and read error.

The selected metadata profile has an explicit32-level JSON support bound,128B
selected-key/prefix capture,8KiB individual header lines and32KiB header/trailer
Expand Down
6 changes: 4 additions & 2 deletions core/docs/architecture/72-owned-sandbox-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,10 @@ The upload is incremental extraction. A failed/lost response never proves an emp
destination and never authorizes another upload, adoption, automatic deletion or
Start. Original declared executable/tar sizes, source bytes read, buffer-accepted
archive bytes, positive socket body/wire bytes and two disjoint pending windows
remain separate. Pending bytes and HTTP diagnostic prefixes have redacted Debug.
The caller retains the original file/config; SDK failure retains original inputs.
remain separate. Pending request bytes stay redacted in Debug. HTTP error diagnostics
show only the retained prefix (at most 1024 bytes) as lossy UTF-8 with controls
escaped, alongside explicit prefix/count/completion metadata. The caller retains the
original file/config; SDK failure retains original inputs.

Each original upload, Start, listener wait, stop and delete has a one-attempt flag.
Known acknowledgments are distinct from attempted/unknown outcomes. Explicit
Expand Down