Repository navigation
feat(sandbox): 本地进程 Sandbox 后端 LAYERFS_SANDBOX=local——在没有 Docker 的 Linux VM / CI 里使用 workspace.* - #402
feat(sandbox): 本地进程 Sandbox 后端 LAYERFS_SANDBOX=local——在没有 Docker 的 Linux VM / CI 里使用 workspace.*#402jiangkoumo wants to merge 48 commits into
LAYERFS_SANDBOX=local——在没有 Docker 的 Linux VM / CI 里使用 workspace.*#402Conversation
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/Cargo.toml (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 108,451 -> 108,451 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/mod.rs (production). No tests changed or deleted. Production LOC: 108,451 -> 108,460 (delta +9) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production). No tests changed or deleted. Production LOC: 108,460 -> 108,505 (delta +45) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/mounts.rs (production). No tests changed or deleted. Production LOC: 108,505 -> 108,576 (delta +71) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production). No tests changed or deleted. Production LOC: 108,576 -> 108,834 (delta +258) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production). No tests changed or deleted. Production LOC: 108,834 -> 109,057 (delta +223) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production). No tests changed or deleted. Production LOC: 109,057 -> 109,166 (delta +109) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/src/backend/mod.rs (production). No tests changed or deleted. Production LOC: 109,166 -> 109,167 (delta +1) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: core/crates/layerfs-sandbox/tests/local_runtime.rs (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 109,167 -> 109,167 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/runtime.rs (production). No tests changed or deleted. Production LOC: 109,167 -> 109,274 (delta +107) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/local.rs (production). No tests changed or deleted. Production LOC: 109,274 -> 109,527 (delta +253) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/registry.rs (production). No tests changed or deleted. Production LOC: 109,527 -> 109,527 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/discovery.rs (production). No tests changed or deleted. Production LOC: 109,527 -> 109,644 (delta +117) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/exec.rs (production). No tests changed or deleted. Production LOC: 109,644 -> 109,633 (delta -11) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/sandbox.rs (production). No tests changed or deleted. Production LOC: 109,633 -> 109,641 (delta +8) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/workspace.rs (production). No tests changed or deleted. Production LOC: 109,641 -> 109,636 (delta -5) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/src/lib.rs (production). No tests changed or deleted. Production LOC: 109,636 -> 109,638 (delta +2) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/native/tests/unit/runtime.rs (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 109,638 -> 109,638 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/src/client.js (production). No tests changed or deleted. Production LOC: 109,638 -> 109,656 (delta +18) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/src/index.js (production). No tests changed or deleted. Production LOC: 109,656 -> 109,658 (delta +2) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/test/local_deployment.mjs (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 109,658 -> 109,658 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
…=local, for a VM or CI box without Docker Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole branch (or the single-commit patch). This commit: sdk/js/README.md (not production (test, manifest or docs)). No tests changed or deleted. Production LOC: 109,658 -> 109,658 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, base 5fb68dd.
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
yifanxuaaa
left a comment
There was a problem hiding this comment.
I reviewed the local Sandbox runtime by reading the code at 819bd2c; I didn't build or run it. Seven inline comments follow, most important first: command processes surviving teardown, the parent-directory permission check for uid 501, and the missing Linux-only gating. The Docker path does look unchanged apart from the enum wrapper.
🤖 Generated with Claude Code
| /// Stops the daemon once: TERM, up to `seconds` for it to end, then | ||
| /// KILL. A daemon already gone is the same stopped state. Crash | ||
| /// teardown, not a graceful filesystem drain. | ||
| pub fn stop(&mut self, seconds: u16) -> Result<(), Box<WireFailure>> { |
There was a problem hiding this comment.
Command processes outlive the local Sandbox. stop/delete signal only the daemon's own pid. Unlike a container, nothing contains or kills the uid-501 processes that commands leave behind.
Example: an exec runs cd $HOME && nohup server >/dev/null 2>&1 & and returns. The Workspace unmounts and the daemon idle-exits after 120 s. In Docker the container's exit would kill server; here it keeps running on the host. On reclaim, remove_dir_all of <dir>/tmp races with server writing there and can fail with ENOTEMPTY. The record then stays and sandbox.list keeps failing until someone kills the process by hand. The same happens after a SIGKILLed daemon.
Suggestion: put each Sandbox's commands in a session, process group or cgroup, and kill it in stop() before delete().
There was a problem hiding this comment.
确实如此,已修。做法:
- 每个命令
process_group(0)自成一个进程组;spawn 后立刻把组登记到 Sandbox 目录下 0700 的groups/(文件名 = 组长的 boot id、pid、start time)。命令结束时组里如果已经没有进程就删掉登记,所以只有“留了东西”的组才会留在那里。 - SDK 的
operator.process.start用setsid起后台进程,会离开命令的组,所以起完后用新的Sandbox::track(pid, start)也登记一次。 stop()在 daemon 结束后、delete()在本对象没跑过 stop 时(也就是 daemon 被 SIGKILL 或空闲退出后 reclaim 的路径),对所有登记的组和 daemon 自己的组发一次SIGKILL,最多等 5 秒确认组里没有活进程,然后才 detach 挂载、remove_dir_all。组长 pid 被别的进程复用(start time 不同)的组不会动;不同 boot 的登记直接跳过。- 为什么是进程组而不是 cgroup:
layerfs-sandbox是#,std 没有安全的pre_exec/CLONE_INTO_CGROUP;spawn 之后再把 pid 写进 cgroup 会和命令的第一次 fork 竞争。进程组 + 登记覆盖了nohup、&、子 shell 和 SDK 自己的setsid后台进程。命令自己再setsid/ 双 fork 守护化出去的进程不跟踪,这一点写进了模块文档。 - 和容器的一个差别也写明了:daemon 空闲退出之后、下一次
sandbox.list/ mount 触发 reclaim 之前,这些进程还在;reclaim 时被杀掉,目录随后删除。
测试:
tests/local_runtime.rs::what_commands_leave_running_ends_with_the_sandbox:exec 留下一个不停往$HOME(Sandbox 的 tmp)写文件的nohup进程和一个setsid sleep(用track登记);(a)stop()后两个都没了、delete()删掉目录;(b) 外部kill -9daemon,再用Sandbox::stopped(...)走delete()(reclaim 路径),两个都没了、目录删掉。- 端到端 V2–V2c:exec 留
nohup写进程 +operator.process.start后台进程 → SIGKILL daemon →operator.sandbox.list回收成功(无 ENOTEMPTY),两个进程都没了,目录删除;L5 是你描述的原场景:exec 留nohup进程、unmount、等 daemon 120 秒空闲退出 →sandbox.list回收后进程没了。修复前的二进制跑同一套,V2c 和 L5 失败,进程以 uid 501 继续存活。
| { | ||
| return Err(RuntimeError::Protocol("absolute local runtime root")); | ||
| } | ||
| let metadata = fs::symlink_metadata(root)?; |
There was a problem hiding this comment.
The root's parent directories are not checked as traversable by the command identity. This checks only the root's own mode. Commands chdir into their Workspace after std drops to uid 501, so every directory above LAYERFS_LOCAL_DIR must also be traversable by that uid.
Example: running as root with LAYERFS_LOCAL_DIR=/root/layerfs-local (0755) inside /root (0700, the default). Create, the daemon start and mount all succeed. Every workspace.exec, plus the HOME/scratch probes, then fails at spawn with EACCES, long after setup and with a confusing error.
Suggestion: walk the ancestors and require o+x, or check traversal as the command identity, here or at start.
There was a problem hiding this comment.
是的,已修。Local::new 现在对给定路径的每一级和 canonicalize 后真实路径的每一级都要求“是目录且 o+x”,不满足就在 create 时(写 intent 之前)返回 PermissionDenied,消息点名是哪一级,例如:
LAYERFS_LOCAL_DIR /data/sbl-e2e/closed-parent/local: runtime I/O: /data/sbl-e2e/closed-parent is not traversable by others (o+x); local Sandbox commands run as another identity and must pass it to reach their Workspace
这里按 o+x 判断,没有按 501:20 去看 owner/group 位或 ACL——更保守一些,/root 这类情况都会在一开始被拒。如果你更希望“以命令身份判断”,我可以改。
测试:every_directory_above_a_root_must_be_traversable_by_others(0700 父目录被拒且点名;经符号链接的父目录同样被拒;改成 0711 后通过);端到端 V1/V1b(0700 目录下的 LAYERFS_LOCAL_DIR:mount 返回 InvalidInput、没有 daemon、什么都没建)。修复前的二进制在同样配置下会挂载成功。
|
|
||
| /// The command identity: this user, or nobody when the tests run as root. | ||
| fn me() -> CommandIdentity { | ||
| let meta = fs::metadata("/proc/self").unwrap(); |
There was a problem hiding this comment.
Linux-only backend and tests are not gated by platform. The backend and these tests rely on /proc (stat, mountinfo, /proc/self), but neither has #[cfg(target_os = "linux")]. On a macOS host, cargo test -p layerfs-sandbox now fails: me() panics here and LocalProcess::observe errors, so all six tests fail. The backend also compiles into the macOS SDK and fails only at run time, with an io error instead of a clear Unsupported.
Suggestion: gate backend::local, its re-exports and this test file on Linux, as mounts.rs already partly does.
There was a problem hiding this comment.
已修:
backend/mod.rs:#[cfg(target_os = "linux")] pub mod local;(mounts.rs里原来的非 Linux 分支随之删掉);tests/local_runtime.rs顶部#![cfg(target_os = "linux")]。- SDK:
mod local、Runtime::Local、Recorded::Local及其所有分支、discovery::root()都只在 Linux 编译。非 Linux 上LAYERFS_SANDBOX=local返回Unsupported: LAYERFS_SANDBOX=local runs its daemon as a process of this kernel, which needs Linux; run it inside a Linux VM or CI box, or use Docker;遇到 local 记录也返回Unsupported,不会报 io 错。
验证:在 Linux 上装了 aarch64-apple-darwin 的 std,cargo clippy -p layerfs-sandbox --all-targets --target aarch64-apple-darwin -- -D warnings 通过。layerfs-node 没法在 Linux 上交叉检查(blake3、zstd-sys 的 C 构建需要 macOS SDK),这部分逐处审读过 cfg;我会再在 Mac 上跑一次 cargo clippy --all-targets 和 cargo test -p layerfs-sandbox,结果补在这里。
| let name = registry::text(value, "name")?; | ||
| if name.is_empty() | ||
| || !directory.is_absolute() | ||
| || directory.file_name().and_then(|n| n.to_str()) != Some(name) |
There was a problem hiding this comment.
The directory check doesn't keep a delete inside the local root. This compares the directory's last component with name, but name comes from the same record, so the check is easy to satisfy. The directory is never compared with the configured LAYERFS_LOCAL_DIR, and name is never compared with <prefix><project>-<id>.
Example: a record that was edited, corrupted, or left from an old LAYERFS_LOCAL_DIR, with directory: "/srv/data", name: "data". It passes this check, and since pid/start no longer match it is classified as stopped. reclaim → Recorded::stopped().delete() then lazily detaches every mount under /srv/data and runs remove_dir_all("/srv/data") as root.
Suggestion: require directory == Local::new(deploy.localDir)?.naming(expected_name)?.paths()?.directory, or at least directory.parent() == configured root.
There was a problem hiding this comment.
这条是安全问题,谢谢指出,已修。discovery::recorded 现在多接 args 和编号:local 记录只有在 directory 恰好等于本部署算出来的 <LAYERFS_LOCAL_DIR>/<prefix><project>-<id>、且 name 是这个目录名时才会被 reopen、列出或删除。算法只用配置(与 create 用同一个 sandbox_name()),不碰文件系统。localDir 没配置、不是绝对路径或含 ./..,同样拒绝。不符时返回 InvalidInput,说明记录写的是哪、本部署期望在哪、“neither used nor deleted”,以及该手动删除的 sandbox.record 路径,不会 detach 也不会删任何东西。
行为变化:只配置了 Docker(没有 LAYERFS_LOCAL_DIR)的进程,如果同一项目里有 local 记录,sandbox.list 会对它报错而不是去碰它——这是有意的 fail-closed。
测试:tests/unit/runtime.rs 覆盖 /srv/data+data、<root>/data、旧 LAYERFS_LOCAL_DIR、别的编号、名字不符、子目录、相对路径、..;tests/unit/reclaim.rs::a_local_record_outside_its_configured_directory_is_never_deleted 端到端:真实建一个有文件的 data/,记录指向它 → reclaim 返回 InvalidInput,文件和记录都还在;旧 LAYERFS_LOCAL_DIR 的记录也不删;位置正确的记录照常回收。
| /// True while this exact process exists and has not exited: a zombie | ||
| /// (or a dead entry) is not alive, and another process under the same id | ||
| /// has another start time. | ||
| pub fn alive(self) -> bool { |
There was a problem hiding this comment.
The process identity has no boot id. Liveness compares only the pid and the start time in clock ticks since boot. After a reboot or VM snapshot restore, with LAYERFS_HOME on a persistent disk, a different process can have the same pid and tick value. Disposable CI/microVM images boot deterministically, so this is more likely there.
A stale record is then taken as the live daemon. Reopen connects to the recorded port and fails auth with a confusing error instead of reclaiming, and any stop on it (e.g. from abandon) SIGTERMs/SIGKILLs an unrelated process as root.
Suggestion: record /proc/sys/kernel/random/boot_id with the pid and start time, and compare all three.
There was a problem hiding this comment.
已修。LocalProcess 增加 boot(/proc/sys/kernel/random/boot_id,每进程读一次),alive() 要求 pid、start、boot 三者都相同;记录新增 boot 字段。命令进程组的登记也带 boot,不同 boot 的登记一律跳过。
旧记录:只有这个分支合并前的构建写过 local 记录,没有任何发布版本写过。缺 boot 的记录仍按 pid + start 判活(这样不会把仍在跑的旧 daemon 误判为停止、删掉它的目录),但永远不会对它发信号(signal() 直接拒绝)。如果你们更希望不保留这段兼容、缺 boot 直接当作损坏记录拒绝,改动只有几行,告诉我就改。
测试:a_process_of_another_boot_is_not_alive_and_one_without_a_boot_is_never_signalled(同 pid + start、不同 boot → 不活,reopen 得到 stopped;缺 boot → 仍活,但经它 stop() 返回错误、daemon 没有被动);记录解析的单测覆盖了 boot 正确、缺失、非法三种情况。
| mounts: Default::default(), | ||
| processes: Default::default(), | ||
| }); | ||
| crate::discovery::save_sandbox( |
There was a problem hiding this comment.
A failure after startup orphans the root daemon and leaves a zombie. By this point the daemon runs and has authenticated. If save_sandbox, save_text or the intent removal fails, ? returns without stopping it. SandboxState and its Child are dropped without a wait. The outer start then calls remove_dir(saved_directory), which fails because the directory isn't empty.
Result: a root daemon keeps the Store open with no record, and later starts are refused by the leftover create.intent. In a long-lived Node process (e.g. dsh), the dropped Child is never waited, so once the daemon idle-exits it stays a zombie until Node exits.
Suggestion: send these errors through abandon(..., Some(sandbox)), or reap the child in a Drop.
There was a problem hiding this comment.
已修。save_sandbox、save_text("container") 和删除 intent 收进一个闭包;任何一步失败都会把 SandboxState 拆回 Sandbox,删掉已经写下的 sandbox.record / container,再走 abandon(..., Some(sandbox))(读日志尾 → stop → delete → 处理 intent,各一次,不重试),外层的 remove_dir(saved_directory) 因此能成功,编号释放。另外给 Sandbox 加了 Drop,只做一次 try_wait,回收本进程启动的、已经退出的 daemon,不会去 stop 任何东西,所以长驻的 Node 进程里不会留僵尸。
在 aarch64(qemu-user)上预跑时,这条的端到端检查还抓到一个竞态,一并修了:daemon 是多线程的,被停掉时主线程先变成 Z,alive() 已经是 false,但其它线程还没退完,这时 waitpid(WNOHANG) 还拿不到它。原来的 stop() 一看到“不活”就返回,delete() 和 Drop 又各只 try_wait 一次;三次都落在这个窗口里(qemu 下线程退得慢),Child 被丢掉,就留下一个永久僵尸。现在 stop() 要等本进程启动的 daemon 真正被 reap 后才返回(gone = reap() && !alive()),KILL 阶段也对还没 reap 的 Child 直接 kill()(没 reap 之前 pid 不会被复用),上限不变(TERM seconds + KILL 5 秒)。
测试(端到端 V4/V4b,故障注入):把 LAYERFS_DAEMON 换成一个包装脚本,它在 exec 真 daemon 前先建好 sandboxes/1/container,让 save_text 以 EEXIST 失败。结果:返回 Failed: local sandbox records: … File exists …; its daemon's last log lines: …;之后没有 daemon、没有僵尸子进程、LAYERFS_LOCAL_DIR 为空、没有 create.intent、sandboxes/1 不存在;V5 下一次 mount 正常拿到 Sandbox 1。修复前的二进制跑同样的注入,Sandbox 目录残留。
单测 tests/local_runtime.rs::a_stop_reaps_a_daemon_whose_main_thread_ended_first 把上面那个窗口做成确定性的:替身 daemon 用 python3 的 ctypes 调 pthread_exit 让主线程先退、另一个线程继续睡;断言 alive() 为 false 且线程组还在,然后 stop(1),断言进程已被 reap。改之前这个测试稳定失败,改之后通过(重复 20 次);没有 python3 的机器上它打印 skipped。修复后 aarch64 qemu 端到端连续 5 次 72/72。
|
|
||
| /// The image a local Sandbox records: placement keeps local and Docker | ||
| /// Sandboxes of one project apart by it. | ||
| pub(crate) const LOCAL: &str = "local"; |
There was a problem hiding this comment.
The placement key local can collide with a Docker image name. Local and Docker Sandboxes are kept apart only by recording image = "local" and placing by image string. local is a valid Docker image reference, so a Docker deployment with LAYERFS_IMAGE=local (e.g. a locally built image tagged local) would be placed onto running local Sandboxes. Its Workspace would then mount in a host process with no isolation, and the reverse can also happen.
Suggestion: place by (runtime, image), using the record's runtime field, instead of overloading the image string.
There was a problem hiding this comment.
确实会撞,已修。runtime::image() 换成 runtime::placement():记录的 runtime 必须和部署一致(Docker 记录没有这个字段,按 docker 处理,所以所有旧记录都兼容),Docker 再比 image;没有 deploy 时不过滤(和以前一样);Docker 部署没配 image 时也绝不会接 local 记录。local 记录仍写 image: "local",只为让 sandbox.list 的输出不变。
测试:placement_keeps_runtimes_apart_even_for_an_image_named_local(两个方向、旧 Docker 记录、没有 image、没有 deploy);端到端 V3/V3b:local Sandbox 1 在跑时,用 LAYERFS_SANDBOX 未设、LAYERFS_IMAGE=local 的客户端 mount,不再被放到 local Sandbox 上,而是走 Docker 自己的检查被拒(image must be an exact local image digest),local Sandbox 上没有新挂载。修复前的二进制在同样情况下真的挂到了 local Sandbox 上。
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/process.rs (production). No tests changed or deleted except additions. Production LOC: 109,658 -> 109,696 (delta +38) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/groups.rs (production). No tests changed or deleted except additions. Production LOC: 109,696 -> 109,818 (delta +122) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/exec.rs (production). No tests changed or deleted except additions. Production LOC: 109,818 -> 109,835 (delta +17) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/sandbox.rs (production). No tests changed or deleted except additions. Production LOC: 109,835 -> 109,868 (delta +33) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/src/backend/local/create.rs (production). No tests changed or deleted except additions. Production LOC: 109,868 -> 109,882 (delta +14) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: core/crates/layerfs-sandbox/tests/local_runtime.rs (not production (test)). No tests changed or deleted except additions. Production LOC: 109,872 -> 109,872 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/src/runtime.rs (production). No tests changed or deleted except additions. Production LOC: 109,872 -> 109,906 (delta +34) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/src/local.rs (production). No tests changed or deleted except additions. Production LOC: 109,906 -> 109,958 (delta +52) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/src/discovery.rs (production). No tests changed or deleted except additions. Production LOC: 109,958 -> 109,995 (delta +37) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/src/exec.rs (production). No tests changed or deleted except additions. Production LOC: 109,995 -> 110,005 (delta +10) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/src/sandbox.rs (production). No tests changed or deleted except additions. Production LOC: 110,005 -> 110,015 (delta +10) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/src/workspace.rs (production). No tests changed or deleted except additions. Production LOC: 110,015 -> 110,018 (delta +3) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/src/lib.rs (production). No tests changed or deleted except additions. Production LOC: 110,018 -> 110,019 (delta +1) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/tests/unit/runtime.rs (not production (test)). No tests changed or deleted except additions. Production LOC: 110,019 -> 110,019 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
…l-AI-Lab#402) Part of one change pushed file by file through the GitHub contents API; the reviewable unit is the whole set (local commit 9334dbd22, one patch). This commit: sdk/js/native/tests/unit/reclaim.rs (not production (test)). No tests changed or deleted except additions. Production LOC: 110,019 -> 110,019 (delta 0) Scope: combined core+sdk+dsh, tools/production_loc.py, parent 85255766e.
|
@yifanxuaaa 谢谢细致的评审,7 条都成立,已全部按建议修复,追加在 概要:
验证(测试机 x86_64,Debian 13,HEAD = 本次提交,工作树干净):
另外顺带说明一点(不是这次的评审项):main 上 b40da46 之后, |
…hemeral-AI-Lab#402 (local port) Sync by merge (no rebase, no force-push). Ephemeral-AI-Lab#428 moved sdk/js to api/agent/sdk and sdk/js/native to api/binding, made Sandboxes explicit (sandbox.create / sandbox.close / workspace.mount(sandbox, branch), no placement) and made LayerFS plugin-neutral (LAYERFS_* names only). Resolved: - api/binding/src/sandbox.rs: upstream's create/close/get_log kept; they now go through the Sandbox's recorded runtime (Docker or local) instead of naming the Docker backend: create reports runtime.image() (`local` for a local Sandbox), close reopens the recorded runtime and stops and deletes it, get_log reads the container's log or a local daemon.log. - api/binding/src/workspace.rs (upstream's new file): the mount result's image is runtime.image(); observed() reopens the recorded runtime, so list_sandboxes, reclaim_gone and runtime.sandbox.reclaim see local Sandboxes too. - api/binding/src/runtime.rs: Runtime::stop and Runtime::log_tail added; Placement removed with upstream's placement (its unit test with it). - api/agent/sdk/src/client.js: deployment() keeps the local branch (LAYERFS_SANDBOX=local, LAYERFS_LOCAL_DIR); the DSH_LAYERFS_ twins are gone as upstream removed them. - api/agent/sdk/src/index.js: the client reads LAYERFS_SANDBOX and LAYERFS_LOCAL_DIR from the environment (there is no options argument). - sdk/js/README.md -> api/how-to-use/agent.md: the two settings rows and the "Local Sandboxes (no Docker)" section, rewritten for explicit Sandboxes. - sdk/js/native/src/workspace.rs: deleted as upstream did; its placement change has no counterpart (there is no placement). - local.rs, runtime.rs, tests/unit/runtime.rs, local_deployment.mjs moved to api/binding and api/agent/sdk/test; the test uses the stand-in binding since clients are made only by LayerFS.open/init.
…497b0c v1.1.0) into feat/macos-native-on-main Sync by merge (no rebase, no force-push): the branch history others pulled stays as it was. ada323c is Ephemeral-AI-Lab#402's head 8c22d17 with upstream main 4497b0c merged in (Ephemeral-AI-Lab#428: sdk/js -> api/agent/sdk, sdk/js/native -> api/binding, dsh/ moved to the dsh-layerfs repository, explicit Sandboxes, plugin-neutral LAYERFS_* settings), so this one merge brings both. Resolved: - dsh/ (README and 8 host files we changed, plus tests/native.mjs and tests/native_real.mjs we added): removed with upstream; the Host's native route goes to dsh-layerfs as its own change (core-only plan). - api/agent/sdk/src/client.js: deployment() keeps `native` beside `local`; the DSH_LAYERFS_NFS_* fallback (nfsFallbacks) is dropped, as Ephemeral-AI-Lab#428 drops every DSH_LAYERFS_* name (AGENTS.md "Plugin-neutral LayerFS", owner 2026-10-24); an integration maps its own names. - api/binding/src/runtime.rs: native() kept, Placement removed (there is no placement since Ephemeral-AI-Lab#428); Runtime::stop and log_tail cover macOS too. - api/binding/src/discovery.rs: with no placement, a local or native record is used only by a deployment of its own runtime (InvalidInput otherwise), which keeps local and native Sandboxes apart as placement did. - api/binding/tests/unit/runtime.rs: placement tests dropped; native selection kept; the record test checks the runtime match. - sdk/js/README.md -> api/how-to-use/agent.md: the `native` settings text and the "Native Sandboxes (macOS)" section, for explicit Sandboxes and without the DSH_ fallback. - core/Cargo.lock: layerfs-nfs follows the workspace version 1.1.0.
|
merge 提交 ada323c(tree c14bac2c),父提交是 8c22d17 和 4497b0c,只用 merge。#402 原来和新 main 有 9 个冲突文件,处理方式: 回归(Linux):fmt 0;Linux/Darwin clippy -D warnings 0;sandbox 38、binding 11、store-service 3 全过;SDK/CLI/runtime/backend JS 测试和 main 一致;LAYERFS_SANDBOX=local smoke(新 API,包括 sandbox.create/close、kill 后回收)48/48。 另外:local_runtime 用 4 个线程跑时偶尔有一个用例失败,同步前后都是 15 次里失败 1 次,单独跑能过。现在它会进 CI 的 fast lane,建议后续修一下。 |
…name) into PR Ephemeral-AI-Lab#402 (local port)
Sandbox::stop() ended the daemon first and left the Sandbox's mounts in place until Delete detached them. Detach them first, while the daemon can still answer, then stop the daemon as before. A failed detach is reported and does not block the crash teardown; Delete keeps its own checked detach before removing anything through a mount. On Linux FUSE this is ordering only: Stop now returns with the mounts already gone (measured 21.6 ms with a live holder and a dirty writer; before the change the mounts stayed until Delete), and reclaim is unchanged (0.09-0.10 s). MNT_DETACH does not flush dirty pages, so it adds no data guarantee; Stop stays the crash teardown it was. Same hunk as the local-backend part of the macOS native branch's NFS unmount-hang fix, so merging this branch there stays clean.
|
新提交 766e67e:fix(sandbox): detach local mounts before stopping the daemon 问题 改法(只改 core/crates/layerfs-sandbox/src/backend/local/sandbox.rs,+8/−1) 在 Linux FUSE 上这只是顺序变化: 这一段和 #427 里 NFS 卸载挂死修复中的本地后端部分完全相同(#427 那边还有 NFS helper 有上限回收和 binding 卸载期间放锁,这些只和 NFS 路线有关,没放进本 PR),所以以后 #402 merge 进 #427 不会冲突。 验证(Linux x86_64,头 766e67e) |
…er_reclamation The Store binds no busy handler, so reclaim_space can be refused Busy without effect while the concurrent reader holds its snapshot. The test unwrapped that outcome and failed intermittently (4 of 80 local runs, and PR #402 CI). Treat Busy as the documented refused job and run a later one, bounded by JOBS. No tests deleted; no assertion changed. Production LOC: 102165 -> 102165 (delta 0) Test-only. core 38241, engine 61291, api 2633, unchanged; reference absent (retired R9c; last counted 65,417 at tree 498dd19). Method: tools/production_loc.py --root on git archive of the first parent and the committed tree (bb3f715 -> 7cf6e7b). Added by the maintainer after the rebase onto main 3828268 (#465 moved the crate to engine/).
…amation The Store binds no busy handler, so reclaim_space can be refused Busy, without effect, when it meets the concurrent reader; the test unwrapped that and failed intermittently (seen in #402 CI). The test now treats Busy as the documented refused job and sends a later one, at most JOBS times; other errors still fail and no assertion changed. Test-only; the owner accepted this caller-side retry in a test as consistent with the reclaim_space contract (a refused job is never replayed automatically, a caller may issue another). Rebased by the maintainer onto main after #465 moved the crate to engine/, and the missing LOC line added. Checked on macOS (cargo 1.85.1): immutable_reads_under_reclamation passes (1147 jobs during 89 read rounds); all 36 layerfs-persistence test binaries pass (118 tests); fmt and Clippy -D warnings clean. Production LOC: 102165 -> 102165 (delta 0) First-parent comparison (main 3828268 -> this merge). core 38241, engine 61291, api 2633, unchanged; reference absent (retired R9c; last counted 65,417 at tree 498dd19). Counted with tools/production_loc.py; test-only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…) into PR Ephemeral-AI-Lab#402 (local port)
…meral-AI-Lab#402) Upstream now keeps the Node binding's tests outside the crate source: they load the ordinary built addon and call its exported methods, never including private binding source (engine/AGENTS.md, engine/crates/layerfs-node/tests). The Ephemeral-AI-Lab#402 unit tests that were included by runtime.rs and sandbox.rs (tests/unit/{runtime,reclaim}.rs before D8) are rewritten as engine/crates/layerfs-node/tests/native_local.rs, which writes a Sandbox record as an earlier call would and calls the public `runtime.sandbox.reclaim` with a local deployment: - a local record naming any other directory (a data directory, another LAYERFS_LOCAL_DIR, another Sandbox or name, a relative or `..` path, `/`) is refused InvalidInput, "neither used nor deleted", and nothing is removed; - a local record is used only with an absolute, plain deploy.localDir; - a stopped local Sandbox where this deployment puts it is deleted and its number freed; a record from before boot ids is read without one; a damaged boot or port is refused and removes nothing; - a Docker record is not read as a local Sandbox. The Docker reclaim cases of the old reclaim.rs are covered by upstream's native_discovery.rs. Selection of a misspelled deploy.sandbox is refused only by sandbox.create after the project's Store is installed, so it is not reachable without a Store and is left to the SDK test (local_deployment.mjs). Production LOC: 103762 -> 103762 (delta 0) (tools/production_loc.py --root ., core+engine+api; test-only commit)
Production LOC: 103762 -> 103762 (delta 0) (tools/production_loc.py --root ., core+engine+api; formatting of a test file only)
…-Lab#402) Production LOC: 103762 -> 103762 (delta 0) (tools/production_loc.py --root ., core+engine+api; documentation only)
…e built addon) into PR Ephemeral-AI-Lab#427 (local port) Brings Ephemeral-AI-Lab#402's engine/crates/layerfs-node/tests/native_local.rs. One conflict: api/binding/tests/unit/runtime.rs, deleted on Ephemeral-AI-Lab#402 and kept by Ephemeral-AI-Lab#427's D8 merge, is deleted here too. api/binding/ no longer exists after D8, the file was included by nothing, and its cases move to native_local.rs in the next commit. Production LOC: first-parent comparison 1931032 -> this merge, see tools/production_loc.py --root .; tests only, delta 0.
…nto PR Ephemeral-AI-Lab#427 (local port) Production LOC: 108596 -> 108596 (delta 0), first-parent comparison (tools/production_loc.py --root ., core+engine+api; documentation only)
改了什么
设置
LAYERFS_SANDBOX=local时,一个 Workspace 的 Sandbox 不再是 Docker 容器,而是本机上的一个layerfs-daemon进程。这样workspace.mount / exec / status / commit / unmount可以在只有 FUSE、没有 Docker 的 Linux 环境里运行(例如 Apple Silicon 上的 microVM、CI job)。docker:行为与现在完全相同(Docker 路径一行没改,只是在 SDK 里多包了一层枚举)。lokal):在产生任何副作用之前以InvalidInput拒绝,而不是静默落回 Docker(理由见下方「设计取舍」)。改动范围(22 个文件,+2191 / −120;Production LOC 108,451 → 109,658,+1,207):
core/crates/layerfs-sandbox/src/backend/local/(新)Local::create:建 Sandbox 目录、写 0600 的 daemon 配置、以干净环境和独立进程组启动layerfs-daemon --config、等它打印LAYERFS_DAEMON_LISTEN(只接受 loopback)。Sandbox:用 pid + 内核启动时间判活(僵尸、pid 复用都不算活);stop先 TERM 后 KILL;delete只在确认停止后执行,先MNT_DETACHdaemon 死后留下的 FUSE 挂载、确认没有挂载了,再删目录。run:以命令身份在 Workspace 里执行一条命令,双路流式输出,被信号杀死报 128+sig(与 Engine 相同)。core/crates/layerfs-sandbox/Cargo.tomlnix同一个钉死版本多开signal/mount/userfeature;没有新 crate,Cargo.lock不变sdk/js/native/src/runtime.rs(新)Runtime枚举:Docker(容器)/Local(本地 Sandbox),加后端选择sdk/js/native/src/local.rs(新)ControlReadysdk/js/native/src/{sandbox,discovery,registry,exec,workspace,lib}.rsRuntime;本地记录多存 runtime、目录、pid、启动时间、端口;目录不是「绝对路径 + 无..+ 末段等于记录名」的记录一律拒绝sdk/js/src/{client,index}.js、README.mddeployment()读LAYERFS_SANDBOX/LAYERFS_LOCAL_DIR(及DSH_同名变量);README 新增「Local Sandboxes (no Docker)」一节和无隔离声明layerfs-sandbox/tests/local_runtime.rs(6 个,用替身 daemon,不需要 root);layerfs-node/tests/unit/runtime.rs(2 个);sdk/js/test/local_deployment.mjs。没有删改任何已有测试。layerfs-api、layerfs-daemon、layerfs-bridge、backend::docker均未改动。为什么
layerfs-api的SandboxApi泛化成多后端(会改公开类型并波及 tests/examples/benchmark),也没有写 Docker API 兼容 shim(要逐字段伪造 inspect/topology)。SDK 里本来就有用公开 API 自己握手的代码(pool.rs),本地启动直接复用它。local 后端只适合在本身就是隔离边界的 VM 或 CI 机器里使用。 daemon 以 root 运行在机器自己的命名空间里;命令与机器共享进程表、网络和文件系统,只是换成 uid 501 / gid 20(去掉附加组)执行。Docker 后端的 pid/mount/net/user 命名空间、seccomp、no-new-privileges 在这里都没有。另外:
/dev/fuse收紧为 0600 root(daemon 既有行为),在共享主机上会影响其它非 root 的 FUSE 用户;dialout;若 VM 里恰好存在真实的 uid 501 账号(如 Lima/colima 映射 macOS 用户),命令就以那个账号的身份运行。需要 root:非 root 时在任何副作用之前以
Unsupported拒绝,并说明原因。设计取舍(请维护者判断)
LAYERFS_SANDBOX被拒绝,而不是默认走 Docker。 拼错的人本意是 local,静默落到 Docker 只会在没有 Docker 的 VM 里得到一个难懂的docker.sock ENOENT。对现有用户零影响(以前没有这个变量;未设或设为docker时行为一致,有单测和端到端覆盖)。如果更希望「未知值回落 Docker」,改runtime::local()和client.js各一行即可。sandbox::setup()自 SDK redesign (JS-only over a Rust implementation by group/method), commit-time merge and conflicts, and live wiring of the sdk-sim-terminals UI #327 起写死、Docker 后端也在用的同一个常量;保持一致可以让同一个 Store 在两种后端之间切换时属主语义相同。若要做成可配置,建议两个后端一起改,不在本 PR 范围。storePath,所以只在这次传给 daemon 的配置副本里把store和existing_store.locator都写成这个路径(locator 的定义就是「daemon 文件系统里的路径」);Init 写进 manifest 的原值不改。怎么验证的
测试机(x86_64 Linux,Docker 原生)——全部在最后一次源码改动之后、对干净 HEAD 复跑
smoke-local.mjs:61/61。 覆盖每个 workspace.* 调用及失败路径(NotMounted、NoSuchBranch、NoSuchSandbox、非零退出、用户自己的timeout→ 124、无隐式超时、3 MiB 输出)、后台进程与 unmount Busy、第二个进程重开 Sandbox、同一 Branch 挂两次与提交冲突、3 个 Workspace 并发、merge 与 merge 冲突、SIGKILL daemon → status/exec 被拒 → sandbox.list 回收(卸掉残留的 ENOTCONN 挂载)→ 重新 mount、已提交数据仍在、空闲 120 s 自关、无残留进程/挂载/临时文件。smoke-docker.mjs的 39 行签名,本分支(未设 /=docker)与 origin/main 逐行相同;上游真实 Docker verify 测试新旧结果一致;没有 Docker 时新旧给出相同的拒绝。LAYERFS_LOCAL_DIR→ InvalidInput;非 root → Unsupported 且不留 intent/记录/目录。cargo +1.85.1 test --release -p layerfs-sandbox -p layerfs-node44/44;本改动文件cargo fmt --check、clippy -D warnings干净(剩下的报错都在上游既有文件,另有 fmt / clippy 小 PR 处理)。Apple Silicon microVM(aarch64)
docker run --platform linux/arm64 --privileged node:22-trixie-slim,Store 在 ext4):61/61,非 root 拒绝 PASS,无残留。node:22-trixie-slim,2 vCPU / 1 GB,Store 在 VM 私有 ext4 磁盘:LOCAL_SMOKE 61/61,非 root 拒绝 PASS,leftovers none;smoke 135.9 s(其中 130 s 是等待空闲自关),整次msb run137 s,宿主进程最大 RSS 306 MB(含 VM),Mac 负载前后无明显变化。guest:aarch64,kernel 6.12.111,glibc 2.41,node v22.23.4。风险
LAYERFS_SANDBOX=local启用,默认路径不受影响。MNT_DETACH并确认 mountinfo 中已无挂载才删除目录;端到端 K4/K5 覆盖。..、末段等于记录名,否则拒绝,避免删除被引到别处。/tmp(HOME 与后台日志已放进 Sandbox 私有 tmp)。在 VM/CI 的一次性场景里可接受,需要时可后续补。port/,macOS 上仍走 Docker 容器)不冲突;feat(rutis): add Tier 1 Workspace support #393(已关闭)若重开、把sdk/js/native挪目录,本 PR 新增的两个文件需随之搬动;fix(sdk): clear layerfs-node clippy and fmt checks on 1.85.1 #397、Rust LayerFS MCP #388、fix(core): clear the boundary guard's core violations; rustfmt; LF line endings #395 无重叠文件。