Skip to content

Support atomic delete-by-identity for sandboxes (no compare-and-delete primitive today) #3210

Description

@yanyunl1991

Problem Statement

OpenShell's sandbox delete API takes only a mutable sandbox name (openshell sandbox delete -g <gateway> <sandbox-name>). There is no way for a client to bind a delete request to a specific sandbox's immutable identity (internal ID / resource version) and have OpenShell refuse the delete if the name now resolves to a different sandbox.

This means any client-side "read identity, then delete by name" pattern has an unavoidable TOCTOU race: another OpenShell client can delete the sandbox and create a replacement under the same name between the client's identity read and OpenShell processing the delete. No amount of re-checking identity immediately before issuing the delete closes this window, because the check and the delete are not atomic.

Impact

We hit this concretely in NVIDIA/NemoClaw#10863 / NVIDIA/NemoClaw#10867: nemoclaw destroy needed a safe way to clean up a sandbox left "retained" after an interrupted onboarding. Even with a durable identity fingerprint recorded for the retained sandbox, we could not safely automate the delete, because OpenShell could not guarantee the name still pointed at the same sandbox at the moment of deletion. The fix had to make deletion of a live retained sandbox permanently fail-closed (always require a human to run the delete manually after out-of-band identity confirmation), which is a worse operator experience than a race-free automatic cleanup would be.

Proposed Design

Add an atomic delete-by-identity primitive, e.g. a resource_version / immutable ID precondition on the delete RPC (compare-and-delete semantics): the delete succeeds only if the sandbox at that name still has the identity/resource-version the caller expects, and otherwise fails with a distinguishable "identity mismatch" error rather than deleting an unexpected sandbox.

This is related to #3050 (unifying sandbox references across gateway RPCs) — a canonical sandbox reference type that includes an immutable ID could be a natural carrier for this precondition, but #3050's stated scope does not currently call out atomic/conditional delete semantics, so filing this separately to track the specific capability gap.

Acceptance Criteria

  • Sandbox delete RPC accepts an optional identity/resource-version precondition.
  • Delete fails with a distinguishable error (not a silent no-op or wrong-sandbox deletion) when the precondition does not match the current sandbox at that name.
  • Behavior documented for SDK/CLI consumers.

cc @jyaunches (raised during review of NVIDIA/NemoClaw#10867)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions