Skip to content

atelet checkpoint idempotency - #948

Draft
Troy Chiu (troychiu) wants to merge 3 commits into
agent-substrate:mainfrom
troychiu:fix-372-checkpoint-idempotency
Draft

atelet checkpoint idempotency#948
Troy Chiu (troychiu) wants to merge 3 commits into
agent-substrate:mainfrom
troychiu:fix-372-checkpoint-idempotency

Conversation

@troychiu

Copy link
Copy Markdown
Contributor

Towards #372

Makes Checkpoint safe to replay. Today a lost response (an atelet restart,
a deadline exceeded mid-call) leaves the caller unable to tell "never
started" from "finished, and the answer went missing" — and retrying drives
the sandbox runtime against state the first attempt destroyed, which is the
runsc fscheckpoint … exit status 128 in the issue.

  • atelet probes the destination for manifest.json before doing
    anything and returns success when it is there. The manifest is a
    trustworthy commit marker because both persist paths write it last, and
    the control plane mints the destination once per suspend/pause, so a
    manifest there names this checkpoint and no other. moveLocalCheckpoint
    becomes resumable to match: a rename is not repeatable, so a file already
    at the destination is recognized as this checkpoint's own earlier work.
  • ateom (both runtimes) records the snapshot file list on disk once the
    files are written, and replays it instead of re-running the checkpoint.
    A failure with the sandbox confirmed gone is now reported as DataLoss
    with the actor-crash directive, so the control plane records unrecoverable
    state instead of an opaque runtime exit status.

Run and Restore need a per-activation idempotency key (actor UID is
stable across activations) and follow in a second PR.

  • Tests pass
  • Appropriate changes to documentation are included in the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant