[Feature]: Add durable terminal-scope API for cleanup and compensation

[zhongkechen]

What would you like?

Add an SDK-supported durable terminal-scope API for registering cleanup and compensation actions that run when a scope reaches a true terminal outcome, but do not run when the current invocation merely suspends.

The exact API name and shape should be decided through an RFC/ADR. An illustrative Java API is:

return context.terminalScope(
        "review-with-microvm",
        ReviewResult.class,
        (scopeContext, terminal) -> {
            var vm = scopeContext.step(
                    "launch-microvm",
                    MicroVm.class,
                    stepContext -> launchMicroVm());

            terminal.cleanup(
                    "terminate-microvm",
                    cleanupContext -> cleanupMicroVm(vm.id()));

            terminal.compensate(
                    "cancel-review",
                    compensationContext -> cancelReview(vm.id()));

            var result = scopeContext.waitForCallback(
                    "review-complete",
                    ReviewResult.class,
                    (callbackId, stepContext) ->
                            dispatchReview(vm, callbackId));

            return result;
        });

The important capability is the lifecycle contract, not the proposed names:

Scope outcome	Compensation actions	Cleanup actions
Success	Do not run	Run
Application/operation failure	Run in reverse registration order	Run
Explicit cancellation/termination	Policy-controlled; preferably enabled by default	Run when the runtime can execute terminal work
Durable suspension/replay boundary	Do not run	Do not run

Problem

Java normally encourages cleanup through try/finally or AutoCloseable. That model is unsafe around durable operations because the Java SDK currently suspends by throwing SuspendExecutionException, which unwinds the synchronous call stack. Java therefore executes active finally blocks during a normal suspension.

This pattern can release a resource while the durable execution is still logically using it:

try {
    var vm = context.step(
            "launch-microvm",
            MicroVm.class,
            stepContext -> launchMicroVm());

    return context.waitForCallback(
            "review-complete",
            ReviewResult.class,
            (callbackId, stepContext) -> dispatchReview(vm, callbackId));
} finally {
    context.step(
            "terminate-microvm",
            Void.class,
            stepContext -> terminateMicroVm());
}

When waitForCallback suspends, finally runs and terminates the MicroVM. The same issue applies to any suspending operation, including waits, wait-for-condition polling, invokes, retry delays, and suspension within map or parallel branches.

Users can manually duplicate cleanup after the success path and in catch (Exception):

try {
    var result = performDurableWorkThatMaySuspend(context);
    context.step("cleanup", Void.class, stepContext -> cleanup());
    return result;
} catch (Exception error) {
    context.step("cleanup", Void.class, stepContext -> cleanup());
    throw error;
}

However, that workaround:

• duplicates orchestration code;
• becomes difficult to maintain with multiple acquired resources;
• makes reverse-order compensation cumbersome;
• is easy to implement inconsistently across nested, map, and parallel scopes;
• does not provide a clear cancellation policy;
• encourages users to reach for finally, which has the wrong durable lifecycle semantics;
• cannot express the intent as clearly as an SDK lifecycle primitive.

Goals

• Provide an idiomatic way to express work that must occur on logical completion/failure rather than invocation exit.
• Make suspension a distinct non-terminal outcome and guarantee that terminal actions do not run because of it.
• Support general orchestration scopes, not only callbacks or resource acquisition.
• Support both unconditional terminal cleanup and failure-only compensation.
• Preserve deterministic replay and stable durable-operation identity.
• Ensure terminal actions are themselves durable, retryable, observable, and replay-safe.
• Work inside top-level handlers and isolated child contexts used by map/parallel operations.
• Leave room for a language-neutral lifecycle contract that other Durable Execution SDKs can expose idiomatically.

Non-goals

• Guarantee that cleanup runs after infrastructure-level hard termination when no invocation is available to execute it.
• Replace external leases, TTLs, or reapers for resources that must eventually be reclaimed under every failure mode.
• Provide exactly-once external side effects. Cleanup and compensation steps still require normal durable-step idempotency.
• Treat suspension as failure, cancellation, or scope completion.
• Reuse AutoCloseable, try-with-resources, or JVM finally; those constructs are tied to stack unwinding, not durable terminal state.

Possible Implementation

1. Scope and registration model

terminalScope could execute a deterministic body against a child durable context and provide a registration object:

<T> T terminalScope(
        String name,
        Class<T> resultType,
        DurableTerminalScopeFunction<T> body);

Illustrative registration API:

interface DurableTerminalActions {
    void cleanup(String name, DurableTerminalAction action);
    void compensate(String name, DurableTerminalAction action);
}

Possible future overloads could accept configuration for action ordering, retry policies, cancellation behavior, and cleanup-error aggregation.

Registrations should be deterministic declarations. On each replay, execution reruns the scope body and reconstructs the same registrations before reaching the same suspension or terminal path. Registration names and ordering must remain stable for a given checkpoint history.

2. Suspension handling

With the current synchronous Java execution model, the scope implementation can distinguish internal suspension from a terminal failure:

try {
    T result = body.run(childContext, actions);
    runCleanup(actions);
    return result;
} catch (SuspendExecutionException suspension) {
    // Suspension is not terminal. Run no compensation or cleanup.
    throw suspension;
} catch (Exception failure) {
    runCompensation(actions);
    runCleanup(actions);
    throw failure;
}

This is only conceptual pseudocode. The implementation must preserve all SDK control-flow errors and should not accidentally convert or swallow Error instances. The final design should use an internal outcome classifier rather than encourage broad user-visible catch (Throwable).

3. Durable identity and replay

The scope should own stable namespaces for:

• body operations;
• each registered compensation;
• each registered cleanup;
• terminal phase/progress if explicit checkpointing is required.

Terminal actions must execute through durable child contexts or equivalent SDK-managed operations. Completed actions must consume checkpoints on replay and must not repeat their bodies.

The design must define behavior when:

• the body suspends before all registrations are reached;
• the body fails after acquiring several resources;
• a compensation action suspends;
• a cleanup action suspends;
• a compensation or cleanup action fails and is retried;
• replay observes some terminal actions completed and later actions pending;
• a scope is nested inside another terminal scope;
• a scope is used within each item of a map or branch of a parallel operation.

Registration should not rely only on ephemeral in-memory state after a terminal phase begins. Either deterministic replay must reconstruct the complete registration set before resuming terminal work, or the SDK must durably record sufficient scope metadata to resume it safely.

4. Ordering

Suggested defaults:

• compensations run in reverse registration order, matching the usual acquisition/rollback model;
• cleanups run in reverse registration order so resources unwind like a stack;
• cleanup still runs if a compensation fails, where execution policy permits;
• multiple failures are preserved through a documented primary/suppressed or aggregate-error model.

Configuration could later permit forward or parallel execution, but the initial API should favor deterministic sequential behavior.

5. Error semantics

The RFC should specify:

• whether all terminal actions are attempted after one fails;
• how the original body failure is preserved;
• how cleanup/compensation failures are attached or aggregated;
• which retry configuration applies to each action;
• whether a failed cleanup changes an otherwise successful scope into a failed scope;
• what happens if terminal actions exhaust retries;
• how cancellation arriving during terminal processing is handled.

A reasonable initial policy is:

• on body failure, preserve the body failure as primary;
• attempt remaining compensation and cleanup actions;
• attach terminal-action failures as suppressed/aggregate details;
• on body success, a cleanup failure fails the scope;
• allow per-action retry configuration using existing step retry semantics.

6. Cancellation and hard termination

Cancellation must be defined separately from suspension.

The API could expose a policy such as:

TerminalScopeConfig.builder()
        .compensateOnCancellation(true)
        .cleanupOnCancellation(true)
        .build();

The contract must acknowledge that terminal actions can run only when the Durable Execution service schedules code to process the terminal transition. For externally terminated compute or lost invocations, users still need resource-native leases, TTLs, idempotent deletion, or a reaper as a safety net.

7. Observability

Execution history and telemetry should make the lifecycle explicit:

• scope started;
• scope suspended without terminal actions;
• terminal outcome selected;
• compensation started/completed/failed;
• cleanup started/completed/failed;
• scope terminal processing completed.

This is important for diagnosing cleanup that is delayed, retried, or partially completed.

8. Compatibility and rollout

This can be introduced as an additive API, so it should not be a breaking change. Because it creates a new durable orchestration primitive and potentially a cross-SDK lifecycle contract, it should go through an RFC/ADR and conformance review.

If a new operation/checkpoint type is introduced, backward compatibility and older-runtime behavior must be specified. If implemented initially as SDK composition over existing child contexts and steps, operation identity and upgrade behavior still require tests.

9. Testing and acceptance criteria

• Suspension from each suspending operation type runs no registered terminal action.
• Replay reconstructs registrations deterministically.
• Success runs cleanup exactly through durable/checkpointed semantics.
• Failure runs compensation and cleanup in the documented order.
• Cancellation follows an explicit, tested policy.
• A terminal action can itself suspend and resume.
• Completed terminal actions are skipped on replay.
• Terminal-action retries do not repeat already completed actions.
• Nested terminal scopes behave deterministically.
• Map/parallel branches use isolated scope contexts and do not interfere.
• Multiple body/compensation/cleanup failures preserve documented error information.
• API docs warn that cleanup side effects remain at-least-once and should be idempotent.
• Examples include the MicroVM lifecycle use case.
• Java error-handling documentation cross-links the feature when available.
• Language-neutral requirements are proposed if this becomes a cross-SDK capability.

10. Open design questions

• Should cleanup run on success and failure while compensate runs only on failure/cancellation, or should outcome-specific hooks be exposed directly?
• Is registration-before-acquisition required, or may a resource be acquired and then registered?
• Must registrations be checkpointed when declared, or is deterministic reconstruction sufficient?
• Should terminal actions receive the body result or failure?
• Should actions be Java lambdas captured during replay, serializable descriptors, or named operations reconstructed by user code?
• Should terminal processing be a new service-visible operation type or SDK composition over existing primitives?
• What is the cancellation contract supported by the service today?
• How should cleanup behave when a map uses early completion and abandons unfinished branches?
• Should compensation be included in the first version or added after a cleanup-only terminal scope?
• What naming best distinguishes durable terminal lifecycle from JVM lexical scope?

Is this a breaking change?

No. The proposal is an additive API.

Does this require an RFC?

Yes. It introduces new lifecycle semantics, replay rules, error aggregation, cancellation policy, and likely cross-SDK considerations.

Additional Context

The proposed terminalScope shape is a synthesis for Durable Execution rather than a direct copy of another SDK. Related established patterns include:

• the Saga pattern and reverse-order compensation;
• Temporal Java's Saga.addCompensation(...) / compensate();
• Cadence Java's similar Saga helper;
• workflow-engine compensation handlers such as BPMN compensation;
• asynchronous resource APIs such as Reactor usingWhen, which distinguish completion, error, and cancellation cleanup but are not durable/replay-aware.

The durable-specific requirement is to distinguish suspension from all terminal outcomes. A normal invocation boundary must not trigger cleanup or compensation.

Related issues:

• Java documentation for the current finally hazard: [Docs]: Document finally cleanup behavior during durable suspension #645
• Cross-language documentation pattern: Add pattern: Cleanup on completion or failure (not on suspension) aws-durable-execution-docs#131
• Detailed cross-language discussion: Add pattern: Cleanup on completion or failure (not on suspension) aws-durable-execution-docs#131 (comment)