Skip to content

SdkInternalException raised by user code stalls the attempt until inactivity_timeout #239

Closed

Description

@sharosoo

If a handler or wrapper raises restate.exceptions.SdkInternalException, ServerInvocationContext.enter() swallows it (python/restate/server_context.py:573-574, except SdkInternalBaseException: pass) without calling vm.notify_error. The attempt then sits in leave() inside block_until_http_input_closed() until the server's inactivity_timeout expires and forces a suspend. Every retry stalls for the full inactivity timeout.

Observed behavior

Measured on restate-sdk 1.0.5 (main 63d2fe5) against restate 1.7.13, on a service with inactivity_timeout=timedelta(seconds=10):

  • Handler raising RetryableError: 6 attempts in 30 s (retry gaps 0.6 s, 1.3 s, 2.3 s, 4.5 s, 8.0 s).
  • Handler raising SdkInternalException: 3 attempts in 30 s (retry gaps 10.6 s, 11.3 s; each gap is the full 10 s inactivity_timeout).
  • Nothing is logged on the SDK side when enter() catches SdkInternalBaseException.

The pass branch makes sense when the SDK raises SdkInternalException itself inside must_take_notification or create_poll_or_cancel_coroutine after the VM has already recorded the failure. Because SdkInternalException and is_internal_exception are importable from restate.exceptions, user code or middleware that raises or re-raises it enters the same except SdkInternalBaseException: block without notifying the VM. Related: #225 and #233 hit the same exception class from wrapper code.

Suggested fix

  1. Track whether the internal exception originated from the SDK's own VM/poll coroutine, and treat an SdkInternalBaseException raised directly by the user handler before any SDK abort as a retryable error via on_error (or log a warning before returning).
  2. Document next to TerminalError / RetryableError that SdkInternalBaseException is reserved for internal SDK control flow and must not be raised by handlers.

Minimal handler snippet

from datetime import timedelta
import time
import restate
from restate.exceptions import SdkInternalException

svc = restate.Service("probe", inactivity_timeout=timedelta(seconds=10))


@svc.handler()
async def fast_fail(ctx: restate.Context) -> None:
    print(f"fast_fail at {time.monotonic():.1f}s")
    raise restate.exceptions.RetryableError("boom")


@svc.handler()
async def stalled_fail(ctx: restate.Context) -> None:
    print(f"stalled_fail at {time.monotonic():.1f}s")
    raise SdkInternalException()

Activity

  1. sharosoo commented on Oct 19, 2026

    @sharosoo
    Author

    Closing: SdkInternalException is an internal SDK control-flow exception and is not intended to be raised by user handlers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions