Gracefull shutdown with MJPEG streams connected

[julianstirling]

If an MJPEG stream is connected then the server will not graefully shutdown.

Partial fix
Add timeout_graceful_shutdown=2, to the uvicorn.run line to ensure that it shuts down within 2 seconds even if the server cannot shut down the streams.

Full fix

In the OpenFlexure server each camera has a kill_mjpeg_streams() method which sets the _streaming attribute of the MJPEG stream to False.

When creating the server we define this function:

def shutdown_call() -> None:
    try:
        camera_thing = server.things["camera"]
        if not isinstance(camera_thing, BaseCamera):
            raise RuntimeError("Camera thing is not a BaseCamera")
        camera_thing.kill_mjpeg_streams()
    except BaseException as e:
        # Catch anything and log as it is essential that this
        # function cannot raise an unhandled exception or Uvicorn
        # will never get a shutdown signal.
        LOGGER.error(e, exc_info=True)

And run it with set_shutdown_function(shutdown_call) where:

from uvicorn.main import Server

def set_shutdown_function(shutdown_function: Callable[[], None]) -> None:
    """Ensure a function is called before the shutdown.

    This monkey patches the Uvicorn Server's handle_exit. This is needed because
    the uvicorn ``lifecycle`` events and FastAPI ``shutdown`` events only fire once
    background tasks have completed.

    Without this the system exits cleanly only if no client is receiving a
    StreamingResponse. This patch is used to stop the async generators that
    send streaming responses.

    :param shutdown_function: A callable with no arguments or outputs. This
        should stop any async generators that may be sending to streaming responses.
    """
    original_handler = Server.handle_exit

    @wraps(Server.handle_exit)
    def handle_exit(*args: Any, **kwargs: Any) -> None:
        shutdown_function()
        original_handler(*args, **kwargs)

    # Ignore the MyPy doesn't want us monkey patching. We have to unless the
    # FastAPI lifecycle is fixed.
    Server.handle_exit = handle_exit  # type: ignore[method-assign]

LathThings FastAPI could keep a list of MJPEG streams on the thing server itself. So then ThingServer.kill_streams() method could kill all registered streams. The Uivcorn.Server would still need to be monkey patched.

[rwb27]

This does feel like it wants to be adopted in LabThings rather than downstream - I think we said that when you came up with the fix but never got around to implementing it.