fix(ds4): cancel abandoned inference - #11822
Open
FiloSpaTeam wants to merge 1 commit into
Open
Conversation
Propagate gRPC cancellation into DS4 prompt synchronization and poll it at decode boundaries. Stop on failed stream writes and skip parser finalization and KV persistence for abandoned partial requests. Assisted-by: Codex:gpt-5.6-sol Signed-off-by: Claudio Maradonna <git@codeshifter.xyz>
FiloSpaTeam
force-pushed
the
fix/ds4-request-cancellation
branch
from
September 1, 2026 14:35
852c096 to
178157a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Propagate gRPC cancellation into DS4 prompt synchronization and poll it at decode boundaries.
Stop on failed stream writes and skip parser finalization and KV persistence for abandoned partial requests.
Assisted-by: Codex:gpt-5.6-sol
Description
This PR fixes DS4 inference continuing after the originating gRPC request has been cancelled or disconnected.
Cancellation is propagated into prompt synchronization and checked at safe decode boundaries, between accepted MTP tokens, during distributed route waits, and after streaming write failures.
Abandoned requests do not finalize incomplete parser state or persist partial KV-cache state. Backend failures retain their existing error status rather than being incorrectly converted to cancellation.
Notes for Reviewers
grpc-server.cpp, so this branch may require a small rebase after the generation-boundary PR merges.Signed commits