fix(interruption): bind inference responses to the overlap they were requested for - #2306
Open
mastermanas805 wants to merge 2 commits into
Open
fix(interruption): bind inference responses to the overlap they were requested for#2306mastermanas805 wants to merge 2 commits into
mastermanas805 wants to merge 2 commits into
Conversation
…requested for `bargein_detected` / `inference_done` were matched to whichever overlap was open when they landed, not the one their request was cut for. A response arriving after its own overlap ended, while a later overlap was open, was accepted and attributed to that later overlap — emitting `isInterruption: true` for user audio the model never scored that way, and cutting the agent off. The cache clear at each overlap boundary was not a filter: `setOrUpdate()` recreates a missing entry from scratch, and `detectionDelayInS` was computed from the new overlap's start, so the result looked like a well-formed verdict. Plain response latency reaches this — the gap between two overlaps in one agent turn is often only a few hundred ms. Stamp the overlap generation onto each request in `sendAudioData()` and reject responses whose generation is no longer open. The generation ledger is kept outside `cache`, since `cache` is cleared at exactly the boundary where a late response still needs to be identifiable, and is bounded well above the request rate of a single overlap. The check fails open: a request whose generation is no longer on record counts as current, so losing the bookkeeping degrades to the previous behaviour rather than suppressing a genuine interruption. Fixes livekit#2119
🦋 Changeset detectedLatest commit: b078075 The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
`overlapCount` restarts at 0 every agent turn, and the request ledger was cleared at that same boundary, so a response outliving its turn found no record, took the fail-open path, and was credited to whatever overlap was open in the next turn — the same misattribution this fixes within a turn. Track `overlapGeneration` separately and never reset or clear it. `overlapCount` keeps its per-turn meaning for the audio-prefix shift. Add the release-notes changeset.
Author
|
Both review points addressed in b078075. The cross-turn one was real: clearing the ledger at the agent-speech boundary sent stale responses straight down the fail-open path, and |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The issue notes this wasn't reproduced at runtime, so this starts with a failing test: overlap A sends a request, A ends, B opens, and A's
bargein_detectedlands and is reported as an interruption of B. Ordinary response latency is enough to hit it.InterruptionAudioSlice.overlapGenerationfrom #2116 isn't onmain— #2116 and #2117 were both closed unmerged. So this adds anoverlapGenerationthat bumps per overlap and never resets, stamps it onto each request insendAudioData(), and rejects responses whose generation has since closed. It's deliberately separate from the existingoverlapCount, which restarts each agent turn for the audio-prefix shift — reusing that would let turn N's generation collide with turn N+1's.inference_doneneeded the same guard. It emits no event, but it writes intocache, and a stale write poisons the entryoverlap-speech-endedlater pops for its verdict.The check fails open: an unrecorded generation counts as current, since treating an evicted request as stale would suppress a real interruption — worse than the misattribution being fixed. Same reason the ledger sits outside
cache, which is cleared at every overlap boundary.Regression covers the late response, the cross-turn case, the fail-open path, and a control that still interrupts normally.
Fixes #2119