allow pushing frames to VAD when agent speech is uninterruptible

[chenghao-mou]

This should close #4413

What happened:

• VAD received audio frames, changing user stage to speaking;
• Uninterruptible speech created, discarding audio frames for both STT and VAD. User state is stuck in speaking.

This PR should allow VAD to operate separately. Tested with

    @function_tool
    async def get_weather(self, location: str) -> str:
        """
        Called when the user asks about the weather.

        Args:
            location: The location to get the weather for
        """
        await asyncio.sleep(5) # <- interrupt here!
        self.session.say("And tomorrow is going to be sunny too.", allow_interruptions=False)
        return f"The weather in {location} is sunny today."

[longcw]

lgtm! do you think if we should ignore the user silence event if the speech is uninterruptible?

[chenghao-mou]

lgtm! do you think if we should ignore the user silence event if the speech is uninterruptible?

Do you mean skip waiting for user silence when the uninterruptible agent speech hasn't started? I think we should keep it because to the user, the agent is not yet speaking and they are not done talking, especially now with VAD being enabled at all time.