Skip to content

Conversation

@SheikhSarvar
Copy link

@SheikhSarvar SheikhSarvar commented Jan 12, 2026

feat: Blocking Acknowledgment System for Voice Agents

Description

This PR introduces a "blocking acknowledgment" system to the LiveKit Agents framework. This feature allows developers to play an immediate response (e.g., "Looking that up for you...") after a user finishes speaking, ensuring the conversation feels responsive even when the LLM takes time to process.

The system intelligently manages the speech queue: if the LLM responds quickly, the acknowledgment is automatically skipped to keep the conversation fluid.

✨ Key Features

  • wait_for_acknowledgment(timeout): Blocks the speech queue with a high-priority handle immediately after user speech.
  • set_acknowledgment(text): Dynamically sets the message to be played.
  • Automatic Skip: The system automatically skips the acknowledgment if the LLM response arrives before the acknowledgment is set.
  • Observability: Developers can monitor acknowledgment status (Played/Skipped/Timed Out) via the speech_created event on the AgentSession.
  • Robust Cleanup: Background tasks are strictly managed to prevent resource leaks (verified by automated tests).

💡 Use Cases

  • RAG / Database Lookups:
    When an agent needs to query a vector database or external API (which might take 1–3 seconds), playing a filler phrase like "Searching our database..." prevents dead air and confirms the agent heard the request.

  • Complex Reasoning:
    For agents using reasoning models (like o1 or DeepSeek-R1) or multi-step tool calls that have high latency, an acknowledgment assures the user that work is in progress.

  • Network Latency Compensation:
    In environments with variable network latency, this provides a consistent "I heard you" feedback loop regardless of backend performance.

  • Transaction Confirmations:
    Before processing a sensitive action (like "Transferring funds"), the agent can say "One moment while I access your account..." to bridge the gap between authorization and execution.

✅ Verification

  • Automated Tests:
    Added 4 comprehensive tests in tests/test_acknowledgment.py covering:

    • Normal flow
    • Fast LLM skip
    • Timeouts
    • Interruptions

    All tests pass with 0 resource leaks.

  • Manual Verification:
    Verified using the new examples/voice_agents/acknowledgment.py script.

feat(voice): implement blocking acknowledgment system

- Adds wait_for_acknowledgment, set_acknowledgment, and
skip_acknowledgment APIs
- Implements acknowledgment logic in AgentActivity with high-priority
handles
- Configures automatic acknowledgment skipping on early LLM responses
- Includes standard verification tests and a new example agent
- Fixed potential task leaks in background speech tasks
@CLAassistant
Copy link

CLAassistant commented Jan 12, 2026

CLA assistant check
All committers have signed the CLA.

@chenghao-mou
Copy link
Member

We already have a similar example: fast-preresponse. Not sure if this is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants