Skip to content

VAD doesn't trigger model response after barge-in during interactive audio session #1987

@random-droid

Description

@random-droid

Environment details

  • Programming language: Python
  • OS: Linux (Cloud Run) / macOS
  • Language runtime version: Python 3.11+
  • Package version: google-genai (latest)

Steps to reproduce

config = types.LiveConnectConfig(
    response_modalities=["AUDIO"],
    realtime_input_config=types.RealtimeInputConfig(
        automatic_activity_detection=types.AutomaticActivityDetection(
            disabled=False,
            start_of_speech_sensitivity='START_SENSITIVITY_LOW',
            end_of_speech_sensitivity='END_SENSITIVITY_HIGH',
            silence_duration_ms=500
        )
    )
)

Input 16kHz PCM, output 24kHz PCM, streamed continuously from browser mic

  1. Connect to gemini-live-2.5-flash-native-audio with VAD config (see above)
  2. Send text prompt: "Please read the entire script now" with end_of_turn=True
  3. Model starts speaking (audio briefing)
  4. User clicks "Ask Question" button → starts streaming mic audio to Gemini
  5. Model detects barge-in (interrupted=True) and stops speaking ✓
  6. User asks a question, then stops talking
  7. Expected: VAD detects end of user speech → model responds to question
  8. Actual: Model never responds. Audio chunks continue being sent, input_transcription=None, no new model_turn received.

Expected: VAD detects end of user speech → model responds
Actual: VAD detects speech onset (barge-in works, interrupted=True), but never detects offset. Model never responds. input_transcription=None.

Notes
Possibly related to continuous streaming vs. paused stream
Docs mention audioStreamEnd for paused streams - unclear interaction with VAD

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions