Discussions

Hello everyone,
I've been integrating Heygen avatars with my custom backend solution and ran into an interesting problem that I wanted to share in case others are experiencing it.

My Setup:

Using Heygen only for the avatar rendering
Custom backend server handling STT, LLM responses, and TTS (via ElevenLabs)
WebSocket communication between my server and Heygen's LiveKit WebSocket

The Issue:
When a user interrupts the avatar mid-speech via my WebSocket connection, the entire avatar video freezes. After this freeze, the avatar becomes unresponsive even though my server continues to process audio and send it to Heygen.

Reproduction Steps:

Create a session and initialize with Streaming SDK (Frontend)
Connect my Python WebSocket server to the LiveKit URL provided by Heygen
Process speech and generate responses in my server
Generate audio via ElevenLabs API
Send audio to LiveKit WebSocket (append + commit)
Avatar speaks normally until interrupted
When sending an interruption command via WebSocket while the avatar is speaking, the video freezes permanently

Working Solution:
I've switched to using Heygen's REST API endpoints instead of WebSocket for sending tasks and interrupting the avatar (with ElevenLabs configured in Heygen). This approach works perfectly - interruptions happen smoothly with no freezing.
Has anyone else encountered this issue with the WebSocket approach? I'm curious if there's a way to make interruptions work properly with WebSockets, or if the REST API is the recommended approach for handling interruptions.
Any insights from the community or Heygen team would be appreciated!