Discussions

I am developing an interactive avatar project using the Streaming Avatar SDK with Vite 6.0.6 and TypeScript 5.6.3. I have implemented an advanced system with the following functionalities:

• Avatar interruptions when the user speaks (interruptCurrentSpeech())
• Real-time updating chat history
• Voice detection and processing with WebAudio API
• Multilingual support (7 languages)
• Inappropriate content filtering
• Inactivity management and automatic reconnections

Problem: I have identified a significant latency (6-9 seconds) between the moment a query is sent and when the avatar begins to speak. Interestingly, the response text appears almost instantly in the chat history through the AVATAR_TALKING_MESSAGE event, but the avatar takes much longer to start facial animation and audio.

Technical details:

• The system is deployed on robust servers and is already in production
• I use TaskType.TALK for user queries
• I have already tried reducing the avatar quality to AvatarQuality.Low without significant improvements
• The askAvatar() function uses taskMode: TaskMode.ASYNC
• The AVATAR_TALKING_MESSAGE event displays the text quickly
• I am using the latest version of the SDK (@heygen/streaming-avatar)

Query: Is there any recommended configuration or approach to reduce this response time? Are there any additional parameters I can adjust to make the avatar start speaking more quickly after receiving a query?

I would appreciate any suggestions to optimize the user experience, as the current delay significantly affects the fluidity of the interaction.