Discussions

Hey team, there is still an ongoing production issue with the interactive avatar.

I raised this last week and thought it had been fixed. But issues linger.

Below, I have captured 15 interactive avatar sessions.

• 6 Failed - Avatar DOES NOT HEAR/RESPOND To User
• 9 OK - Avatar Does Respond

For all these sessions, I'm using GLADIA as STT. However, I've been seeing issues with Deepgram too. I had switched over, hoping it would resolve, but no joy.

In my logs I can see:

DEBUG] USER_START detected: CustomEvent {isTrusted: false, detail: {…}, type: 'user_start', target: EventTarget, currentTarget: EventTarget, …}
useAvatarEvents.ts:249
[DEBUG] USER_STOP detected: CustomEvent {isTrusted: false, detail: {…}, type: 'user_stop', target: EventTarget, currentTarget: EventTarget, …}
useAvatarEvents.ts:241

After these events a USER_TALKING_MESSAGE is never sent.

I can see at least two other threads below where users are experiencing similar issues.

I've been pulling my hair out thinking it's an issue with my code, but after extensive testing, I can only assume it's on Hey Gen's side.

TEST SESSIONS

1 -OK Session - Avatar Hears User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=92eb7fae-2f68-11f0-9704-8e4662c3c8ae&session_token=eyJ0b2tlbiI6ICJiOWExNGI3ZDI2ZWM0YzQ5OGIzMGJjOGQ1M2UxMzdkNCIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3ODY2Mn0=&silence_response=false

2 - FAILED Session - Avatar DOES NOT HEAR/RESPOND To User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=b3e38219-2f68-11f0-9704-8e4662c3c8ae&session_token=eyJ0b2tlbiI6ICI5OTk1YzcxNzIzNzc0Yjc4ODBjOWY0YWUzNzFjMzQzOSIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3ODcxOH0=&silence_response=false

3 - FAILED Session - Avatar DOES NOT HEAR/RESPOND To User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=04a38aba-2f69-11f0-996b-22704fc0d79b&session_token=eyJ0b2tlbiI6ICI3MDU4NTc1ZWMxNGQ0ZDNkOWY3ZDU3NzZlN2Q3M2RiNSIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3ODg1M30=&silence_response=false

4 - OK Session - Avatar Hears User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=23c42fed-2f69-11f0-b322-ca721bab18c4&session_token=eyJ0b2tlbiI6ICJjYzNhYWZjM2RhNTI0OTc1YmI2MTI5ZTRiN2FjZTUzNSIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3ODkwNX0=&silence_response=false

5 - OK Session - Avatar Hears User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=3ca4f33a-2f69-11f0-b322-ca721bab18c4&session_token=eyJ0b2tlbiI6ICJjYjE0MWFiMWQ4YzI0ODFiOTkxMTlmYWExYTliMGZhOSIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3ODk0N30=&silence_response=false

6 - OK Session - Avatar Hears User`

wss://api.heygen.com/v1/ws/streaming.chat?session_id=4f787bfb-2f69-11f0-b322-ca721bab18c4&session_token=eyJ0b2tlbiI6ICI4Mzc5OWZjNjhjN2E0ZDU1OWQwZGFjY2NlMjYwODdiMCIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3ODk3OX0=&silence_response=false

7 - FAILED Session - Avatar DOES NOT HEAR/RESPOND To User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=6b37ab57-2f69-11f0-b322-ca721bab18c4&session_token=eyJ0b2tlbiI6ICJkZjQ5OGNhYzgxYjE0OTRmYjIyMjMxZjYyMTA5ODk0NiIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTAyNH0=&silence_response=false

8 - OK Session - Avatar Hears User`

wss://api.heygen.com/v1/ws/streaming.chat?session_id=984dece7-2f69-11f0-9704-8e4662c3c8ae&session_token=eyJ0b2tlbiI6ICI0ZDMxZjExZDc3N2Q0Mzk3OGE3NzkwYjljZGZiMmQwZCIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTEwMX0=&silence_response=false

9 - FAILED Session - Avatar DOES NOT HEAR/RESPOND To User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=ab7d0127-2f69-11f0-b322-ca721bab18c4&session_token=eyJ0b2tlbiI6ICI3NDdmNDYxYWRjMTI0Zjc2ODc0ZGQ5N2I2MmFjM2NkOSIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTEzM30=&silence_response=false

10 - FAILED Session - Avatar DOES NOT HEAR/RESPOND To User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=c38872f5-2f69-11f0-8446-6a9d03d69793&session_token=eyJ0b2tlbiI6ICJhM2E5MTRlMWNiOGI0MmY1YTUwMWM3OTcxYWU2NWM3NyIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTE3M30=&silence_response=false

11 - OK Session - Avatar Hears User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=db94846f-2f69-11f0-8041-aafedb6f6c4d&session_token=eyJ0b2tlbiI6ICIwY2FiOTM0ZDU2YTY0M2I4YTk5ZDBiZTU4Yzc4YjExYiIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTIxNH0=&silence_response=false

12 - OK Session - Avatar Hears User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=4b8cfef7-2f6a-11f0-b322-ca721bab18c4&session_token=eyJ0b2tlbiI6ICI2ZGNkZDY5YTliZmE0YzMyYWNmMWZhZDJkNjNmYjdmNyIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTQwMX0=&silence_response=false

13 - OK Session - Avatar Hears User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=70ea3956-2f6a-11f0-9704-8e4662c3c8ae&session_token=eyJ0b2tlbiI6ICJhZDkyY2ZmYTFlNzg0ZGMwOGY5NmMxODcyOGUxNmM3YiIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTQ2NH0=&silence_response=false

14 - OK Session - Avatar Hears User

wss://api.heygen.com/v1/ws/streaming.chat?session_id=8cf49da1-2f6a-11f0-8041-aafedb6f6c4d&session_token=eyJ0b2tlbiI6ICI4NzkyZDMwZWNmYTE0N2RmOGEyNGUzZTFkNWU2NmMyMSIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTUxMX0=&silence_response=false

15 - FAILED Session - Avatar DOES NOT HEAR/RESPOND To User Session - Avatar Hears User**

wss://api.heygen.com/v1/ws/streaming.chat?session_id=ac7acb95-2f6a-11f0-8446-6a9d03d69793&session_token=eyJ0b2tlbiI6ICI1MjNjNzhjYjY2YTg0OGU1YTRlMTI3YzZjMTA3MDViZSIsICJ0b2tlbl90eXBlIjogInNhX2Zyb21fcmVndWxhciIsICJjcmVhdGVkX2F0IjogMTc0NzA3OTU2NH0=&silence_response=false

OTHER IMPACT USERS

https://docs.heygen.com/discuss/6821da261567b000101d2894

https://docs.heygen.com/discuss/6820c7bf5ab9cc0070e8c0e1 (see Erin's comments)