Discussions

Dear Heyen support. I NEED HELP :)

I have spent the last two weeks trying implement your live streaming in my new EGS app - were are developing a huge use case for your live interactive AI avatars in ESG management wwww.greenr.org.uk so I very much hope you can spend a little bit of time helping me :) I have rad all you API documentation, but It is very very hard to follow & seems inconsistent to me! Probably me to be fair!!

To help us get up & running could you please provide answers to the following questions, because i cannot find the answers in any of your content & I am NOT a eloper so cannot fill the gaps? THANK YOU!!! Kind Regards, James

1. Core WebSocket Setup

Question: What is the exact format of the WebSocket URL we will use to connect after a session has been created? Answer Needed:Provide the complete URL structure, including any required parameters.
Question: What specific headers (e.g., authorization tokens, API keys, session IDs) are mandatory for the WebSocket connection? Answer Needed: List all required headers and their expected formats/values.
Question: What are the key lifecycle events we need to handle when working with the WebSocket connection (e.g., connection established, message received, connection closed, error)? Answer Needed: Describe the events and their associated data.
Question: Can you provide a sample code snippet demonstrating how to create a session and establish a WebSocket connection? Answer Needed: A code example showing the session creation and WebSocket connection process.

2. Message Protocol Details

Question: Can you provide complete message format examples for each type of interaction?Answer Needed: Provide the exact structure for:
InitMessage
TextInput
AvatarResponse
Question: Is a heartbeat mechanism required to maintain the WebSocket connection? If so, what is the required interval and message format? Answer Needed: Specify the heartbeat details if applicable.
Question: How can we detect and handle disconnections from the WebSocket? Answer Needed: Describe the process for detecting and handling disconnections.

3. Audio Requirements

Question: What are the exact audio specifications required for sending audio data to the avatar?Answer Needed: Specify:
Required sample rate
Bit depth
Channel configuration (mono/stereo)
Supported codecs
Maximum chunk size for streaming
Question: Which browser APIs are needed for audio capture? Answer Needed: List the necessary browser APIs.
Question: Can you provide an example of audio processing code? Answer Needed: Provide a code example showing how to process audio data.

4. Real-time Flow Control

Question: Can you provide a complete sequence diagram illustrating a typical interaction between the client and HeyGen's API? Answer Needed: A sequence diagram showing the message flow.
Question: How should we handle overlapping messages? Answer Needed: Describe the recommended approach.
Question: How can we detect when the avatar is speaking versus listening? Answer Needed:Explain how to determine the avatar's state.
Question: What state management requirements are there? Answer Needed: Describe any state management requirements.

5. Response Handling

Question: What is the exact structure of the response messages we will receive from the avatar? Answer Needed: Provide the exact format for AvatarResponse, including the possible values for type and the structure of payload.
Question: Does the API support streaming responses? Answer Needed: Confirm if streaming is supported.
Question: What are the error response formats?Answer Needed: Describe the error response formats.
Question: What are the expected latency ranges for planning UI feedback? Answer Needed:Provide expected latency ranges.

6. Error & Connection Management

Question: What are the possible error codes and their meanings? Answer Needed: List all possible error codes and their descriptions.
Question: What are the recommended retry strategies? Answer Needed: Describe the recommended retry strategies.
Question: What are the session timeout details?Answer Needed: Provide session timeout information.
Question: What is the browser compatibility matrix? Answer Needed: Provide browser compatibility information.
Question: What is the connection recovery process? Answer Needed: Describe the recommended approach for handling disconnections and reconnecting.

7. Avatar Configuration

Question: What is the complete list of configurable parameters for the avatar? Answer Needed: List all configurable parameters.
Question: What voice customization options are available? Answer Needed: Describe the voice customization options.
Question: What performance settings are available? Answer Needed: Provide the options for AvatarConfig.
Question: What are the browser resource requirements? Answer Needed: Provide information on browser resource requirements.