Streaming Avatar SDK API Reference
The @heygen/streaming-avatar
package provides a TypeScript SDK for interacting with HeyGen's streaming avatar service. For detailed information about the available methods, interfaces, enums, and event handling, refer to the API reference provided below.
Classes
StreamingAvatar
StreamingAvatar
This class is the core of the SDK, responsible for managing avatar streaming sessions, including session creation, controlling avatars, and handling real-time communication.
const avatar = new StreamingAvatar({ token: "access-token" });
constructor({ token, basePath }: StreamingAvatarApiConfig)
Interfaces
Configuration object for initializing StreamingAvatar
.
Property | Type | Description |
---|---|---|
token | string | Authentication token. |
basePath | string | Base API URL (optional, defaults to https://api.heygen.com ). |
const config: StreamingAvatarApiConfig = {
token: "access-token",
};
Request payload to initiate a new avatar streaming session.
Property | Type | Description |
---|---|---|
avatarName | string | Interactive Avatar ID for the session. (default, default ) |
quality | AvatarQuality | (Optional) The desired quality level of the avatar stream. |
voice | VoiceSetting | (Optional) Voice settings for the avatar. |
knowledgeId | string | (Optional) Knowledge base ID for the avatar's knowledge. Retrive from labs.heygen.com. |
knowledgeBase | string | (Optional) Your customized prompt content. |
const startRequest: StartAvatarRequest = {
quality: AvatarQuality.High,
avatarName: avatarId,
knowledgeId: knowledgeId,
// knowledgeBase: knowledgeBase,
voice: {
voiceId: voiceId,
rate: 1.5, // 0.5 ~ 1.5
emotion: VoiceEmotion.EXCITED,
},
language: language,
};
The response received when an avatar session is successfully started.
Property | Type | Description |
---|---|---|
session_id | string | The unique ID of the streaming session. |
access_token | string | Token to authenticate further interactions. |
url | string | WebSocket URL for establishing the streaming session. |
is_paid | boolean | Indicates whether the session is under a paid plan. |
session_duration_limit | number | Maximum allowed duration for the session in seconds. |
{
"session_id": "eba59f0d-71f5-11ef-b8af-d2e5560124bc",
"sdp": null,
"access_token": "eyJhbGc...",
"url": "wss://heygen-feapbkvq.livekit.cloud",
"ice_servers": null,
"ice_servers2": null,
"is_paid": true,
"session_duration_limit": 600
}
Request payload for sending a speaking command to the avatar.
Property | Type | Description |
---|---|---|
text | string | The text the avatar should speak. |
task_type | string | Task type. |
const speakRequest: SpeakRequest = {
text: "Hello, there!",
task_type: TaskType.REPEAT
};
Methods
createStartAvatar
createStartAvatar
Starts a new avatar session using the provided configuration and returns session information.
createStartAvatar(requestData: StartAvatarRequest): Promise<any>
const startRequest = { /* configuration settings */ };
const response = await createStartAvatar(startRequest);
console.log(response.session_id); // Store session ID for future operations
startVoiceChat
startVoiceChat
Starts a voice chat within the active avatar session. You can optionally enable or disable silence prompts during the chat by setting the useSilencePrompt
flag
startVoiceChat(requestData: { useSilencePrompt?: boolean } = {}): Promise<any>
await startVoiceChat({ useSilencePrompt: false });
closeVoiceChat
closeVoiceChat
Ends the active voice chat session within the avatar interaction.
closeVoiceChat(): Promise<any>
await closeVoiceChat();
newSession
newSession
Creates and starts a new session using the provided StartAvatarRequest
data, returning detailed session information such as the session ID and other metadata.
newSession(requestData: StartAvatarRequest): Promise<StartAvatarResponse>
const sessionData = await newSession(startRequest);
console.log("New session created:", sessionData);
startSession
startSession
Starts an existing avatar session by using the previously stored session ID or configuration from a StartAvatarRequest
.
startSession(): Promise<any>
const response = await startSession(sessionRequest);
console.log("Session started:", response);
speak
speak
Sends a command to the avatar to speak the provided text. Additional parameters like task_type
allow for more advanced control, like repeating or talking.
speak(requestData: SpeakRequest): Promise<any>
await speak({ text: "Hello, world!", task_type: TaskType.REPEAT });
startListening
startListening
Activates the avatar’s listening mode, allowing it to process incoming audio or messages from the user.
startListening(): Promise<any>
await startListening({});
stopListening
stopListening
Stops the avatar from listening to incoming audio or messages.
stopListening(): Promise<any>
await stopListening({});
interrupt
interrupt
Interrupts the current speaking task.
interrupt(): Promise<any>
await interrupt({});
stopAvatar
stopAvatar
Stops the avatar session.
stopAvatar(): Promise<any>
await stopAvatar({});
on
on
Registers an event listener for specific streaming events.
on(eventType: string, listener: EventHandler): this
on('messageReceived', (message) => {
console.log("Received message:", message);
});
off
Unregisters an event listener.
off(eventType: string, listener: EventHandler): this
off('messageReceived', listenerFunction);
Types and Enums
Defines the quality settings for the avatar.
- High:
'high'
- 2000kbps and 720p. - Medium:
'medium'
- 1000kbps and 480p. - Low:
'low'
- 500kbps and 360p.
EXCITED
: Excited voice emotion.SERIOUS
: Serious voice emotion.FRIENDLY
: Friendly voice emotion.SOOTHING
: Soothing voice emotion.BROADCASTER
: Broadcaster voice emotion.
TALK
: Avatar will talk.REPEAT
: Avatar will repeat.
Enumerates the event types for streaming. See Event Handling for details.
AVATAR_START_TALKING
: Emitted when the avatar starts speaking.AVATAR_STOP_TALKING
: Emitted when the avatar stops speaking.AVATAR_TALKING_MESSAGE
: Triggered when the avatar sends a speaking message.AVATAR_END_MESSAGE
: Triggered when the avatar finishes sending messages.USER_TALKING_MESSAGE
: Emitted when the user sends a speaking message.USER_END_MESSAGE
: Triggered when the user finishes sending messages.USER_START
: Indicates when the user starts interacting.USER_STOP
: Indicates when the user stops interacting.USER_SILENCE
: Indicates when the user is silent.STREAM_READY
: Indicates that the stream is ready for display.STREAM_DISCONNECTED
: Triggered when the stream disconnects.
Event Handling
The SDK emits a variety of events during a streaming session, which can be captured to update the UI or trigger additional logic. Use the on
and off
methods to manage event listeners.
AVATAR_START_TALKING
AVATAR_START_TALKING
This event is emitted when the avatar begins speaking.
avatar.on(StreamingEvents.AVATAR_START_TALKING, (event) => {
console.log('Avatar has started talking:', event);
// You can update the UI to reflect that the avatar is talking
});
AVATAR_STOP_TALKING
AVATAR_STOP_TALKING
avatar.on(StreamingEvents.AVATAR_STOP_TALKING, (event) => {
console.log('Avatar has stopped talking:', event);
// You can reset the UI to indicate the avatar has stopped speaking
});
AVATAR_TALKING_MESSAGE
AVATAR_TALKING_MESSAGE
Fired when the avatar sends a message while talking. This event can be useful for real-time updates on what the avatar is currently saying.
avatar.on(StreamingEvents.AVATAR_TALKING_MESSAGE, (message) => {
console.log('Avatar talking message:', message);
// You can display the message in the UI
});
AVATAR_END_MESSAGE
AVATAR_END_MESSAGE
Fired when the avatar sends the final message before ending its speech.
avatar.on(StreamingEvents.AVATAR_END_MESSAGE, (message) => {
console.log('Avatar end message:', message);
// Handle the end of the avatar's message, e.g., indicate the end of the conversation
});
USER_TALKING_MESSAGE
USER_TALKING_MESSAGE
Fired when the user sends a message to the avatar.
avatar.on(StreamingEvents.USER_TALKING_MESSAGE, (message) => {
console.log('User talking message:', message);
// Handle the user's message input to the avatar
});
USER_END_MESSAGE
USER_END_MESSAGE
Fired when the user has finished sending their message to the avatar.
avatar.on(StreamingEvents.USER_END_MESSAGE, (message) => {
console.log('User end message:', message);
// Handle the end of the user's message, e.g., process the user's response
});
USER_START
USER_START
Fired when the user has finished sending their message to the avatar.
avatar.on(StreamingEvents.USER_START, (event) => {
console.log('User has started interaction:', event);
// Handle the start of the user's interaction, such as activating a listening indicator
});
USER_STOP
USER_STOP
Triggered when the user stops interacting or speaking with the avatar.
avatar.on(StreamingEvents.USER_STOP, (event) => {
console.log('User has stopped interaction:', event);
// Handle the end of the user's interaction, such as deactivating a listening indicator
});
USER_SILENCE
USER_SILENCE
Triggered when the user is silent for a certain period.
avatar.on(StreamingEvents.USER_SILENCE, () => {
console.log('User is silent');
});
STREAM_READY
STREAM_READY
Fired when the avatar's streaming session is ready.
avatar.on(StreamingEvents.STREAM_READY, (event) => {
console.log('Stream is ready:', event.detail);
// Use event.detail to attach the media stream to a video element, for example
});
STREAM_DISCONNECTED
STREAM_DISCONNECTED
Triggered when the streaming connection is lost or intentionally disconnected.
avatar.on(StreamingEvents.STREAM_DISCONNECTED, () => {
console.log('Stream has been disconnected');
// Handle the disconnection, e.g., clean up the UI or try to reconnect the session
});
Error Handling
Always handle errors gracefully when dealing with asynchronous requests to avoid disruptions in the user experience.
try {
await avatar.speak({ text: 'Hello!' });
} catch (error) {
console.error('Error sending speak command:', error);
}
Conclusion
This reference offers a comprehensive overview of HeyGen's streaming avatar SDK, complete with examples and descriptions of methods, events, and configuration options.
Updated 13 days ago