OpenAI Realtime API
The Realtime API enables low-latency, bidirectional communication with models that natively support speech-to-speech interactions as well as multimodal inputs (audio, images, and text) and outputs (audio and text). It supports WebRTC, WebSocket, and SIP connection methods for real-time voice agents and conversational interfaces. The Realtime API is represented as a dedicated tag group in the upstream OpenAI OpenAPI specification with operations covering client/server events, translation client secrets, and voice call lifecycle (accept, hangup, refer, reject).
OpenAI Realtime API is one of 51 APIs that OpenAI publishes on the APIs.io network, described by a machine-readable OpenAPI specification.
This API exposes 2 JSON Schema definitions.
Tagged areas include Real-Time. The published artifact set on APIs.io includes an OpenAPI specification, API documentation, a JSON-LD context, and 2 JSON Schemas.
This event API exposes
37 operations
across 37 channels,
and defines 99 schemas.
It is described by AsyncAPI 2.6.0, at version 2024-10-01.
The API connects through a single server, api.openai.com/v1/realtime (wss).
The identity and technical contract details declared by the specification.
SessionUpdate
message
Update session configuration.
InputAudioBufferAppend
message
Append audio bytes to the input buffer.
InputAudioBufferCommit
message
Commit the input audio buffer.
InputAudioBufferClear
message
Clear the input audio buffer.
ConversationItemCreate
message
Insert a conversation item.
ConversationItemTruncate
message
Truncate an assistant item's audio.
ConversationItemDelete
message
Delete a conversation item.
ResponseCreate
message
Trigger a model response.
ResponseCancel
message
Cancel an in-progress response.
Error
message
Server error.
SessionCreated
message
Session has been created.
SessionUpdated
message
Session configuration updated.
ConversationCreated
message
Conversation created.
ConversationItemCreated
message
Conversation item created.
InputAudioTranscriptionCompleted
message
Input audio transcription completed.
InputAudioTranscriptionFailed
message
Input audio transcription failed.
ConversationItemTruncated
message
Conversation item truncated.
ConversationItemDeleted
message
Conversation item deleted.
InputAudioBufferCommitted
message
Input audio buffer committed.
InputAudioBufferCleared
message
Input audio buffer cleared.
InputAudioBufferSpeechStarted
message
VAD speech started.
InputAudioBufferSpeechStopped
message
VAD speech stopped.
ResponseCreated
message
Response generation started.
ResponseDone
message
Response generation finished.
ResponseOutputItemAdded
message
New output item added to response.
ResponseOutputItemDone
message
Output item on response complete.
ResponseContentPartAdded
message
Content part added to output item.
ResponseContentPartDone
message
Content part on output item complete.
ResponseTextDelta
message
Text delta for assistant message.
ResponseTextDone
message
Text content part complete.
ResponseAudioTranscriptDelta
message
Transcript delta for audio content part.
ResponseAudioTranscriptDone
message
Transcript for audio content part complete.
ResponseAudioDelta
message
Base64 audio delta for audio content part.
ResponseAudioDone
message
Audio content part complete.
ResponseFunctionCallArgumentsDelta
message
Function-call arguments delta.
ResponseFunctionCallArgumentsDone
message
Function-call arguments complete.
RateLimitsUpdated
message
Current rate limit state.
AudioFormat
string
Supported realtime audio codecs.
Voice
string
Realtime model voice.
TurnDetection
object
Server-side voice activity detection config. Set to null to disable.
4 properties
1 required
InputAudioTranscription
object
Input audio transcription config. Set to null to disable. When enabled, the server emits conversation.item.inputaudiotranscription.completed for each user audi…
1 property
1 required
ToolDefinition
object
4 properties
2 required
MaxResponseOutputTokens
object
Max output tokens per response, or inf for unlimited.
Session
object
Server-side session configuration.
14 properties
SessionPatch
object
Subset of session fields that may be supplied on session.update. Only included properties are modified.
11 properties
Conversation
object
2 properties
InputTextContent
object
2 properties
2 required
InputAudioContent
object
3 properties
1 required
TextContent
object
2 properties
2 required
AudioContent
object
3 properties
1 required
MessageItem
object
7 properties
3 required
FunctionCallItem
object
8 properties
4 required
FunctionCallOutputItem
object
6 properties
3 required
Response
object
6 properties
ResponseCreateParams
object
Optional per-response overrides for response.create. Any fields omitted fall back to the session defaults.
8 properties
ErrorObject
object
5 properties
2 required
RateLimit
object
4 properties
4 required
SessionUpdateEvent
object
Client event used to update the session's default configuration. The server responds with session.updated.
3 properties
2 required
InputAudioBufferAppendEvent
object
Append base64-encoded audio bytes to the input audio buffer.
3 properties
2 required
InputAudioBufferCommitEvent
object
Commit the audio currently in the input audio buffer to the conversation as a user message item.
2 properties
1 required
InputAudioBufferClearEvent
object
Clear the input audio buffer without creating a user message.
2 properties
1 required
ConversationItemCreateEvent
object
Insert a conversation item into the conversation history. Items may be messages (system/user/assistant), function calls, or functioncalloutput records.
4 properties
2 required
ConversationItemTruncateEvent
object
Truncate an in-progress assistant audio item at audioendms.
5 properties
4 required
ConversationItemDeleteEvent
object
Delete a conversation item by id.
3 properties
2 required
ResponseCreateEvent
object
Instruct the server to generate a model response.
3 properties
1 required
ResponseCancelEvent
object
Cancel an in-progress model response.
2 properties
1 required
ErrorEvent
object
3 properties
2 required
SessionCreatedEvent
object
3 properties
2 required
SessionUpdatedEvent
object
3 properties
2 required
ConversationCreatedEvent
object
3 properties
2 required
ConversationItemCreatedEvent
object
4 properties
2 required
InputAudioTranscriptionCompletedEvent
object
5 properties
4 required
InputAudioTranscriptionFailedEvent
object
5 properties
4 required
ConversationItemTruncatedEvent
object
5 properties
4 required
ConversationItemDeletedEvent
object
3 properties
2 required
InputAudioBufferCommittedEvent
object
4 properties
2 required
InputAudioBufferClearedEvent
object
2 properties
1 required
InputAudioBufferSpeechStartedEvent
object
4 properties
3 required
InputAudioBufferSpeechStoppedEvent
object
4 properties
3 required
ResponseCreatedEvent
object
3 properties
2 required
ResponseDoneEvent
object
3 properties
2 required
ResponseOutputItemAddedEvent
object
5 properties
4 required
ResponseOutputItemDoneEvent
object
5 properties
4 required
ResponseContentPartAddedEvent
object
7 properties
6 required
ResponseContentPartDoneEvent
object
7 properties
6 required
ResponseTextDeltaEvent
object
7 properties
6 required
ResponseTextDoneEvent
object
7 properties
6 required
ResponseAudioTranscriptDeltaEvent
object
7 properties
6 required
ResponseAudioTranscriptDoneEvent
object
7 properties
6 required
ResponseAudioDeltaEvent
object
7 properties
6 required
ResponseAudioDoneEvent
object
6 properties
5 required
ResponseFunctionCallArgumentsDeltaEvent
object
7 properties
6 required
ResponseFunctionCallArgumentsDoneEvent
object
7 properties
6 required
RateLimitsUpdatedEvent
object
3 properties
2 required
The full machine-readable AsyncAPI contract behind this narrative.
Other APIs OpenAI publishes across the network.
