> ## Documentation Index
> Fetch the complete documentation index at: https://docs.poly.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Dialog-RSN-1 API
> Connect to the Dialog-RSN-1 real-time voice model over WebSocket: endpoints, authentication, the session contract, events, PolyAI extensions and error codes.
Dialog-RSN-1 is a real-time voice model. You stream the caller's audio over one WebSocket and the model decides when the caller has finished speaking. The API speaks the reply and streams its transcript, or sends the reply as text for your own text-to-speech.
The wire format is the OpenAI Realtime API (GA schema). A stock OpenAI Realtime client works by changing the base URL and the key. The model name on the wire is `dialog-rsn-1`.
Concurrency, session duration, context window, audio formats and the connected-minute rate.
The overview, quickstart, prompt guide, cookbook and integration guides.
Field-level documentation for every event. This page lists what we accept and where we differ.
## Connect
Connect to `wss://api.studio.poly.ai/v1/realtime` with an API key from [studio.poly.ai](https://studio.poly.ai). Sign up or log in there and create the key in the **API Keys** tab. The [quickstart](/dialog-rsn-1/quickstart#getting-an-api-key) has the steps.
**Enterprise accounts on `studio.us.poly.ai`** connect to `wss://api.us.poly.ai/v1/realtime` instead, with a key from that Studio. A key only works on the host that matches the Studio that issued it, and a mismatch returns `401`. Dialog-RSN-1 runs in the US only.
Check the service is up with the health route, which takes the same key:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -s -H "X-API-KEY: $POLY_API_KEY" https://api.studio.poly.ai/v1/realtime/health
```
Open a connection and read the first event. The OpenAI SDK appends `/realtime` to the base URL and sends your key as a bearer token.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import asyncio
import os
from openai import AsyncOpenAI
client = AsyncOpenAI(
api_key=os.environ["POLY_API_KEY"],
websocket_base_url="wss://api.studio.poly.ai/v1",
)
async def main():
async with client.realtime.connect(model="dialog-rsn-1") as conn:
async for event in conn:
print(event.type) # session.created
break
asyncio.run(main())
```
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import WebSocket from "ws";
const ws = new WebSocket("wss://api.studio.poly.ai/v1/realtime", {
headers: { "X-API-KEY": process.env.POLY_API_KEY },
});
ws.on("message", (data) => {
const event = JSON.parse(data.toString());
console.log(event.type); // session.created
});
ws.on("unexpected-response", (_req, res) => {
let body = "";
res.on("data", (chunk) => (body += chunk));
res.on("end", () => console.error(`refused: ${res.statusCode} ${body}`));
});
```
The optional query parameter `session_id` (`[A-Za-z0-9_.-]`, up to 128 characters) names the session for your own logs. It comes back as `session.poly_session_id`. The `model` query parameter that OpenAI clients send is accepted and ignored.
The server pings every 20 seconds, which keeps a quiet connection open through the gateway. A connection whose client sends no event for 5 minutes is closed with code `4408`; a client that is streaming audio never meets this.
## Authenticate
Send your API key from [studio.poly.ai](https://studio.poly.ai) in either header. They are equivalent.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
X-API-KEY:
Authorization: Bearer
```
There is no query-string key and no WebSocket subprotocol credential. Browsers cannot set either header on a WebSocket, so put a relay in front of browser clients.
A refused handshake answers with an HTTP status and a JSON body. The close codes below are sent only by a server without the `websocket.http.response` extension, which can't answer the handshake with a status. A client library that can't read the HTTP response sees a failed handshake, not a close code.
| HTTP | Close code | Meaning | What to do |
| - | - | - | - |
| `401` | `4401` | Missing, unknown or inactive key | Check the key, or create a new one in the **API Keys** tab at [studio.poly.ai](https://studio.poly.ai) |
| `402` | `4402` | The account is not enrolled for billing, or has no balance left | Add funds at [studio.poly.ai](https://studio.poly.ai), then reconnect. A session already open is not cut off when the balance reaches zero. |
| `403` | `4403` | A personal access token | Use an API key from the **API Keys** tab at [studio.poly.ai](https://studio.poly.ai) |
| `429` | | Too many new connections this minute for this key, refused at the gateway | Honour `Retry-After` |
| `429` | `4429` | The account is at its concurrent-session cap (body `"code": "session_limit_reached"`) | There is no `Retry-After`. Close a session before opening another. See [Limits](/api-reference/dialog-rsn-1/limits). |
| `503` | `4503` | Authentication service unavailable | Retry with backoff. The key is fine. |
| `503` | `1013` | No capacity for a new session on this server (body `"code": "capacity_exceeded"`) | Retry after `Retry-After` (1 second). The retry very likely lands on another server. |
## Configure the session
Send `session.update` after `session.created`. Updates are merged into the session, not replaced. A rejected update applies nothing and sends no `session.updated`, so what comes back in `session.updated` is exactly what is in effect.
```json session.update theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"type": "session.update",
"session": {
"type": "realtime",
"instructions": "You work the front desk of a hotel. Keep replies to one sentence.",
"output_modalities": ["text"],
"audio": {
"input": {
"format": { "type": "audio/pcm", "rate": 24000 },
"turn_detection": { "type": "server_vad", "create_response": true }
}
},
"tools": []
}
}
```
### Honoured
| Field | Notes |
| - | - |
| `instructions` | The system prompt |
| `tools` | Function tools only. Mark at most one string parameter per tool with `"poly_spoken": true` and the model says that argument to the caller while the tool runs: as `response.output_text.delta` on a text session, for your text-to-speech to speak, or aloud on an audio session. |
| `tool_choice` | `"auto"` only |
| `max_output_tokens` | An integer or `"inf"`. The deployment's own cap is the ceiling. The model reasons before it answers and reasoning tokens come out of the same budget, so a small cap can leave a truncated or empty reply. |
| `audio.input.format` | See [Audio in](#audio-in) |
| `audio.input.turn_detection.create_response` | `true` (default): the model confirms the end of each turn and the server generates a reply. `false`: you send `response.create`, and turns are taken at face value with no model confirmation. The voice-activity detector still commits the turn at each pause it hears, and `input_audio_buffer.commit` commits at once. |
| `output_modalities` | `["audio"]`, the default for a new session, has the API speak the reply: audio arrives as `response.output_audio.delta` and the text as `response.output_audio_transcript.delta`. `["text"]` sends text replies you voice yourself, as `response.output_text.delta`. |
| `audio.output.voice` | The voice for spoken replies: a Cartesia voice id, as a string or `{"id": "..."}`. Unset, the deployment's default voice speaks. An id Cartesia doesn't know is accepted here, and each response then fails with `audio_synthesis_failed`. Kept on a text session, and used if the session switches to `["audio"]`. |
| `poly_knowledge`, `poly_response_language`, `poly_input_rate` | See [PolyAI extensions](#polyai-extensions) |
### Accepted and ignored
Fields that stock clients send by default and that would change nothing a caller notices are accepted. Most are left out of `session.updated`; `audio.input.transcription` and `parallel_tool_calls` are echoed back but change nothing:
* `model` (the session stays `dialog-rsn-1`), `tracing`, `truncation`, `include`
* `audio.input.noise_reduction`; `audio.input.transcription` (transcripts are always sent)
* Turn-detection timing: `threshold`, `silence_duration_ms`, `prefix_padding_ms`, `eagerness`. `type: "semantic_vad"` is reported back as `server_vad`. The server finds the end of the turn: a voice-activity detector proposes a boundary and the model confirms it.
* `turn_detection.interrupt_response: true`, which is already how the service behaves
* `audio.output.voice` set to an OpenAI voice name (`alloy`, `marin` and the rest): the deployment's default voice speaks instead. `audio.output.speed`.
* `parallel_tool_calls` (the model makes one call per turn)
### Refused
Fields that would change what you get if silently ignored are refused with an `error` naming the field in `param`. The whole update is dropped.
| Field | `error.code` | `param` |
| - | - | - |
| Any key not in the OpenAI schema and not prefixed `poly_`, at any depth | `unknown_parameter` | the path, e.g. `session.audio.input.transcrption` |
| `prompt`, `reasoning` | `unsupported_parameter` | `prompt` or `reasoning` |
| `tool_choice` other than `"auto"` | `unsupported_parameter` | `tool_choice` |
| `tools[i].type` other than `"function"` (no MCP) | `unsupported_parameter` | `tools[i].type` |
| `audio.input.turn_detection: null` (push-to-talk) | `unsupported_parameter` | `audio.input.turn_detection` |
| `turn_detection.interrupt_response: false` | `unsupported_parameter` | `audio.input.turn_detection.interrupt_response` |
| `turn_detection.idle_timeout_ms` | `unsupported_parameter` | `audio.input.turn_detection.idle_timeout_ms` |
| `audio.output.format.type` other than `"audio/pcm"` | `unsupported_parameter` | `audio.output.format.type` |
| `output_modalities` asking for `text` and `audio` together | `invalid_request_error` | |
| `max_output_tokens` below 1 | `invalid_request_error` | `max_output_tokens` |
| `poly_session_id`, `expires_at` (set by the server) | `invalid_request_error` | the field |
| `poly_knowledge` that isn't a list of `{ "name", "content", "actions"? }` with `actions` a string | `invalid_request_error` | `poly_knowledge`, or the entry, e.g. `poly_knowledge[2]` |
| `poly_response_language` that isn't a string, is blank, or is over 64 characters | `invalid_request_error` | `poly_response_language` |
| `poly_spoken` set to anything but `true`, on a non-string property, or on more than one property of a tool | `invalid_request_error` | the path under `tools[i].parameters.properties` |
## Audio in
Audio travels as base64 inside `input_audio_buffer.append`. Binary WebSocket frames are refused.
| `audio.input.format.type` | Rate | Notes |
| - | - | - |
| `audio/pcm` | `24000` (default), `16000` (native, no resampling), or any other rate | PCM16, little-endian, mono |
| `audio/pcmu` | 8 kHz, fixed | G.711 mu-law, as a telephony trunk or Twilio media stream sends it |
| `audio/pcma` | 8 kHz, fixed | G.711 A-law |
* Send about 20 ms per frame (640 bytes at 16 kHz, 960 at 24 kHz). One frame may not exceed 256 KiB; a larger frame closes the connection with code `1009`.
* Keep streaming during silence. The server decides where the turn ends from the audio; a client that stops sending looks idle, not quiet.
* One turn may hold up to 300 seconds of audio; anything longer is dropped.
* Audio in `conversation.item.create` is refused with `unsupported_item_type`. Stream caller audio; use `conversation.item.create` for text, tool results and history.
## Run a turn
With `create_response: true` the sequence for one caller turn is:
1. You stream `input_audio_buffer.append` frames.
2. `input_audio_buffer.speech_started`, then `input_audio_buffer.speech_stopped` and `input_audio_buffer.committed` when the model confirms the turn is over.
3. `conversation.item.added` and `conversation.item.done` for the caller's turn.
4. `response.created`, `response.output_item.added`, `response.content_part.added`, then `response.output_text.delta` frames until `response.output_text.done`, `response.content_part.done`, `response.output_item.done` and `response.done`. `conversation.item.input_audio_transcription.delta` and `.completed`, carrying what the model heard, arrive during the reply, usually after its first text delta.
With `create_response: false`, the model doesn't confirm the turn: step 2 follows the first pause the detector hears, or your `input_audio_buffer.commit`. Step 4 starts when you send `response.create`.
The samples below set `["text"]` so they can print the reply. Leave it out and the API speaks the reply instead: read `response.output_audio_transcript.delta` for the text and `response.output_audio.delta` for the audio.
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
"""Stream a WAV file to Dialog-RSN-1 and print the reply."""
import asyncio
import base64
import json
import os
import wave
import websockets
URL = "wss://api.studio.poly.ai/v1/realtime"
KEY = os.environ["POLY_API_KEY"]
async def main(path: str) -> None:
with wave.open(path) as w:
rate, pcm = w.getframerate(), w.readframes(w.getnframes())
async with websockets.connect(URL, additional_headers={"X-API-KEY": KEY}) as ws:
print((await recv(ws))["type"]) # session.created
await ws.send(json.dumps({"type": "session.update", "session": {
"instructions": "You work the front desk of a hotel. Keep replies to one sentence.",
"output_modalities": ["text"],
"audio": {"input": {"format": {"type": "audio/pcm", "rate": rate}}},
}}))
print((await recv(ws))["type"]) # session.updated
# 20 ms of PCM16 mono per frame, as a microphone would deliver it.
step = rate * 2 // 50
for i in range(0, len(pcm), step):
await ws.send(json.dumps({"type": "input_audio_buffer.append",
"audio": base64.b64encode(pcm[i:i + step]).decode()}))
await asyncio.sleep(0.02)
# Keep sending silence: the server finds the end of the turn itself.
silence = base64.b64encode(bytes(step)).decode()
reply = ""
while True:
try:
event = await asyncio.wait_for(recv(ws), 0.02)
except asyncio.TimeoutError:
await ws.send(json.dumps({"type": "input_audio_buffer.append", "audio": silence}))
continue
t = event["type"]
if t == "response.output_text.delta":
reply += event["delta"]
elif t == "conversation.item.input_audio_transcription.completed":
print("heard:", event["transcript"])
elif t == "response.done":
print("reply:", reply)
break
elif t == "error":
print("error:", event["error"]["code"], event["error"]["message"])
break
async def recv(ws):
return json.loads(await ws.recv())
asyncio.run(main("booking_request.wav"))
```
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Stream a 16 kHz PCM16 mono WAV file to Dialog-RSN-1 and print the reply.
import { readFileSync } from "node:fs";
import WebSocket from "ws";
const RATE = 16000;
const pcm = readFileSync("booking_request.wav").subarray(44); // skip the WAV header
const step = (RATE * 2) / 50; // 20 ms per frame
const silence = Buffer.alloc(step).toString("base64");
const ws = new WebSocket("wss://api.studio.poly.ai/v1/realtime", {
headers: { "X-API-KEY": process.env.POLY_API_KEY },
});
const send = (event: object) => ws.send(JSON.stringify(event));
let reply = "";
let keepAlive: NodeJS.Timeout | undefined;
ws.on("message", (data) => {
const event = JSON.parse(data.toString());
switch (event.type) {
case "session.created":
send({ type: "session.update", session: {
instructions: "You work the front desk of a hotel. Keep replies to one sentence.",
output_modalities: ["text"],
audio: { input: { format: { type: "audio/pcm", rate: RATE } } },
}});
break;
case "session.updated":
for (let i = 0; i < pcm.length; i += step) {
send({ type: "input_audio_buffer.append", audio: pcm.subarray(i, i + step).toString("base64") });
}
// Keep sending silence: the server finds the end of the turn itself.
keepAlive = setInterval(() => send({ type: "input_audio_buffer.append", audio: silence }), 20);
break;
case "response.output_text.delta":
reply += event.delta;
break;
case "conversation.item.input_audio_transcription.completed":
console.log("heard:", event.transcript);
break;
case "response.done":
console.log("reply:", reply);
clearInterval(keepAlive);
ws.close();
break;
case "error":
console.error("error:", event.error.code, event.error.message);
clearInterval(keepAlive);
ws.close();
}
});
```
Rules that catch people out:
* **One response at a time.** A `response.create` while a response is open is refused with `conversation_already_has_active_response`. Send `response.cancel` first. Two requests are held rather than refused: one sent while the caller is still speaking is answered by the response to their turn, and one sent while a reply the server speaks is still playing is answered when that audio ends. Only one request is held at a time.
* **`response.create` may set `instructions`, `max_output_tokens` and `metadata`.** `parallel_tool_calls` is ignored. Every other field must match the session or is refused with `unsupported_parameter`, naming `response.`. That includes `conversation: "none"` (no out-of-band responses), a per-response `output_modalities`, `tools` or `tool_choice`, `input`, `audio`, `prompt` and `reasoning`. Change the session with `session.update` instead.
* **A confirmed barge-in always cancels the reply.** `interrupt_response` cannot be turned off. The cancelled response ends with status `cancelled`.
* **Tell the server what the caller heard.** If you speak the text through your own TTS, send the `poly.output_audio.*` events below so that an interrupted reply is kept in history as the words that were actually played.
## Events
Every event is a JSON text frame with a `type` and an `event_id`. Fields follow the [OpenAI Realtime reference](https://platform.openai.com/docs/api-reference/realtime); only the differences are listed here.
**Client to server**
| Event | Notes |
| - | - |
| `session.update` | See [Configure the session](#configure-the-session) |
| `input_audio_buffer.append` | Base64 audio in the session's format |
| `input_audio_buffer.commit` | An empty buffer gets `input_audio_buffer_commit_empty` |
| `input_audio_buffer.clear` | |
| `conversation.item.create` | Messages, `function_call` and `function_call_output` items. Answered with `conversation.item.added` then `.done`. |
| `conversation.item.retrieve` | Returns the item. For a caller's audio turn that is the transcript, not the audio itself. |
| `conversation.item.delete` | |
| `conversation.item.truncate` | Cuts history only for a reply spoken by the server; otherwise validates the item and changes nothing |
| `response.create` | |
| `response.cancel` | With nothing active, `response_cancel_not_active` |
| `output_audio_buffer.clear` | Always refused: WebRTC only. Send `response.cancel`. |
| `poly.output_audio.started`, `.delta`, `.stopped` | Playback reports, see below |
**Server to client**
`session.created`, `session.updated`, `conversation.item.added`, `.done`, `.deleted`, `.retrieved`, `.truncated`, `conversation.item.input_audio_transcription.delta`, `.completed`, `input_audio_buffer.speech_started`, `.speech_stopped`, `.committed`, `.cleared`, `response.created`, `response.done`, `response.output_item.added`, `.done`, `response.content_part.added`, `.done`, `response.output_text.delta`, `.done`, `response.function_call_arguments.delta`, `.done`, `error`. On a session with `output_modalities: ["audio"]`, `response.output_audio_transcript.delta` and `.done` replace `response.output_text.*`, and `response.output_audio.delta` and `.done` carry the speech as base64 PCM16 mono at 24 kHz.
Not sent: `rate_limits.updated`, `conversation.created`, MCP events. Token usage arrives on `response.done` as `response.usage`.
**Differences a stock client notices**
* Text and function-argument deltas arrive per model chunk, not per token. The concatenation is the same.
* Transcription deltas are true increments.
* `previous_item_id` is set on `conversation.item.*` events, not on `response.output_item.*`.
* `speech_started.audio_start_ms` counts audio actually received, not wall-clock time.
* `session.created` and `response.created` omit optional fields that are not in effect (`usage`, `max_output_tokens`, `metadata` and others).
## PolyAI extensions
Extensions are additive fields on existing events, all prefixed `poly_`. Nothing is sent as a new server event type.
| Where | Field | What it does |
| - | - | - |
| `session` | `poly_knowledge` | A list of `{ "name", "content", "actions"? }` the model may cite in replies |
| `session` | `poly_response_language` | An ISO 639 tag (`es`, `pt-BR`) or words (`"Singlish"`). The model answers in that language and no other. `null` clears it. |
| `session` | `poly_input_rate` | Set for you when `audio.input.format.rate` is not 24000 |
| `session` | `poly_session_id` | Read-only echo of the `session_id` query parameter |
| `tools[].parameters.properties.` | `poly_spoken: true` | The model says this string argument to the caller while the tool runs: as `response.output_text.delta` on a text session, aloud on an audio session. One per tool, string parameters only. |
| `conversation.item.create` with `function_call_output` | `poly_knowledge` | Knowledge attached to one tool result |
| `response.done.response` | `poly_cited_topics` | Knowledge entries the reply cited; absent when none |
| `response.done.response` | `poly_out_of_domain` | `true` when the model judged the request outside its instructions |
| `response.done.response` | `poly_compacted_item_ids` | Items the prompt now leaves out, because older history was summarised or dropped to fit the context window. Sent on the first response after that changes. The items stay in history and can still be retrieved. |
| `response.done.response` | `poly_probe_usage` | The token cost of turn detection |
**Playback reports.** When you voice the reply yourself, send these so the server knows what the caller heard:
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{"type": "poly.output_audio.started", "response_id": "resp_123"}
{"type": "poly.output_audio.delta", "response_id": "resp_123", "audio_ms": 1250, "delta": "Sure, what's"}
{"type": "poly.output_audio.stopped", "response_id": "resp_123", "audio_ms": 2100, "dropped_ms": 900, "spoken_text": "Sure, what's your"}
```
Send a `delta` every 250 ms or so with the cumulative milliseconds played and, in `delta`, the text spoken since the last report. The server builds what the caller has heard so far from those increments, so it knows where to cut the reply before `stopped` arrives. On a barge-in, `stopped` carries what was dropped and the words that were actually spoken, and the reply is kept in history as those words.
A client that sends no reports gets an estimate instead: the server treats a reply as audible for as long as it takes to say at 15 characters a second, and after a barge-in keeps the words it estimates were heard. On a reply the server speaks itself, it tracks playback and ignores these reports.
## Errors
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"type": "error",
"event_id": "event_9b71...",
"error": {
"type": "invalid_request_error",
"code": "unknown_parameter",
"param": "session.bogus_field",
"event_id": "evt_2",
"message": "Unknown parameter: 'session.bogus_field'. Extensions to this protocol must be namespaced 'poly_*'; anything else is rejected rather than silently dropped."
}
}
```
Branch on `code`, never on `message`. `error.event_id` names the client frame the error is about, where there is one. `type` is `invalid_request_error` for something you sent and `server_error` for something that failed on our side. A `server_error` that ends a response also arrives on `response.done` with status `failed`. There, `status_details.error` carries the code in both `code` and `type`, for example `generation_failed`.
| `code` | `type` | Retry? |
| - | - | - |
| `unsupported_event_type` | `invalid_request_error` | No. This event is not implemented. |
| `invalid_request_error` | `invalid_request_error` | No. A malformed frame, undecodable audio, a binary frame, or a bad `session_id`. |
| `unknown_parameter` | `invalid_request_error` | No. The path is in `param`. |
| `unsupported_parameter` | `invalid_request_error` | No. The field is real and this service does not honour it; path in `param`. |
| `item_not_found` | `invalid_request_error` | No |
| `unsupported_item_type` | `invalid_request_error` | No. An MCP item or audio in `conversation.item.create`. |
| `input_audio_buffer_commit_empty` | `invalid_request_error` | No. Nothing was lost. |
| `conversation_already_has_active_response` | `invalid_request_error` | Yes, after `response.done`, or send `response.cancel` first |
| `conversation_item_limit_exceeded` | `invalid_request_error` | Not as sent. The conversation already holds as many client-created items, or as many bytes of them, as the deployment allows. Delete some with `conversation.item.delete` first. A `function_call_output` answering the outstanding call is never refused. |
| `response_cancel_not_active` | `invalid_request_error` | No |
| `session_expired` | `invalid_request_error` | Reconnect now. The session reached its maximum duration; the socket closes with `4410` straight after. |
| `context_length_exceeded` | `server_error` | Not as sent. The conversation is too long for the context window; shorten instructions, tools or knowledge, or start a new session. The connection stays open. |
| `capacity_exceeded` | `server_error` | Yes, after a short backoff. No inference capacity for this conversation. |
| `policy_violation` | `server_error` | No |
| `generation_failed` | `server_error` | Yes |
| `audio_synthesis_failed` | `server_error` | Yes. A spoken reply's audio could not be synthesised, so the response fails. Check `audio.output.voice` names a voice that exists. |
| `server_error` | `server_error` | Yes |
**Close codes**
| Code | Why | What to do |
| - | - | - |
| `1008` | Invalid `session_id` query parameter, after an `error` | Fix the id |
| `1009` | A frame over 256 KiB | Send smaller frames |
| `1011` | The session could not be started, after `server_error` | Retry |
| `4408` | No client event for 5 minutes, or the client stopped reading for 5 seconds | Reconnect when you next have audio to send |
| `4410` | The session reached its maximum duration, after `session_expired`. `session.created.expires_at` tells you when. | Reconnect now and carry on |
| `1013` | No capacity for a new session on this server, from a server that can't answer the handshake with a status | Retry after a second |
| `4401`, `4402`, `4403`, `4429`, `4503` | A refused handshake, from a server that can't answer it with an HTTP status | See [Authenticate](#authenticate) |
Handling a refused handshake and a timed-out session:
```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import json
import os
import websockets
from websockets.exceptions import ConnectionClosed, InvalidStatus
URL = "wss://api.studio.poly.ai/v1/realtime"
KEY = os.environ["POLY_API_KEY"]
async def run_session() -> None:
try:
async with websockets.connect(URL, additional_headers={"X-API-KEY": KEY}) as ws:
async for frame in ws:
event = json.loads(frame)
if event["type"] == "error" and event["error"]["code"] == "session_expired":
break # 4410 follows; open a new connection and continue the call
except InvalidStatus as e:
if e.response.status_code == 402:
print("No balance: add funds at studio.poly.ai") # body: {"detail": "..."}
elif e.response.status_code == 429:
# The gateway's rate limit sends Retry-After; the session cap doesn't, so close a session first.
print("Rate limited or at the session cap; Retry-After:", e.response.headers.get("Retry-After"))
else:
raise
except ConnectionClosed as e:
if e.rcvd and e.rcvd.code == 4408:
pass # idle; reconnect when there is audio to send
```
```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
ws.on("unexpected-response", (_req, res) => {
if (res.statusCode === 402) console.error("No balance: add funds at studio.poly.ai");
// Retry-After comes with the gateway's rate limit, not the session cap.
else if (res.statusCode === 429) console.error("Rate limited or at the session cap", res.headers["retry-after"]);
});
ws.on("close", (code) => {
if (code === 4410) reconnect(); // session reached its maximum duration; carry on in a new one
if (code === 4408) {} // idle; reconnect when there is audio to send
});
```
## Examples
There is no PolyAI SDK for this API; any OpenAI Realtime client works. Each guide has a public repo with its full example:
| Repo | What it is | Guide |
| - | - | - |
| [dialog-rsn-1-python-from-scratch](https://github.com/polyai/dialog-rsn-1-python-from-scratch) | Voice agent on plain `websockets`, with tool calling and a live microphone | [Python from scratch](/dialog-rsn-1/guides/python-from-scratch) |
| [dialog-rsn-1-javascript-browser](https://github.com/polyai/dialog-rsn-1-javascript-browser) | Browser microphone through a small key-holding relay | [Browser with JavaScript](/dialog-rsn-1/guides/javascript-browser) |
| [dialog-rsn-1-openai-migration](https://github.com/PolyAI-LDN/dialog-rsn-1-openai-migration) | The same agent on the `openai` SDK, before and after the switch | [OpenAI Realtime API](/dialog-rsn-1/guides/openai-realtime-migration) |
| [dialog-rsn-1-streaming-tts](https://github.com/PolyAI-LDN/dialog-rsn-1-streaming-tts) | Replies spoken through your own TTS as they stream, with barge-in | [Streaming to your TTS](/dialog-rsn-1/guides/streaming-text-to-speech) |
| [dialog-rsn-1-speech-to-text](https://github.com/PolyAI-LDN/dialog-rsn-1-speech-to-text) | Transcriber and support agent in Python and Node | [Speech to text](/dialog-rsn-1/guides/speech-to-text) |
| [dialog-rsn-1-voice-agent](https://github.com/PolyAI-LDN/dialog-rsn-1-voice-agent) | FastAPI voice agent with Cartesia speech and a browser page | [Browser voice agent with Cartesia](/dialog-rsn-1/guides/voice-agent-cartesia) |
| [livekit-plugins-polyai](https://github.com/PolyAI-LDN/livekit-plugins-polyai) and [dialog-rsn-1-livekit](https://github.com/PolyAI-LDN/dialog-rsn-1-livekit) | Dialog-RSN-1 as a LiveKit Agents realtime model, and a hotel agent built on it | [LiveKit](/dialog-rsn-1/guides/livekit) |
| [dialog-rsn-1-realtime-agents](https://github.com/polyai/dialog-rsn-1-realtime-agents) | Multi-agent handoffs, a chat supervisor and a knowledge agent, in plain JavaScript | |