Performance monitoring

Track metrics, diagnose issues, and optimize your agent.

Monitor your agent’s performance to identify issues before they impact customers.

Use Smart Analyst to investigate performance issues at scale. Its deep sampling analyzes up to 500 conversations per query, helping you quickly identify root causes behind metrics like high handoff rates, low containment, or latency spikes — without manually reviewing individual calls.

Quick reference

I need to…	Where to go
Check overall performance	Configure > Dashboards
Find slow responses	Analytics > Conversations → filter by latency
Debug ASR accuracy	Analytics > Conversations → Diagnosis → check transcription
Find knowledge gaps	Configure > Dashboards → unhandled queries
Check function errors	Analytics > Conversations → filter by errors
Track version performance	Compare versions in Deployments > Environments

Key metrics

Response latency

Time from when the user stops speaking to when the agent starts responding. Target: under 2 seconds.

Common causes of high latency: slow function execution, external API delays, complex knowledge retrieval, overly complex prompts.

Containment rate

Percentage of calls handled without human handoff. Target varies by use case (typically 60-90%).

Common causes of low containment: knowledge gaps, complex queries, caller preference for humans, technical errors.

ASR accuracy

How accurately the agent transcribes what the caller says. Target: above 95% word accuracy.

Common causes of low accuracy: background noise, strong accents, uncommon words or jargon, poor phone connection.

Monitoring tools

Dashboards

Go to Configure > Dashboards for high-level metrics: call volume, average duration, handoff rate, top intents, and performance trends. Filter by date range, environment, variant, or version.

Conversation Review

Go to Analytics > Conversations to drill into individual calls. Search, filter, listen to recordings, review transcriptions, and check the Diagnosis panel for technical details (function logs, knowledge retrieval, LLM prompts, timing breakdown).

Smart Analyst

Use Smart Analyst for deeper investigation. Its deep sampling capability analyzes up to 500 conversations per query, surfacing patterns across your data that would take hours to find manually. Launch it directly from dashboard charts using the Generate insights button, or go to Analytics > Smart Analyst and ask questions like “Why are calls failing containment this week?” or “What do low-PolyScore calls have in common?”

Test sets

Use test sets for automated regression testing and version comparison. Run them before promoting versions.

Diagnosing common issues

High latency

1. Filter Analytics > Conversations by high latency
2. Open Diagnosis → check the timing breakdown
3. Identify the bottleneck: function execution, knowledge retrieval, LLM generation, or TTS

Fixes: optimize slow functions, cache common audio phrases, switch to a faster TTS provider, simplify knowledge sources, add delay controls.

Low ASR accuracy

1. Review transcriptions in Conversation Review
2. Compare to audio recordings
3. Look for patterns (specific words, accents, noise)

Fixes: add custom vocabulary, adjust ASR sensitivity, add clarification prompts for ambiguous input.

Knowledge gaps

1. Check Configure > Dashboards → unhandled queries
2. Review common questions without answers in Conversation Review
3. Use Smart Analyst deep sampling to identify gaps at scale — try: “What questions are we not handling well?” or “Where does the agent give incorrect or incomplete answers?”

Fixes: add missing topics to Managed Topics, add Connected Knowledge sources, improve topic descriptions for better retrieval.

Function errors

1. Filter Analytics > Conversations by errors
2. Review Diagnosis → function logs

Fixes: fix code, update API credentials, add error handling and retries, add logging with conv.log for better debugging.

High handoff rate

1. Check Configure > Dashboards → handoff metrics
2. Review handoff reasons for patterns
3. Ask Smart Analyst: “What are the top 5 reasons conversations are transferred to a human agent?” — deep sampling gives you a percentage breakdown across hundreds of calls

Fixes: add knowledge for common handoff reasons, adjust handoff rules, add self-service options before handoff.

Optimization quick wins

Area	Quick wins
Latency	Cache common audio, switch to Cartesia TTS, use Turbo interaction mode, optimize slow functions
ASR	Add custom vocabulary, use clarification prompts, enable noise cancellation
Containment	Add missing knowledge, improve handoff rules, clarify agent capabilities upfront
Quality	Fix pronunciations, improve response clarity, test with real users

Debugging toolkit

All debugging tools available in Agent Studio:

Tool	Purpose	Where to find it
Diagnosis panel	Inspect function calls, knowledge retrieval, LLM prompts, and latency per turn	Analytics > Conversations → select a call → Diagnosis
conv.log	Add structured logging (info, warning, error) from Python functions	Function code → appears in Diagnosis
conv.log_api_response()	Log full HTTP responses from API integrations for debugging	Function code → appears in Diagnosis
Test sets	Automated regression testing across versions	Analytics > Test suite
Alerts API	Automated alerts for latency, errors, and call volume anomalies	API configuration
Smart Analyst	AI-powered analysis across up to 500 conversations	Analytics > Smart Analyst
Dashboards	High-level metrics: call volume, latency, handoff rates, containment	Configure > Dashboards

Related pages

• Smart Analyst — AI-powered conversation analysis with deep sampling
• Health checks — proactive monitoring routines
• QA and analytics — daily dashboard and conversation review workflows
• Function maintenance — debugging and optimizing functions
• Alerts API — automated alerts for latency, errors, and call volume