Voice AI latency: benchmarks and fixes
Mouth-to-ear latency, the delay between a user finishing speech and the AI agent's first audible response, determines whether a voice AI call feels like a conversation or a transaction. Current benchmarks show multiple platforms achieving sub-second or near-sub-second latency, with several competitors reporting competitive figures, while Twilio's own target sits at roughly 1,115ms mouth-to-ear. The industry is converging on acceptable conversational latency, but the gap between median and tail performance still separates production-ready systems from prototypes.
Voicing Team5 min read
Voice AI ArchitectureWhat is latency in Voice AI?
Latency in voice AI systems is not the same as general network latency. It is the total mouth-to-ear turn gap: the time from when a user stops speaking to when the AI agent's reply reaches their ear. Two metrics define this gap. Time-to-First-Token (TTFT) measures how long the LLM takes to generate its first token, the smallest unit of output the model produces. Time-to-First-Audio (TTFA) measures how long the TTS engine takes until the first piece of response audio is available to play and reach the listener. TTFT and TTFA are model-generation metrics; mouth-to-ear latency is the end-to-end figure that includes network transmission on both ends.
Human conversation runs on sub-second delays. The IMC 2025 academic study found that major GenAI calling applications from Google, Meta, Microsoft, and OpenAI all exceed 1-second TTFT, with some reaching 5 seconds at the 95th percentile. Peer-reviewed research places human conversational turn gaps at approximately 200ms (Stivers et al., PNAS 2009), with tolerance degrading from roughly 600ms and breaking down past approximately 800ms (Roberts & Francis 2013; Kohtz & Niebuhr 2017).
Components of latency in Voice AI systems
A cascaded voice agent pipeline runs audio through discrete stages: Speech-to-Text (ASR), LLM inference, and Text-to-Speech (TTS) synthesis, plus network transmission on both ends. Each stage consumes part of a fixed latency budget.
Twilio's published benchmark breakdown shows how this adds up: audio travels from the user's device to the media edge (40ms), buffering (30ms), decoding (25ms), then STT processing (350ms), LLM inference (375ms), and TTS synthesis (100ms), with service hops between stages adding roughly 10ms each. The total reaches approximately 1.1 seconds mouth-to-ear.
The entire STT-to-LLM-to-TTS loop must complete in under 500 to 600ms to feel real-time. Several factors inflate this budget. Network geography adds 50 to 100ms per leg when STT, LLM, and TTS sit in different regions. CPU and GPU availability, model size, and architecture drive LLM inference time higher. Protocol choice matters: the IMC 2025 study found streaming uplink yields roughly 40ms inter-arrival time, the interval between consecutive audio packets arriving at the server, versus 300ms for batch-based delivery, which means the model receives user speech in near-real time instead of waiting for large chunks and can start processing sooner. Cold starts add 10 to 30 seconds when containers initialize from idle, and default audio player buffers of roughly 500ms further inflate the total. The orchestration layer adds overhead if it serializes requests or waits for complete responses before forwarding.
Acceptable latency benchmarks and real-world impact
CreativeGenius.ai's 90-day production benchmark of 11 platforms, covering 12,400 live calls, popularized the under-800ms p95 threshold as a widely repeated vendor heuristic for where conversation starts to feel broken rather than human. The figures from that study, including a custom stack at 540ms p50 and 820ms p95 alongside other platforms clustering in a comparable range, composite results from different methodologies and should not be treated as a matched set. Independent benchmarks have measured some of these platforms at different latencies. What is consistent across studies: platforms above 1.5 seconds p95 see callers hang up.
Call completion rates in the CreativeGenius benchmark ranged from 71% for the custom stack down to 47% for the lowest performer, driven largely by interruption handling and barge-in quality. Published-pricing platforms cluster between $0.09 and $0.22 per minute all-in, while enterprise-only platforms run 3 to 5 times that.
Voicing.ai's voice AI agent targets sub-800ms latency with 97% function calling accuracy, so a call that starts with a spoken request can end with the requested action completed, not just acknowledged. For contact center leaders evaluating AI call assistants, the difference between 540ms and 1.7 seconds p50 is the difference between containment and abandonment.
Strategies for reducing latency
Three architectural decisions drive latency down.
- Streaming the entire pipeline. STT partials every 50 to 100ms, LLM token streaming, and TTS generating audio from partial text all overlap stages that would otherwise run sequentially.
- Co-locating services. Placing STT, LLM, and TTS in the same region or VPC reduces cross-region hops. Twilio recommends colocating at the media edge nearest the customer, since the public network leg is the slowest and least reliable part of the path.
- Trimming audio player buffers. Default buffers of roughly 500ms can be cut to 50 to 150ms on stable connections. Warm pools and prefix caching prevent cold starts that add 10 to 30 seconds of initial latency.
Voicing.ai deploys with tailored infrastructure that supports cloud or on-premises placement, letting enterprises co-locate components for their specific network topology. The platform deploys in 2 to 4 weeks, compared to 8 to 20 weeks for competitors.
Measuring latency in Voice AI systems
Measuring end-to-end mouth-to-ear latency matters more than tracking platform-internal metrics alone. Production teams should track P95 and P99 latency, not medians, because tail latency is what degrades user experience. Component-level metrics like Round Trip Time (RTT) and Time-to-First-Byte (TTFB) for each service, STT, LLM, and TTS, pinpoint which stage consumes the most budget. Voicing.ai's AI Observability capabilities provide real-time, turn-by-turn audit and live latency monitoring so teams can identify and fix bottlenecks during calls, not after.
Frequently asked questions about voice AI latency
- How do you reduce latency in voice AI agents?
- Stream every component (STT partials, LLM tokens, TTS from partial text), co-locate STT, LLM, and TTS in the same region, use warm pools to avoid cold starts, and trim audio player buffers from 500ms to 50 to 150ms on stable connections. Persistent WebSocket or gRPC connections also reduce overhead.
Next steps for hitting your latency target
Human-like response speed is achievable with the right architecture: streaming pipelines, co-located services, tuned buffers, and warm pools. The platforms that deliver it combine custom LLMs, optimized ASR and TTS, and infrastructure placed where the customer is. Measuring mouth-to-ear latency at P95, not median, is what separates systems that feel human in production from ones that only perform in demos.
If your platform's p95 sits above 800ms, you are losing calls you do not need to lose. Talk to our team about deploying a sub-800ms voice AI stack with 97% function calling accuracy in your environment.
Bring one call type. Leave with an architecture.

Voice infrastructure on the contact centre floor
A working session with an engineer who has deployed inside a bank’s perimeter. We map your telephony, data boundary and handoff rules, and tell you what we would not automate.