The number we optimize voice latency against isn't zero, and it isn't "as fast as possible." It's roughly 300ms — the average gap between one person finishing a sentence and the other person starting to respond in a normal human conversation. That's the actual bar. Not a competitor's benchmark, not an internal target that sounded good in a planning meeting. The pause length people are already used to, from a lifetime of talking to other people.
Why that's the right number to chase
Most conversational systems get measured against "fast for a machine" — a couple of seconds, maybe under one second for a good one. That standard comes from comparing against bad IVRs and slow chatbots, not against actual conversation. It's the wrong reference point. Nobody experiences a call with AIVA and thinks "that was fast for an AI." They experience it as a conversation, and they compare it, unconsciously, to every other conversation they've ever had — not to their expectations of automated phone systems.
Human conversational turn-taking has been studied for a long time, and the consistent finding is that the gap between turns is short — close to the 300ms figure we use as our line. Go much slower than that and the listener's brain registers something as off, the same instinctive way you notice a satellite call with delay before you can articulate why it feels wrong. It doesn't read as "the AI is thinking." It reads as "something's wrong with this call."
What's actually happening in that 300ms gap
It's worth being specific about what a human listener is doing in that window, because it explains why the number is a real constraint rather than an arbitrary round figure. A person isn't purely reacting after their conversation partner finishes speaking — they're predicting the end of the sentence before it arrives, based on grammar, intonation, and context, and lining up a response to fire the instant the actual gap opens. That's part of why human turn-taking gaps can stay so short without conversation constantly overlapping into cross-talk: a meaningful share of the "thinking" already happened during the other person's sentence, not after it.
A voice agent doesn't get to borrow that shortcut for free. It has to hear the full utterance, decide the caller is actually done rather than just pausing, understand what was said, decide what to say back, and generate that response as audio — all inside roughly the same window a human fills partly with anticipation. Hitting 300ms isn't "respond quickly." It's compressing a chain of distinct steps into a budget human conversation spreads out by starting early.
Where 198ms comes from
AIVA's voice pipeline currently averages around 198ms — under the 300ms line, not just close to it. That number is the result of a lot of separate work: the pipeline rewrite that cut out redundant network hops, inference optimizations that shaved specific milliseconds out of specific stages, and regional routing across Mumbai, Frankfurt, and Virginia that keeps calls from paying distance they don't need to pay. None of those projects were pursued as abstract speed goals. Each one was justified against the same question — does this get us further under the line where a caller stops noticing they're talking to software.
198ms is an average, not a floor. Individual calls vary with network conditions, language, and what's actually being asked. The target that matters operationally is staying under 300ms consistently, not chasing the average lower for its own sake once we're already there.
Where the average hides real variance
An average is only useful if you also know what it's averaging over. A short, predictable exchange — confirming a time, answering a yes-or-no question — sits comfortably under 198ms most of the time. A longer, more complex request, the kind where AIVA has to check a calendar, reconcile it against something the caller mentioned two sentences earlier, and compose an answer that reflects both, involves structurally more work than "what are your hours." The honest position is that some individual turns run slower than the average by design, because generating a fast wrong answer isn't actually faster than a right one delivered a beat later — it just moves the delay to the follow-up call where the caller has to correct it.
Language and network conditions add their own variance on top of that. Twelve Indian languages don't all carry identical processing cost — some have denser grammar to parse than others — and a caller on a poor mobile connection introduces delay before AIVA's pipeline even starts, from packet loss and jitter that has nothing to do with how fast our servers respond. None of that is hidden inside the headline number; it's why we track consistency under 300ms as the actual operational target, rather than treating 198ms as a promise every single call has to individually clear.
Speed alone isn't the whole story
None of this works if the fast response is wrong. A caller who gets an instant reply that doesn't answer their question, or gets rushed past what they were trying to say, isn't having a better conversation than one who waited slightly longer for a correct, well-handled answer. Response speed and resolution are the same project from two different angles — it's the combination that gets AIVA to resolving around 96% of calls without a human, not either one alone. A fast wrong answer just fails faster.
This is also where handling interruptions well and hitting a latency target turn out to be the same underlying problem wearing two faces. A caller who corrects themselves mid-sentence — "Tuesday — actually, no, Wednesday" — needs that correction processed inside the same tight budget as an ordinary turn. Slow down for the hard case and the easy cases stop feeling special; they just become the baseline everyone was quietly expecting anyway.
Why we don't chase the average lower for its own sake
It would be easy to treat 198ms as a number to keep shrinking indefinitely, the way a lot of engineering metrics invite you to. We don't, because the number stops mattering once it's reliably under the human baseline it's measured against. Getting from 198ms to 150ms wouldn't make a call feel more natural to a caller who already can't consciously perceive the gap — human perception of a pause has its own floor, and once you're under it, further gains stop showing up as "feels more human" and start showing up as engineering effort spent on a margin nobody notices. What we actually watch for is the tail, not the average: the calls that spike well past 300ms because of a bad connection or an unusually complex request, since those are the ones a caller actually feels.
What this looks like when it goes wrong
The clearest way to understand why 300ms matters is to picture missing it. A caller asks a simple question, and there's a pause just long enough to notice — not dramatic, not a full second, just enough that the rhythm of the exchange breaks. The caller doesn't think "the model is running inference." They think "is this thing still there," and often say "hello?" into the silence, the same reflex people use when a phone call drops mid-sentence. That reflex is the whole argument for the benchmark in one moment: the failure mode of bad latency isn't a complaint about speed, it's a caller quietly deciding the call feels broken and disengaging a little, even when the eventual answer is completely correct.
The actual point of the benchmark
300ms isn't a marketing number. It's a design constraint that forces a specific kind of honesty: you can't hide behind "acceptable for AI." You either get under the gap a real conversation leaves, or the caller feels the difference whether or not they could tell you why. We picked that line because it's the only one that actually measures what we're trying to build — not a fast bot, a conversation that doesn't feel like one.
If you want to hear where that number actually lands on a real call, try AIVA yourself — it's a better test than any number in this post.