The ticket that changed how we think about escalation.

The ticket that changed how we think about escalation wasn't dramatic. Nobody raised their voice. Nobody asked for a manager. No keyword fired, no flag went off, and by every measure our system had at the time, the call had gone fine. A support agent flagged it anyway, almost as an aside, and it's the single ticket I think about more than any other we've received.

The ticket#

An agent on our customer success team was reviewing calls for one of our healthcare customers — nothing unusual, part of the routine spot-checks we do. One transcript was a patient, older, calling to confirm an appointment time. She asked the same question three times, in slightly different words, across the call. AIVA answered it correctly each time. Technically, nothing went wrong. The agent's note was one line: "Something about this doesn't sit right. Can someone listen to the audio, not just read it?"

I listened to the audio. The words were unremarkable. The pacing wasn't. There were long pauses before each repeated question, a slight waver the transcript stripped out entirely — the sound of someone who wasn't confused about the answer so much as unsettled by something the call hadn't addressed. AIVA had answered the question she'd asked, correctly, three times, and had completely missed the person underneath it.

What our detection actually looked like at the time#

It's worth being specific about what we'd built, because the gap only makes sense once you see the shape of what existed. Our distress detection at the time ran two layers: a set of explicit trigger phrases — "let me speak to someone," "this is ridiculous," things a person says when they've decided to escalate themselves — and a sentiment model that scored the emotional tone of what was said, tuned to catch frustration, anger, and their more polite cousins. Both layers were doing their job. Neither layer had any concept of pattern across turns — the fact that a question had already been asked and answered wasn't information either system carried forward into how it scored the next turn. Each turn was evaluated close to independently. A caller could ask the identical thing five times, calmly, and every single instance would score as a normal, resolved exchange, because nothing about the sentiment or word choice in any individual turn looked wrong.

Why our system didn't catch it#

At the time, our distress detection looked for what most systems look for: specific words, an angry tone, an explicit request for a human. This call had none of those. It had repetition, without any of the language that usually comes attached to it — no "I've already told you this," no frustration in the words themselves, just the same question, asked again, quietly. We hadn't built a category for that yet. We'd built for the customer who says "this is unacceptable." We hadn't built for the one who just quietly asks the same thing three times because something else is going on that she isn't going to say out loud to a piece of software.

That gap is obvious in hindsight, which is exactly what makes it worth writing about. Nothing crashed. Every individual answer AIVA gave was correct. The system did exactly what we'd told it to do, and what we'd told it to do turned out to be an incomplete definition of when a person needs a person. It's a well-documented failure mode in conversational systems generally, not something unique to us — usability researchers who study conversational UX point to the same pattern: a system can score every individual turn as correct and still fail the person having the conversation, because correctness per turn was never the same thing as understanding the person across turns.

What changed#

We added repetition itself as a signal, separate from the angry-language detection we already had — a caller asking the same thing more than once in a call, regardless of tone or word choice, now routes to a person. It's a blunter rule than I'd like, and it occasionally escalates calls that didn't strictly need it. We've kept it anyway, because the alternative is optimising for a metric that would have called that patient's call a success.

The trade-off I don't want to hide#

I want to be direct about the cost, because it's real and it's worth naming rather than glossing over: this rule does escalate some calls that, judged purely on correctness, resolved fine without a person. Every one of those is a call that could have stayed automated and instead now waits on a human, which runs slightly against the resolution rate we otherwise care a great deal about. We accepted that trade on purpose. AIVA's voice pipeline resolves the large majority of calls without a person, and that number matters to us — but it was never supposed to be a number we protected by narrowing our definition of what counts as a problem. A rule that occasionally over-escalates is a cost we can see and measure. A rule that under-escalates a quietly distressed caller is a cost we mostly can't see at all, until someone happens to listen closely enough to notice, the way one agent did on one ordinary Tuesday.

What it's caught since#

The rule hasn't only caught cases like the original one, and I think that's worth being honest about too. Plenty of the repetition-triggered escalations turn out to be someone who's just genuinely confused by a menu of options, or didn't hear the first answer clearly over a bad connection — not the same underlying situation as that first patient at all. We escalate those the same way regardless, because building a system that tries to guess which repeated question is "the serious kind" puts us right back to designing for the pattern we already know we can't detect reliably. Better to route every instance to a person and let a human make the finer judgment call that the software shouldn't be trusted with.

How this connects to the rest of escalation#

Repetition is now one signal among several, sitting alongside the more obvious triggers — explicit requests for a human, frustrated language, questions genuinely outside what a business has configured AIVA to handle. We've laid out the fuller logic elsewhere, and the throughline across all of it is the same lesson this ticket taught us: a handoff is only as good as the context that travels with it. When a call escalates, the person picking it up needs to hear what already happened, not start the conversation from zero — that expectation is exactly what we tell prospective customers to ask about before they buy anything in this category, because a handoff without context just moves the frustration to a new person instead of resolving it.

Why we still go looking for calls like this on purpose#

The habit that came out of this ticket isn't really a rule at all — it's a review practice. Our support team now spends part of every week specifically listening to calls that never triggered any flag, on the theory that the next gap in our detection is, by definition, going to look exactly like this one did: quiet, technically correct, and easy to skip past if nobody's deliberately looking. Our design lead runs a version of the same discipline from a completely different angle — calling the line himself, over and over, specifically to catch what a transcript won't show. Neither of us found this by trusting the dashboard. We found it by listening past the point where the dashboard said we could stop.

I still don't know what was actually going on for that caller. We escalate that pattern to a person now, every time, precisely because we don't know, and a transcript full of correct answers was never going to tell us. The support agent who flagged a call that technically hadn't gone wrong taught the engineering team something no dashboard full of green metrics would have: sometimes the conversation your system says succeeded is exactly the one you should have looked at harder.