A voice agent that answers the phone has a problem a text-based system doesn’t: the listener can’t check a source while the conversation is happening.
In text, a doubtful claim can be checked before you act on it. On the phone, a wrong answer delivered naturally leaves nothing to catch it on: the person hears it, believes it, and the conversation has already moved on. This changes what’s worth measuring in a voice agent, and it’s why the most interesting number in this project isn’t 90%.
The number you notice first
90% of calls falling within the defined scope are resolved without human intervention. It’s the number that goes on a slide, and it’s true — but it only holds for the requests the agent was actually designed to answer. Out-of-scope requests are routed to an operator by design, not because of a limit discovered in production: the scope was defined upfront, based on historical calls that genuinely recur.
The number that matters more
On the remaining 10%, the system correctly routed to an operator in 70% of the low-confidence cases, instead of answering without being sure.
This is the number that describes whether the architecture actually works. A system that always answers, even when it shouldn’t, can look more useful in a demo and become a liability in production. A system that knows when to stop has a harder property to achieve and a more important one to measure, the same way abstention matters in document systems: it holds equally here, with one thing making it worse — in voice, there’s no time for a second thought.
How you measure it, when you can’t show the source
The evaluation set is made of real conversations, re-listened to and scored, including the calls where the agent correctly declined to answer. Three quantities are measured: correctness of the answers given, the share of calls completed without human intervention, and the escalation rate. The last one shouldn’t be read as a flaw to minimize: a well-judged escalation is worth more than a risky answer.
The constraint that shaped the architecture more than the quality of any single answer is latency: on a phone call, a two-second pause is already a broken conversation. The system builds every answer on a knowledge base queried in real time — not on the model’s memory alone — precisely because content you can update by editing the archive, instead of retraining the system, is the only way to stay correct over time within that time budget.
What’s still to be confirmed
The observation period for the measurement isn’t public yet. And the latency constraint remains structural: the architecture prioritizes conversational response times over the quality of any single out-of-scope answer — a declared design choice, not a hidden compromise.
In closing
A voice agent that always answers is easier to show in a demo. A voice agent that knows when to hand off the call is harder to build and safer to put into production. The 70% correct routing on uncertain cases is the number that tells the two apart, which is why we publish it alongside the 90%, not in its place.
The full case study describes architecture and scope. If you’re evaluating a voice agent for inbound calls, we’re available to talk through where to draw the scope for your case.
