Voice is where customer service automation is heading, and where it is hardest to get right. Gartner has predicted that conversational AI will cut contact centre agent labour costs by US$80 billion in 2026, with roughly one in ten agent interactions automated. Much of that shift is now moving from chat to voice, and in Asia it lands on the telephone, which remains the channel customers reach for when something has gone wrong.
The demonstrations look finished: a clear question, a fluent answer, a pause that feels natural. Production is a different environment. Callers hesitate, interrupt, switch languages mid-sentence, and phone from crowded streets on mobile networks. Requests depend on business systems that were never designed to answer inside a conversational window, and some questions are sensitive enough that the right response is to stop automating and find a person.
What is less understood is that the industry has settled on a way of reporting its own responsiveness that makes most of this invisible. The figures vendors publish are real. They are also taken at a point inside the system that has very little to do with what the caller experiences, and enterprises are making investment decisions on them.
The yardstick problem
Ask a voice AI vendor how quickly its system responds and you will usually be given a figure between two hundred and eight hundred milliseconds. Ask where it was taken and the answer, if you get one, is almost always the same: at the model boundary, the gap between a transcript arriving at the AI service and the first generated audio leaving it. That measurement is easy to produce, easy to improve, and easy to compare across vendors. It is also taken at a point where nobody is listening.
Telephony has never worked this way. It has long measured mouth-to-ear delay, the time between a person speaking and the other person hearing that speech, because that is the transmission delay a caller can perceive. ITU-T Recommendation G.114 has framed its guidance for conversational voice in those terms for decades, treating one-way delay of up to about 150 milliseconds as acceptable for most applications and recommending that 400 milliseconds should not be exceeded for general network planning. Linguists who have measured turn-taking across languages put the typical gap between speakers at around 200 milliseconds. Callers do not know these numbers, but they feel them.
An industry that grew up inside the network measures where the human is. An industry that grew up inside the model has ended up measuring where the model is.
What the number hides
“Voice AI” is shorthand for a considerable amount of machinery. A live service has to connect the call and carry its audio across a network, detect that someone is speaking and decide when they have stopped, transcribe the speech, interpret the request and decide what the system may do with it, retrieve information or call a business application, then compose a response, convert it to speech, package it as audio, and deliver it back. A model-boundary figure covers one or two of those steps.
The largest of the missing pieces usually comes first. Before the model is called, the system waits for silence to decide the caller has finished, and that threshold is typically several hundred milliseconds, set cautiously so that people are not cut off mid-sentence. It tends to be set more cautiously still where callers mix languages or pause in unfamiliar places, which describes much of Asia. After the model responds, the synthesised audio still has to be streamed, carried across the network, and reassembled at the far end, and a call arriving over the public telephone network rather than a browser may cross two or three further systems on the way back.
So an enterprise comparing two vendors on model-boundary latency is comparing the one part of the system both have already optimised heavily, while ignoring the parts where their architectures genuinely differ. Two suppliers quoting four hundred milliseconds may be describing experiences that differ by a second once the call reaches a handset.
A leadership problem, not an engineering detail
What an organisation measures is what it ends up funding. A team working to a model-boundary figure will put its effort into inference speed, since that is the only thing the number responds to, and usually buy faster capacity or a faster model. A team working to an end-to-end figure will often find that the largest single component of the delay is the silence threshold, a configuration decision rather than a hardware one, and therefore the cheapest improvement available. Return paths, lookups under load, and interruption handling become visible too.
Procurement inherits the same problem. If you cannot establish where a supplier’s figure was taken, you cannot compare it fairly with anyone else’s, or hold anyone to it in a service review a year later. And the customer bears the consequence. The savings in every board paper assume the automated call is completed; a caller who gives up on a stilted conversation and redials to reach a person is a cost, not a saving, and one that no component dashboard will show.
A roadmap for measuring what callers hear
- Map the whole path before selection. Trace where audio, transcripts, and business data travel, every supplier hand-off and every dependency, so that technology, security, compliance, and procurement teams share one picture of what they are approving.
- Set the yardstick in the request for proposal. Define responsiveness as the interval from the moment the caller stops speaking to the moment response audio leaves the platform, on the kind of call path real customers use, and ask every vendor to demonstrate its figure on that basis. Willingness to do so is itself informative.
- Write the measurement point into the contract. Service levels should reference the end-to-end figure, with network transit and handset buffering characterised separately and reported alongside it, so the number quoted at selection is the number the vendor is held to afterwards.
- Instrument for outcomes. Tracing one interaction end to end, with consistent identifiers and reliable timestamps, shows where the time went and whether the caller got what they rang for. Dashboards should track abandoned calls, repeated questions, interruptions, and hand-offs rather than component uptime. This is the same instrumentation needed to run the service well, so most of its cost is already in the budget.
- Give someone the caller’s outcome. A production voice service may cross a telecoms provider, a cloud platform, one or more AI vendors, and the organisation’s own applications, each meeting its own service levels while the call is poor. One team needs the visibility to investigate across those boundaries and the authority to change the design when the answer is inconvenient. That is far easier to settle before launch than during the first serious incident.
Where this is heading
The gap between the published number and the caller’s experience is about to widen, not close. Speech-to-speech models that skip transcription altogether are shrinking the model-boundary figure toward zero, and as that number falls it will say less about the conversation, not more, because everything that remains sits outside it. The vendors that measure honestly will treat an end-to-end figure as a differentiator, and buyers will come to demand it the way they demand uptime commitments today.
Enterprises that set their own yardstick now will be able to hold suppliers to account, fix the cheapest things first, and scale their services. A capable model sits inside a service that has to be measured honestly before anyone can improve it. A system measured from the caller’s position gets built differently from one measured from the vendor’s, and after a few iterations the two have diverged a long way.
