The agent is accurate at three in the afternoon and useless at seven

In short

If an ordering agent is right at 15:00 and wrong at 19:00, the variable is audio, not the model — orders barely change shape between those hours. Three sources produce indistinguishable transcripts: the caller's environment, the venue bleeding through the handset, and the audio path itself. Bucket order-level error rate by hour and by line, and the split names which one you have.

BuildspaceLabs28 July 20268 min read7 sections

Key takeaways

An accuracy curve that follows service periods and not order complexity is an audio finding, not a model finding.
Split errors by hour, by line and by whether the call was forwarded. Three splits separate the three noise sources.
A narrowband phone path discards the frequencies that separate s, f and th — which is exactly where menu words collide.
Errors that rise with concurrent calls rather than with the clock are a latency symptom wearing a noise costume.
Hardware is an integration boundary. Specify what the capture has to deliver, then test against it; owning the device is someone else's job.

At 15:00 the agent takes a four-item order without a correction. At 19:00 it asks twice, mishears a side, and the ticket reaches the pass wrong. Nothing about the orders changed between those hours — a burger with no onions is the same object at both — so the variable is not the model, the prompt or the menu. It is the audio reaching the model, and there are three places that degrades.

The three are the caller's environment, the venue leaking into the handset or lane microphone, and the audio path between the two. In a transcript they are indistinguishable: all three produce a word that is wrong, missing or replaced by a plausible neighbour. Listening to a handful of failed calls will convince you of whichever one you noticed first. What separates them is where you cut the data.

Cut the errors three ways before touching anything

The confirming check takes an afternoon and it is worth doing before any tuning, because every fix below is expensive and only one of them is yours to make.

Score at the order level, by hour. Take 200 completed calls across a normal trading week, mark each order right or wrong against the ticket, and plot it by hour of day rather than by day. A curve that follows opening, lunch and dinner is the finding.
Split the same set by line. The main number, a second number, the lane and any handset staff answer directly are separate acoustic environments. If one degrades and another does not, you have halved the search.
Split by call path. Calls that terminate directly at the voice platform and calls forwarded from an old number take physically different routes. Tag which is which as the call arrives; you cannot recover it afterwards.
Record concurrency per call — how many calls the agent was handling when this one started. If errors correlate with that more strongly than with the clock, stop: the diagnosis is latency, not noise.
Listen to matched pairs. From the same hour and the same line, take one clean call and one failed call. Note what is behind the caller, what is behind the handset, and whether the audio sounds thin, clipped or gargled.

Three sources, and the split that identifies each

What the split shows	Source	What you hear in the matched pair
Errors rise by hour on the phone lines and not on the lane	Caller-side environment — cars, streets, other people, a group ordering at once	The caller is the noisy one. Road noise, a second voice, a hand over the microphone
Errors rise by hour on every inbound line at once, including calls where the caller is plainly somewhere quiet	Venue-side noise through the handset or lane microphone	The room. Extraction fans, plates, the pass calling out, another conversation at the counter
Errors are flat by hour but concentrated on forwarded calls	The audio path — extra hops, re-encoding, packet loss	Thin, watery or intermittently clipped audio, with both sides sounding fine otherwise
Errors track concurrent calls rather than the clock	Latency, causing the caller to talk over the agent	Overlap. The caller starts speaking before the agent has finished, and half the utterance is lost
Errors concentrate on the same handful of items at every hour	Menu ambiguity that fast speech exposes	Perfectly clean audio and a wrong item. Nothing is wrong with the recording at all

The same wrong word, produced three different ways

The first two rows are the ones teams conflate, and they have opposite fixes. Caller-side noise is not yours to remove — you cannot quieten a car. What you change is how the agent behaves once it detects it: shorter prompts, one field per turn, an earlier confidence threshold, a hand-off before it commits. Venue-side noise is yours, it is a placement problem, and it is the one that reliably worsens at 19:00 because that is when the room is loudest.

What the path removes before the model hears anything

A traditional telephone path carries roughly 300 Hz to 3.4 kHz. That band was chosen for two humans who can ask each other to repeat, and it discards most energy above 4 kHz — precisely where the fricatives live. The consonants that separate one menu word from another go first, so confusions cluster on short words with a single distinguishing consonant.

Count the hops. A mobile call reaching a number that forwards to a second number that forwards to the platform can be encoded and decoded three times. Each transcode is lossy and none of them is recoverable. Terminating the published number directly at the platform removes the whole class.
Check packet loss and jitter separately from bandwidth. Loss of a few percent is barely noticeable to a person, who fills the gap from context, and destructive to a transcriber, which does not. Ask your provider for per-call figures and join them to your error table.
Ask whether the path is narrowband end to end. One narrowband leg makes the whole call narrowband, and a wideband codec on the platform leg buys nothing behind an 8 kHz carrier leg.
Check for one-sided problems. If the agent hears the caller badly but the caller hears the agent perfectly, that is a capture or inbound-path fault rather than general line quality, and it halves the places to look.

Every one of these failures reads the same way in a transcript: a word that is wrong, or a word that is not there. The recording is the evidence; the transcript is the summary of the evidence.

The one that is not noise at all

Between 18:30 and 20:00 the agent holds several calls at once, and if response times stretch under that load, callers speak into the gap. The overlapped utterance is the one that gets lost, and in a transcript that looks exactly like a mishearing. The tell is that error rate tracks concurrent calls rather than the hour — a quiet Tuesday at the same concurrency fails at the same rate, which noise would not do. That is a latency problem, taken apart stage by stage in the pause after the guest stops talking.

The two also amplify each other. Poor audio makes the agent ask more clarifying questions, each question adds a turn, and longer calls raise concurrency for every other caller — which is why noise and latency look like one thing from the outside during the exact 90 minutes you care about.

The last row of the table is the cheapest to fix and the easiest to miss. At 15:00 a caller says the item name slowly and alone. At 19:00 it arrives inside a run-on sentence with three modifiers and a drink, and two items that were already close now collide. If one dish fails at every hour and simply fails more as speech speeds up, the recording is fine and the catalog is the problem — the job of the alias and pronunciation layer in teaching an ordering agent to hear your menu.

A related case is not ambiguity but impossibility. If the guest asks for something the item cannot express — a topping on one half of a pizza, when the modifier has no portion dimension — the agent will produce a confident, clean, wrong ticket at any hour of the day. That is traced in half-and-half orders arriving as whole ones, and no amount of audio work touches it.

What a better model fixes, and what it cannot

Play the failed recording to a person who knows the menu. If they cannot make the words out either, no model will. The fix is capture: handset placement away from the pass, a headset instead of a shared handset, or a different lane microphone.
If a person hears it clearly and the transcript is wrong, a different speech engine may genuinely help. Re-run your failed recordings through the candidate before changing anything live — a one-day experiment with a real answer at the end.
If the words are transcribed correctly and the wrong item lands, this is a retrieval and alias problem. Fix the catalog layer and leave the model alone.
If the loss is overlap, tune end-of-speech detection and barge-in and cut the number of turns. Accuracy work here is wasted effort.
If failures cluster on forwarded calls, this is routing: shorten the chain, or port the number so it terminates where the agent runs.
If none of the above can be improved inside the room — a lane at peak, a counter with no quiet corner — change the channel rather than the model. Moving repeat orders to a screen removes the microphone from the problem entirely, and the readiness checks for that live in what a kiosk needs before it faces a queue.

What changes in the build once you know which one it is

Caller-side noise calls for a behaviour change rather than an accuracy change: detect low input quality early, drop to one question per turn, confirm harder, hand off sooner. That last step only helps if the destination is real — a transfer ringing an unattended handset is a lost order with extra steps, the failure taken apart in the handoff that fires and the call still dies.

Venue-side noise is layout: where the line terminates, what device answers it, and whether that device sits two metres from an extraction fan. Path problems are telephony configuration. None of the three is solved by a prompt, and all three are visible in a table built from data you already hold — standing instrumentation that belongs in internal tools and operations software rather than a quarterly investigation repeated from scratch.

The wider decisions around a phone agent sit in voice and phone ordering, and the surrounding systems in software for restaurants and food service.

Frequently asked questions

Short answers to the follow-ups this page tends to raise.

Would switching speech models fix accuracy at peak?

Only if a person listening to the same recording can hear the words the transcript got wrong. That single test separates a model problem from a capture problem, and in a restaurant the capture problem is more common. Re-run your archive of failed recordings through a candidate engine first: a day's work, answered with your own audio rather than a vendor benchmark.

Is a drive-thru lane a harder audio problem than a phone line?

Yes, for reasons software cannot reach. A lane microphone sits in the open, close to an idling engine, and often runs half-duplex so guest and agent cannot speak at once. A phone caller at least holds a handset near their mouth. Lane and phone therefore need separate error tables and separate targets — averaging them hides both.

How much of this can be fixed without buying anything?

Usually the path and the behaviour, which is more than teams expect. Removing a forwarding hop, terminating the number where the agent runs, tightening the confidence threshold at peak, cutting turns and moving a handset away from the pass are all configuration or layout changes. Buying better capture only helps once you have shown venue-side noise is the dominant source.

Should the agent just refuse to take orders during the rush?

No, but it should change what it commits to. A useful pattern is to keep taking simple orders and to escalate the complex ones earlier when input quality is poor, rather than switching the whole line off. Blanket refusal at peak gives up the calls with the most revenue attached, and it hides the underlying fault instead of making it measurable.

voice ordering
audio quality
diagnostics
telephony

// shipped work

The work behind this page

Builds from our portfolio that this page draws on.

Low Latency Food Ordering Platform

Unified events operations platform: vendor management, order tracking, payments, automated settlements.

Marketplace

Related across the site

Restaurants & Food Service AI agents in production: what actually breaks

Working on something in this space?

Tell us where you are in a sentence or two. We'll tell you honestly whether we're the right team, and what a sensible first slice of the work looks like.

Start the conversation