A no-heat call got booked as a routine tune-up

The observable is specific: a job on the board at routine priority, scheduled 6 days out, whose recording has a caller saying the heating has been off since yesterday and there is a baby in the house. Nothing errored. The booking is well formed, the customer was polite, and the agent's log shows a completed call.

The reflex is to blame the model and start rewriting the prompt. That is almost always the wrong first move, because two much duller causes account for most of these: the tier policy does not exist in a form anything can apply, and nothing forces the agent to escalate when it is unsure. Both are measurable in an afternoon.

Confirm the mis-tiering before you touch the agent

One bad booking is an anecdote, and a dispatcher who was already sceptical will supply three more. What you need is a rate, split by the two dimensions that actually move it.

1. Pull every job booked by the agent over 14 days. If volume is high, sample 200 at random rather than taking the most recent 200, which over-weights whatever changed last week.
2. Join each booking to its recording or transcript. Any booking you cannot join is its own finding: an agent whose decisions are unauditable cannot be improved, only replaced.
3. Have 2 people who know the business score each call against the operator's own tier table, blind to the tier the agent chose.
4. Where the 2 scorers disagree, keep the call in a separate pile. That pile is the single most useful artefact of the exercise.
5. Compute under-tiering and over-tiering rates separately, then split both by trade and by hour of day, in 3 buckets: business hours, evening, and overnight.
6. Read 10 mis-tiered transcripts end to end before drawing conclusions. The pattern is usually visible in the wording of the questions, not in the statistics.

What the numbers point at

The overnight split matters more than teams expect, because after-hours calls often run through a different path, a different prompt or a different fallback entirely — the same seam that produces overnight bookings missing from the morning board.

Five causes, ranked by how often they are the one

When the policy was never actually written

This is the most common finding and the least popular one, because it is not a software fix. If 2 experienced people listening to the same call assign different tiers, no agent — human or otherwise — can be consistent, and every prompt change simply moves the errors around. The disagreement pile from the audit is the agenda: work through it with whoever sets policy and turn each contested call into a condition that can be asked about.

Write the conditions in the caller's language, not the trade's. 'Is the heating producing no warm air at all, or is it running but not keeping up?' is answerable by a homeowner in a cold kitchen. 'Is the system in hard lockout?' is not. The definitions themselves belong on their own page — see emergency, urgent, same-day and routine — but the version that ships is the operator's, dated and owned.

When the agent is confident and wrong

If the policy is clean and scorers agree, the failure is that nothing stops the agent from proceeding when it should not. Fluency is not calibration: an intake agent will state a tier as confidently on a call it half-understood as on one it read perfectly. The containment is a threshold with an action attached, and the action has to be a real branch in the flow, not a note in the log.

Two adjacent faults distort this measurement, so rule them out before concluding the threshold is wrong. If the caller is not matched to their history, an agent cannot know this is the third visit for the same fault — the mechanism is in why every repeat caller becomes a new customer record. And if the audit is drawn from bookings alone, it silently excludes the calls that never became bookings, which is the reconciliation problem behind an ad account counting more leads than the board.

Question order decides the tier before anyone notices

Read the mis-tiered transcripts and one pattern recurs: the agent offered a time before it established what was wrong. Once a slot is on the table the conversation is about the slot, the caller accepts what is offered, and the tier is backfilled to match. Establish the condition first, then offer only the windows that tier permits.

There is a real cost to this, and it should be measured rather than assumed. Every question before the first useful answer is time a caller may not give you, and abandonment in the opening seconds is its own failure — see callers hanging up before the greeting finishes. Keep the severity path to 2 or 3 questions and let the rest of the record be completed after the tier is fixed.

The decision tree

1. Do 2 scorers disagree on more than 1 in 10 calls? Fix the policy. Nothing downstream can be consistent until they agree.
2. Do they agree, and the agent still under-tiers? Check whether the severity questions were asked at all. If they were not, the fault is question order, not judgement.
3. Were the questions asked and answered, and the tier still wrong? Add the confidence gate and the escalation branch, then re-run the same audit in 14 days.
4. Do errors spike only when the board is full? The tier is being rewritten by capacity. Separate tier from slot and alert on emergencies with nowhere to go.
5. Do errors spike only overnight? Compare the after-hours path against the daytime one; you are probably auditing a system that is not the one taking those calls.

Re-run the audit on the same protocol after each change, and keep the transcripts. The point is not to reach zero mis-tiered bookings, which is unattainable when the underlying information is what a stranger says on a phone. It is to make the residual errors visible, cheap and owned. That measure-then-contain loop is how we scope intake work in AI agents and automation, and the rest of this cluster sits under intake and call capture, within our field service and trades work.

• triage
• voice agents
• intake
• diagnostics