The AI add-on billed for more resolutions than your team saw tickets closed
In short
Export the vendor's billed events for one period, join them to your helpdesk's own conversation records on conversation id, and classify every unmatched row before anyone argues about the total. Two different problems produce the same gap — a counting definition nobody read, and real traffic nobody was measuring — and only the second one is fixable by changing your own configuration.
Key takeaways
- Reconcile on conversation ids, never on totals. A total cannot tell a definition dispute from unmeasured traffic.
- Bot-only conversations that never became tickets are the single most common source of the gap.
- Duplicate billed events sharing 1 conversation id are a contract question; unmatched rows are a configuration question.
- Set both exports to the same period boundary and the same timezone, or hours of traffic move between periods.
- The reconciliation ends in exactly 4 places: change a trigger, exclude a surface, renegotiate the rule, or accept real demand.
Do not argue about the total. Export the vendor's billed events for one closed period, export your helpdesk's own conversation records for the identical window, join them on conversation id, and sort every row into 4 classes. The classification is the diagnosis; the total is only the symptom that sent you looking.
There are 2 families of cause and they need opposite responses. A counting-definition disagreement means the vendor billed events you would not have called resolutions, and the remedy lives in the contract. Traffic you were not measuring means real conversations happened that never reached your queue, and the remedy is your own configuration. Only the second is fixable this week.
The check: one period, two exports, one join key
- Pick a closed billing period, not the current one. A period still accruing changes underneath you while you work.
- Request the vendor's per-event export for that period with, at minimum, conversation id, channel, first and last message timestamps, and the trigger that fired. If that export does not exist, stop and read the contract — this is the term that makes an audit possible at all.
- Export your own conversations for the same window: id, channel, created and closed timestamps, requester, assignee, status, and whether any human message exists on the thread.
- Fix the boundary. Agree the exact start and end instants and the timezone both sides use, then re-cut both files to match. This alone resolves a surprising share of small discrepancies.
- Join on conversation id. If the ids do not correspond, join on channel plus requester plus first-message timestamp within 60 seconds, and treat everything that follows as approximate.
- Count 4 buckets before you read anything: matched and agreed, matched and disputed, unmatched with a human requester, unmatched with a machine requester.
- Write the bucket totals down before you form a theory. A theory formed first will find evidence for itself in a large export.
Classify the rows, do not total them
Every unmatched row belongs to one of 4 classes, and the class determines who can fix it. This table is the artefact to take into the vendor conversation.
| Class | What it looks like in the join | What it proves | Whose problem |
|---|---|---|---|
| Matched, agreed | Billed event, matching conversation, you also closed it | Nothing is wrong with this row | Nobody. Set it aside |
| Matched, disputed | Conversation exists, but your record shows a human replied, or the same id bills twice | A counting-definition disagreement | Contract, not configuration |
| Unmatched, human | No conversation on your side; the transcript reads like a real customer | Traffic you were not measuring | Yours, and fixable this week |
| Unmatched, machine | No conversation on your side; the transcript is a test, a monitor, an autoresponder or a crawler | Noise reaching an entry point nobody guards | Filters and credentials |
The causes, ordered by how cheaply you can rule each one out
Work down this list rather than starting with the theory you find most annoying. The first 3 checks take an afternoon between them and account for most gaps.
| Cause | Signature | Cheapest check | What it changes |
|---|---|---|---|
| Bot-only conversations that never became tickets | Large block of unmatched rows, transcripts read like customers | Read the add-on's ticket-creation rule: many create a ticket only on escalation | Your configuration |
| The same conversation billed twice | Duplicate conversation ids, timestamps days apart | Group billed events by id and count groups above 1 | The contract |
| One customer arriving on 2 channels | 2 conversations, same requester, minutes apart, different channel | Group unmatched rows by requester inside a 60-minute window | Configuration, then the contract |
| Automated senders and monitoring | Sender domains that never belong to customers | Group unmatched rows by sender domain and read the top 20 | Filters |
| Sandbox or staging using production credentials | Bursts on weekdays in office hours, repeating identical text | Search transcripts for your own test strings and fixture names | Credentials and environments |
| A widget left enabled where nobody expected it | Originating page or referrer is a marketing or documentation URL | Group unmatched rows by originating page | Your configuration |
| Out-of-hours arrivals from regions you do not staff | Arrival timestamps cluster outside your working hours | Histogram billed events by hour in a single timezone | Nothing. This is real demand |
The traffic you did not know you had
The unmatched-human bucket is the interesting one, because it is evidence rather than a dispute. Those conversations happened. Nobody in support saw them, no ticket was created, no first-reply clock ran, and none of it appears in the reporting your leadership reads.
- A widget on pages nobody thinks of as support. Pricing pages, documentation and status pages generate pre-sales and self-serve questions all day, and they are genuine demand you were not counting.
- Hours you do not staff. A queue that only ever reported business-hours volume will show a different shape once something answers at 03:00, and the difference is not new demand — it is demand that used to go unanswered.
- Questions that never became tickets by design. Password resets, plan questions and invoice copies get answered and closed, which is what you bought. They still bill.
- Machine traffic that has always been there. Bounce notices and monitoring pings were being silently discarded by a spam rule your helpdesk applied — the same population described in what lands in the suspended queue — and the new entry point does not apply that rule.
A number nobody reconciles drifts until it is decorative. That is the same failure mode as a dashboard where every account is green and one just cancelled: the figure was never joined to anything that could contradict it, so nobody noticed when it stopped describing reality.
The decision: 4 endings, and how to tell which one you are in
- Change the trigger configuration. If the unmatched-human bucket is dominated by conversations your own settings routed to the assistant, tighten what it picks up: which pages, which intents, which hours. This is the ending most reconciliations reach.
- Exclude a surface or a channel. If the unmatched-machine bucket is large, the fix is a filter, a separate credential for test environments, and removing the widget from pages that never needed it. None of this requires the vendor.
- Renegotiate the counting rule. If the matched-disputed bucket is large — reopens billing twice, duplicates across channels, conversations a human finished — you have a definition problem, and the definition is set out in what your vendor counts as a resolution. Bring the row-level evidence, not the total.
- Accept the number. If the rows are real customers asking real questions in hours you never covered, the invoice is correct and your previous volume reporting was wrong. That is uncomfortable and it is also the most valuable finding on the list.
A total cannot tell you whether you have a contract problem or a configuration problem. Only the unmatched rows can, and just one of those is yours to fix.
Making the next period reconcile itself
Reconciling once by hand is a week you do not want to repeat. Three changes make the next period cheap. Write the vendor's per-event export into your own warehouse on a schedule, so the join is a query rather than a project. Give every environment its own credential, so test traffic is separable by construction rather than by grepping transcripts. And define, in writing, which intents the vendor's assistant is allowed to pick up at all — the boundary drawn in letting the vendor answer the documented half.
The export right itself is something to secure before signature, alongside the other terms in what to get in writing before you sign. If you are still choosing, the same instrumentation makes a comparison possible at all: fix the queue, the intent set and the measurement first, as in running both on the same queue for a quarter.
The reconciliation job, the warehouse table and the small dashboard that flags a divergence over a threshold are a few days of work, not a platform — the kind of narrow internal system described in MVP and product builds. The rest of this silo sits under build, buy or extend what you already pay for, inside our SaaS and customer support work.
Frequently asked questions
Short answers to the follow-ups this page tends to raise.
Why does the vendor's resolution count exceed our closed-ticket count?
Most often because conversations the assistant handled never became tickets at all, so they were never in your closed count to begin with. Many add-ons create a ticket only when a conversation escalates to a human, which means a successful automated answer is invisible in helpdesk reporting and fully visible on the invoice. Check the ticket-creation rule before you check anything else.
Should a reopened conversation be billed twice?
That depends entirely on the agreement, and all 3 positions exist in the market: one charge, two charges, or a credit for the second. What matters is that you can detect it — duplicate conversation ids in the billed export, days apart — and quantify it before raising it. A vendor will discuss a list of 40 duplicate ids; a vendor will not discuss a feeling that the invoice looks high.
How do we exclude our own test traffic from the count?
Give every non-production environment its own credential or API key and ask the vendor to exclude it at source. Filtering by email domain or by a naming convention in transcripts is second best, because both fail silently the moment someone runs a test with a realistic-looking address. Whichever route you take, get the exclusion written into the agreement rather than configured once and forgotten.
Can we dispute a usage invoice after we have paid it?
Practically, your leverage drops sharply after payment and again at renewal, so raise it inside whatever dispute window the agreement specifies. Bring the joined dataset — row counts by class, duplicate ids, and a sample of transcripts — rather than a total. A disagreement expressed as evidence is a conversation about the counting rule; a disagreement expressed as a number is a conversation about goodwill.
- usage reconciliation
- vendor billing
- support AI
- helpdesk data
The work behind this page
Builds from our portfolio that this page draws on.
Read next
- First-reply SLA timer: start, pause and breach are three different eventsA first-reply SLA is three settings, not one, and most breach reports that look like a staffing problem are one of the three configured differently from how the team believes.definition
- Intent taxonomy: the label set a support router can act onAn intent is not a description of a ticket. It is the instruction a router acts on, which means a label no queue, macro or automation consumes is not a label at all.definition
Working on something in this space?
Tell us where you are in a sentence or two. We'll tell you honestly whether we're the right team, and what a sensible first slice of the work looks like.
Start the conversation