The method is correct and the arithmetic is wrong

The explanation is right and the number is wrong because the two came from different processes, and only one of them is arithmetic. The prose is the thing a language model is genuinely good at. The digits in the middle of it are produced by the same mechanism as the prose — one token continuing the last — and that mechanism has no place value, no carry, and no memory of what the previous line evaluated to.

So the first instinct, which is to reach for a larger model, is usually the wrong purchase. A larger model shifts the error rate; it does not change what the operation is. The repair is to stop asking the model to compute at all, and then to check that the number on screen is the one the calculator produced. But do the classification first, because the same symptom is produced by a fault with a completely different fix.

Re-execute the numbers in fifty transcripts before touching anything

This is an afternoon of work and it decides everything that follows. You are not judging whether answers were helpful. You are extracting every numeric claim the tutor made and recomputing it independently.

1. Sample 50 transcripts containing at least one numeric step, stratified by topic and by the unit the learner was working in. Do not sample from complaints — complaints over-represent the visible failures and hide the ones nobody noticed.
2. Extract each numeric claim as an expression with its inputs: what was being computed, from which values, and what the tutor said the result was. One row per claim, not one row per transcript.
3. Recompute every row independently with exact arithmetic — a computer algebra system for symbolic work, decimal rather than binary floating point for anything involving money or fixed decimal places.
4. Classify each mismatch as method, computation, presentation or transcription, using the table below. Two markers on the first 20 rows, to check the classification itself is stable.
5. Record the position of the first wrong number in each chain. A distribution clustered at step one and a distribution that rises with step index are different problems.
6. Report the rate per unit of instruction, not per model. If one topic carries most of the failures, the cause is probably not arithmetic at all.

Method errors are not this page's problem and cannot be fixed by a calculator. If the tutor is deriving from something the class has not been taught, or from a different edition of the material, the repair is corpus-side and belongs with grounding a tutor in the module you actually teach. If it is numerically correct but methodologically foreign, that is a separate diagnosis, in the tutor is right and using notation the course never taught.

Five ways a correct method lands on a wrong number

In rough order of how much of the total they account for. Each has a distinct signature in the sample you just built, which is the point of building it.

Give the arithmetic to something that does arithmetic

The model's job is to decide what to compute, explain why, and read the result back in the language of the course. The computing itself goes to a tool: a computer algebra system for symbolic manipulation and exact fractions, a decimal library for anything with fixed decimal places, a unit-aware evaluator where quantities carry dimensions.

• Exact where the course is exact. A symbolic engine such as SymPy keeps one-third as a fraction rather than as 0.3333, which matters because the course's answer key does too.
• Decimal where money or fixed places are involved. Python's decimal documentation states that in decimal floating point 0.1 + 0.1 + 0.1 - 0.3 is exactly equal to zero, while in binary floating point the result is 5.5511151231257827e-017 — near zero, but enough to break an equality check the tutor then reports to a student.
• One expression per step, evaluated as it is produced. Batch evaluation at the end tells you the answer is wrong without telling you which line to reteach.
• The expression stored alongside the rendered text. When a teacher disputes an answer next term, the arguable artefact is the expression, not a paragraph.
• A deterministic path for anything the item author already knows. If the item ships with a worked solution, the tutor should be comparing against it rather than rederiving it, which is the same argument behind a hint ladder authored as content.

Check the sentence against the tool before the student sees it

Calling a tool is not the same as using its result. The common half-built version calls a calculator, gets 4.75, and then writes a sentence containing 4.7 because the sentence was generated separately. Verification is a distinct step: parse the numbers out of the drafted response, compare each against the evaluated value, and refuse to render on mismatch.

There is no such thing as a small arithmetic slip in a tutor. The student cannot tell which part of the answer to distrust, so they stop trusting the method as well.

A mismatch should block, not annotate. A warning banner beside a wrong number is worse than a retry, because it hands a judgement call to the person least equipped to make it. Make the mismatch rate a standing number, and put the numeric behaviours into the graded set described in an evaluation set that measures teaching rather than checking it once at launch.

The build is small and unglamorous: an expression format, an evaluator, a comparison step, a refusal path and a log of every mismatch with its transcript. That is the shape of work we scope as internal tools and operations — narrow, testable, and owned by somebody after launch.

What a solver behind the tutor will not fix

It will not fix a wrong answer that was never arithmetic. A tutor solving the wrong equation computes it perfectly, and the sample you built is the only thing that tells the two apart. It also will not fix a wrong answer the learner caused, by mistyping the problem — which argues for reading the problem back before solving it.

It also makes one existing problem worse. A tutor that computes reliably is a more attractive thing to extract answers from, which is the failure diagnosed in students pressing hint until the answer falls out. And once computation is free and correct for every learner, any assessment whose difficulty was the computation has stopped measuring anything — a design question settled in watch them, or change the assessment, not a guardrail you can bolt on afterwards.

Everything here sits inside AI tutoring and learning support, part of our education and edtech work.

• tutoring
• tool use
• evaluation
• maths