The anomaly score is a ranking; the threshold is a business decision

An anomaly score is an ordering. It says this part looks further from normal than that one, and nothing more. It has no physical units, it is not a probability, it does not encode how large or serious the defect is, and it cannot be compared across models or builds. The threshold is where somebody cuts that ordering into accept and reject.

So the number on screen — 0.62, say — is meaningless alone. It means something only against the distribution this model produced, on this variant, under this lighting. Move any of those and the same 0.62 is a different decision.

Four things a score is routinely mistaken for

• A probability. 0.90 does not mean a 90% chance of a defect. Most scoring functions are unnormalised distances, and those squashed into 0–1 are squashed for display.
• A defect size. A 2 mm scratch and a 15 mm faint stain can land within 0.01 of each other; the score measures deviation, not extent.
• A severity. The ordering does not know your dispositions, so a critical void can rank below a cosmetic mark that is merely more unusual.
• A portable number. Retraining or a different normalisation rewrites the scale, and the old cut is no longer the old policy.

The ordering, made concrete

Below is an hour of borderline parts from a coated-bracket cell, ranked, with the cut at 0.60. Read the rightmost column before anything else.

Raising the cut to 0.70 removes both overkills. It also loses rank 5, a critical coating void, because the score does not rank by severity. That row is the whole argument: no threshold protects a critical class without also being loose enough to admit whatever cosmetic oddity scores higher. The trade is between counted consequences, and they are counted in good parts pulled off the belt per shift.

The threshold is configuration, not a slider

1. Score a fresh sample of known parts, drawn from current production rather than the original training pull.
2. Build the curve. For each candidate cut, count overkill and escapes per class, in parts per shift rather than rates.
3. Quality picks the cut against the class dispositions. Engineering advises on what the ordering can separate; it does not choose.
4. Record it: value, the scored sample, the model build, the variants it applies to, the approver, the effective date.
5. Set the review triggers — retrain, material lot change, optics service, a new variant — plus a scheduled re-check independent of all of them.

If a threshold can be changed by someone with a mouse and no record, then the accept criterion for the product is whatever that person believed on a Tuesday afternoon.

When the distribution moves under a fixed cut

A stable threshold does not mean a stable decision: the score distribution can move while the model stays byte-identical. A rejection rate that climbs after dark is the classic instance, and it is a lighting-geometry fault rather than a model fault — the confirming check is in the model finds the scratch on day shift and misses it after dark. The same problem shows up in space rather than time when a second site goes live, which is why a threshold is a per-cell setting: see plant one live, then plant two.

Two boundaries are worth stating. A check with physical units — a dimension, a presence, a code read — needs a tolerance rather than a threshold and belongs with classical tools, as argued in the measurement your caliper agrees with. And a supervised classifier emits per-class confidences rather than this kind of score, governed differently — the choice is in when you have forty defect images, and when you have four thousand.

Governing a live numeric setting with an owner, a log and a trigger list is ordinary discipline for any unattended system — the pattern in AI in logistics operations, and built into how we scope automation work. The surrounding decisions sit under visual inspection and defect detection and the wider industrial vision practice.

• anomaly detection
• thresholds
• inspection governance
• model operations