The citation points at the wrong paragraph

A citation that lands on the wrong paragraph belongs to one of 2 families, and telling them apart takes 1 search. Copy the text the citation claims to support, and look for it verbatim in the original file. If it is there but at a different location, the claim was grounded and the pointer is broken: a deterministic pipeline bug that will reproduce every time. If it is not in that file at all, the pointer is the only thing the model produced — a plausible source attached after the fact to a claim it wrote itself.

Do the search against the original file, not against your extracted text. The extraction is a suspect in this investigation, and verifying a citation against the artefact that may have broken it is how a whole class of these goes undiagnosed for months.

The verification routine, run once per broken citation

1. Take the sentence in the answer that carries the citation, and the span the citation resolves to. Record both as they were rendered to the reader.
2. Open the original source file — the PDF, the wiki export, the contract as issued — and search it for a distinctive 8 to 12 word run from the cited span.
3. If the run is found at a different location, record the difference: how many characters, how many paragraphs, how many pages. The size and sign of that gap is the diagnosis.
4. If the run is not found, search instead for the claim the answer made. If the claim appears somewhere in the file, the citation resolution is wrong. If it appears nowhere, move to the grounding branch.
5. Repeat on 2 more citations from the same document, 1 near the start and 1 near the end. A gap that grows between them is offset drift; a gap that stays constant is pagination.
6. If the exact-string search itself is unreliable — because the span contains a part number, a clause label or a code that your search tool normalises differently — treat the result as inconclusive and read why identifiers come back empty from a search before drawing conclusions from it.

Three classes, and what each one licenses you to fix

Offset drift, and why page 2 is fine and page 40 is not

Extraction does not just read text out of a file. It normalises it. Runs of whitespace collapse to 1 space. Ligatures expand from 1 character to 2. Words hyphenated across a line break are rejoined, removing a hyphen and a newline. Headers and footers are stripped, taking 40 or 50 characters out of every page. Each of those edits changes the length of the text, and every character offset recorded after the edit is wrong by the accumulated total.

The durable fix is to stop treating offsets as something to recompute. Capture the coordinates once, at the moment of extraction, against the original file, and carry them through every downstream transform as opaque values that are never recalculated. Chunking, cleaning, deduplication and re-embedding all become safe, because none of them touches the pointer. Building that properly is its own piece of work, described in wiring citations a reader can check.

A printed page number is not a file page index

The second mechanical class is pagination, and it has a different signature: the offset within the page is correct and the page itself is wrong by a constant. Someone recorded the page index the extractor reported and rendered it as though it were the number printed on the sheet.

• Front matter. A cover, a contents page and 2 pages of roman-numbered preliminaries mean printed page 12 sits at file index 16, on every citation, forever.
• Appendices with their own numbering. A document whose annexes restart at A-1 has 2 pages that both answer to 'page 3', and a citation to either is ambiguous.
• Merged bundles. Several documents concatenated into 1 file makes every page number after the first document wrong, and the drift changes at each boundary rather than being constant.
• Two-up scans. A scan of a printed booklet puts 2 printed pages on 1 file page, so the file index advances at half the rate of the printed number.

Where a document has both, cite the printed number and the file index together. It looks redundant and it is the difference between a reader who finds the clause in 10 seconds and one who gives up.

A citation is a promise that a specific piece of text says a specific thing. A pointer that is 1 paragraph out breaks that promise as completely as a pointer that is invented — the reader who checks it finds the wrong words either way.

When the citation was invented rather than misplaced

If the claim exists in no retrieved passage, the model produced it and then attached the nearest plausible source. This is rarely random. It concentrates on claims that bridge 2 passages — where the evidence supports the first step and the last, and the middle step came from general knowledge — and the citation lands on whichever passage was topically closest to the finished sentence.

• Check whether the claim is a bridge. Take the answer apart into individual assertions and try to source each one. The unsourceable assertion is almost always the connective one.
• Check whether the passage set could have supported it at all. If the supporting passage was retrieved last week and is not retrieved now, the citation did not degrade — the evidence did, in the way a new source pushes good passages out describes.
• Require a quoted span, not a document reference. A claim that must reproduce the sentence it came from cannot be attached to a document that does not contain that sentence, which converts an invisible failure into a visible refusal.

Two variants worth naming separately

The first is a citation that resolves correctly and points at something the reader is not entitled to see. That is not a citation bug, it is a filter that was applied to search and not to rendering, and it is usually discovered on the day someone tightens the search side and watches results collapse — the sequence set out in recall collapsing after permission filtering. Resolve permissions once, and let the same predicate govern what is retrieved, what is cited and what is linked.

The second is citations that were fine and drifted with no release to blame. Extraction libraries are upgraded transitively, an OCR path changes, a re-index runs against a newer parser, and every offset moves together. When nothing in your own repository changed but the behaviour did, work through answers that changed and nobody shipped anything rather than the code.

What to fix, in the order that keeps the evidence

1. Pin the extraction toolchain and record its version alongside every document you extract. Without that field you cannot tell a drifted corpus from a broken one.
2. Add the round-trip assertion at ingest: for every chunk, resolve its stored coordinates back into the original file and confirm the retrieved text matches. Run it on 1 percent of chunks continuously and on 100 percent after any parser change.
3. Render both coordinates a human can use — printed page number and file index — plus the quoted span itself, so a reader can verify without opening anything.
4. Only then tighten the model side by requiring per-claim quoted spans. Doing this first hides mechanical misalignment behind a stricter prompt and makes the underlying bug harder to find.

The round-trip assertion is the single highest-value item on that list, and it is about 30 lines of code that will outlast every other decision in the pipeline — the sort of small, load-bearing component we build under MVP and product builds. The rest of this silo, from how passages get chosen to how they get used, sits in retrieval and grounding, inside the wider engineering library.

• citations
• grounding
• extraction
• debugging