Diarisation: clinician, patient, and the relative who came along
In short
Speaker diarisation answers who was talking, separately from the step that answers what was said. It cuts audio into turns and labels each one, usually as anonymous speakers rather than named people. In a consult room those labels carry clinical meaning: a symptom attributed to the relative who drove the patient in is a different fact.
Key takeaways
- Diarisation answers who spoke; transcription answers what was said. A pipeline can be perfect at one and wrong at the other.
- Diarisation emits anonymous labels. Binding speaker A to the clinician is a second step, and most role errors enter there.
- A misattributed symptom is a content error: the note asserts something the patient never said, in fluent prose.
- Measure patient-to-third-party confusion on your own recordings. A pooled DER percentage hides exactly that failure.
Diarisation is the step that decides who was speaking. Transcription turns audio into words; diarisation cuts the same audio into turns and labels each — speaker A, speaker B, speaker C. The two fail independently, so a transcript can be word-perfect and still credit every sentence in it to the wrong person.
A note is a set of assertions about one patient, and the speaker label decides whose assertions they are. Section mapping, template filling and the clinician's review all sit downstream, which makes attribution the quietest dependency in ambient documentation and the clinical note.
Three jobs on one recording, and only one is transcription
- Speech recognition. Sound to words. It has no opinion about how many people are present and returns one paragraph from a three-way conversation.
- Diarisation. Segments audio into turns and clusters them by voice. Labels are arbitrary: speaker A is whoever the clustering saw first, not the clinician.
- Voice identification. Matches a voice to an enrolled template to say this is Dr Rao. Biometric, needs consent, and carries governance diarisation does not.
Clinical pipelines need the first two and avoid the third, leaving a step nobody budgets for: binding anonymous labels to roles. Bind by talk time and the mapping inverts the moment a distressed patient outtalks the clinician.
The relative's symptom, written into the patient's history
A daughter brings her mother in and mentions her own back pain and the tablets she takes for it. If those turns cluster with the patient's, the drafted history carries a complaint the patient does not have. Nothing reads oddly, which is why the error survives review.
It propagates: an unchallenged line today is inherited by the next draft, which is how attribution errors become text nobody wrote today. It is a different fault from a note attaching to the wrong record, traced in where the note-to-encounter binding broke, but both reach the vendor as one sentence: the note is wrong.
The four rooms where attribution reliably breaks
| Condition in the room | What the labelled transcript shows | What lands in the note |
|---|---|---|
| A third person who speaks under 2 minutes in total | Their turns absorb into the nearest large cluster | Their words are credited to whoever sat closer |
| Overlapping speech at the emotionally loaded moment | One turn holding both voices, given a single label | Question and answer merge into one assertion |
| The clinician reading measurements aloud while examining | Numbers transcribed right, attached to the dominant cluster | Objective findings read as things the patient reported |
| An interpreter relaying in the first person | Two labels carrying one account, or one carrying two | The history splits, or the interpreter's phrasing becomes the patient's |
Only the fourth row is really about audio. The first three are properties of the encounter, which is why ambient capture versus dictation for a busy outpatient list changes the risk profile: a dictated note has one speaker, so attribution error is zero.
Measure it on your own rooms, not on a public leaderboard
Quality is usually reported as diarisation error rate, one percentage pooling distinct mistakes: speech nobody was credited with, speech credited to a silent speaker, and speech credited to the wrong one. The pyannote.audio project publishes DER across public corpora — on the AMI meeting corpus recorded on individual headset microphones, it reports 17.0% for the open community-1 pipeline and 12.9% for the premium precision-2 one.
Those are meeting-room figures on a research corpus, useful only as a scale check. The number deciding whether your notes are safe is narrower — how often patient speech is credited to a third party, and how often the reverse happens.
- Sample 30 consented recordings weighted to hard cases: 10 with three or more people present, 5 with an interpreter, 5 where a relative does the talking.
- Mark only turns carrying clinical content — symptoms, medications, timelines, plans. Greetings and scheduling chatter inflate the error rate without touching the note.
- Score both directions separately: patient speech credited elsewhere, and other speech credited to the patient. The second is what puts a false assertion in the chart.
- Re-run the same 30 after every pipeline change, reporting per encounter, not per turn. One bad line makes one unsafe note.
Run that inside a clinical pilot staged so the result is a decision, not as a post-launch audit. It is the discipline any AI system needs inside live work, described in AI in logistics operations: pick the error that costs something and keep measuring it.
What to change when the audio cannot get better
- Bind roles explicitly. A fixed opening line from the clinician gives an anchored reference turn, better than inferring role from talk time.
- Pass the expected speaker count from the booking. A clustering step told to expect 3 speakers behaves differently from one left guessing.
- Keep third-party speech visible. Rendering it as reported by an accompanying person preserves the distinction the clinician must check.
- Let the pipeline abstain. A turn below the confidence threshold should surface unassigned and go to review.
Attribution is a build decision, not a feature you either get or do not. It is the pipeline work we do as AI agents and automation inside the healthcare and medtech practice: name the clinically meaningful error, instrument it, and make uncertainty cost a review.
Frequently asked questions
Short answers to the follow-ups this page tends to raise.
What is speaker diarisation in a medical transcription pipeline?
It splits a recording into speaker turns and labels each one, so the transcript records who spoke rather than only what was said. Diarisation names nobody: it emits anonymous labels such as speaker A and speaker B, and mapping those to clinician and patient is a separate step.
How is diarisation different from voice identification?
Diarisation groups turns by voice without knowing whose voice it is; voice identification matches a voice to a stored template to name a person. The second needs enrolment, consent and biometric-data handling the first does not, which is why clinical builds pair diarisation with explicit role binding instead.
What happens when a relative or interpreter speaks during the consultation?
Their speech has to be labelled as a third party or it contaminates the patient's history. If an accompanying person mentions their own symptoms and those turns cluster with the patient's, the draft asserts something that was never true. Keep third-party statements marked as reported by someone else.
- ambient documentation
- diarisation
- clinical notes
- speech pipelines
The work behind this page
Builds from our portfolio that this page draws on.
Read next
- Copy-forward: how a chart fills with text nobody wrote todayCopy-forward is the habit of carrying a previous note into today's entry. It is fast, it is defensible, and it slowly makes a chart unreadable to the next clinician.definition
- S, O, A and P: what belongs in each section, and which ones a recording can fillFour sections with four different sources of truth. Subjective is spoken, Objective is half instrument data, and Assessment and Plan are judgement a microphone never hears.definition
- The attestation statement: what a clinician is signing when the draft was machine-writtenA sentence at the foot of a note proves nothing. Attestation is a record: who signed, when, which draft version they saw, what they changed, and what the signature covers.definition
- The HPI's elements, and which of them the conversation actually containsOnset and location are nearly always said out loud. Severity, radiation and timing frequently are not, and no model can transcribe a question nobody asked.definition
Working on something in this space?
Tell us where you are in a sentence or two. We'll tell you honestly whether we're the right team, and what a sensible first slice of the work looks like.
Start the conversation