Speaker labels are the tags in a transcript that mark who is speaking, usually as "Speaker 1," "Speaker 2," or named participants. The technical name for producing them automatically is diarization: splitting a recording by speaker so each line of text is attributed to the person who said it. If you have ever read an interview transcript where every turn is labeled, you have seen diarization at work.
This guide covers what speaker labels are, how a tool decides which voice is which, the specific ways it fails and how to spot each one, and what to do about it before and after you record.
What speaker labels (diarization) actually do
A raw speech-to-text transcript is one continuous block of text. It tells you what was said but not who said it. Diarization adds the missing dimension: it segments the audio into speaker turns and attaches a label to each one, so a two-person interview reads as a back-and-forth instead of an undifferentiated wall of text.
Two things often get confused:
- Diarization answers "who spoke when" by grouping similar-sounding voices. It does not know anyone's name. The output is "Speaker 1 / Speaker 2," which you then rename.
- Speaker identification matches a voice to a known identity, a specific named person. Most transcription tools do diarization, not identification. Most of the naming is a human step you do in the editor.
How AI diarization works, step by step
The label on each line is the end of a short pipeline. Each stage mostly builds on the decisions of the one before it, which is why the errors in the next section look the way they do.
- Speech detection and segmentation: the audio is first split into stretches that contain speech, then cut again at points where the voice appears to change. Everything downstream works on these chunks, so a cut in the wrong place (mid-word, or in the middle of someone's sentence) is baked in from here.
- Embedding: each chunk is converted into a numeric "voiceprint," a list of numbers that roughly captures the character of the voice, such as its pitch and timbre. Short chunks produce noisy voiceprints, because there is little audio to describe.
- Clustering: chunks with similar voiceprints are grouped, and each group becomes one speaker label. This is where the speaker count matters. If you tell the tool how many speakers to expect, it is limited to that many groups. If you leave it on auto, it has to guess the number from how the voiceprints bunch together.
- Alignment: the words from speech-to-text carry timestamps. Each word is matched to whichever speaker segment covers that moment, so every sentence inherits a label.
Two consequences follow from this order. First, the clustering decides the speaker boundaries and the words are fitted to them afterwards, so a diarization mistake shows up as a whole block of text under the wrong name rather than a single wrong word. Fixing it is a relabeling job, not a retyping job. Second, transcription and diarization are usually separate models working on the same audio. A transcript can have accurate words and wrong labels, or the reverse, so check each on its own. Bad audio tends to hurt both.
Where diarization gets things wrong, and how to spot each failure
Speaker labels are an estimate, not ground truth. Each failure mode below leaves a recognizable fingerprint in the transcript, which tells you where to look when you review.
| What went wrong | Why it happens | How it shows up |
|---|---|---|
| Two people merged into one label | Similar voices (same gender, age, accent), or one speaker who talks very little, so their voiceprints fall inside the other cluster | A single label that answers its own questions, or a quiet participant who never appears |
| One person split into two labels | Tone, volume or distance from the mic changes mid-recording, so the second half looks like a new voice | More speakers than people in the room; the same person's name needed on two labels |
| Turn boundary off by a few words | The cut between speakers lands slightly early or late, and alignment assigns the edge words to the wrong side | The first or last few words of a turn sit under the other speaker |
| Short interjection mislabeled | "Right," "exactly," "mm-hm" give too little audio to build a reliable voiceprint | Backchannel words attached to whoever was talking, or to a one-off extra label |
| Overlapping speech assigned to one voice | Two voices in the same moment cannot both own one chunk | One speaker's sentence missing or garbled where the other talked over them |
| Everything on one mic sounds alike | A single shared microphone in a noisy room blurs the voiceprints | Labels that flip back and forth every few lines |
A fast way to review labels
You do not need to read the whole transcript to check the labels. Three quick passes catch most problems:
- Count the labels against the people you know were there. More labels than people means a split. Fewer means a merge.
- Scan the question-and-answer joins. In an interview, a question and its answer should almost always carry different labels. A question and its answer sitting inside the same labeled turn is the quickest tell for a merge.
- Check the first minute. It is where people usually introduce themselves, so a wrong label is easiest to spot and name there.
What you can change before you hit record
Most label quality is decided by the recording, not the software. Four things are usually within your control:
- Give each speaker their own microphone where you can. Separate close-mic audio is the single biggest improvement. One laptop mic in the middle of a table is the hardest case. For remote calls, record so that each participant has their own track if your setup allows it.
- Ask people to name themselves in the first minute. "Priya here" at the top gives you an anchor for renaming labels later, and it takes seconds.
- Count the speakers before you upload and enter that number rather than leaving it on auto. A known count constrains the clustering. If people may drop in or out, auto is the safer setting, and you correct splits afterwards.
- Discourage talking over each other in the first few minutes especially, because overlap at the start is when people are still being introduced, and it is the hardest stretch to untangle by ear later.
What to do when the labels are still wrong
Fix by type, and fix the cheap ones first:
- Splits (too many labels): rename both labels to the same person. Every turn under either label updates, so this is one action, not many.
- Merges (too few labels): these cost more, because the editor has to separate turns that the tool believed belonged together. Fix the turns that matter for your use, such as the quotes you will publish or the decisions you will record, instead of the whole file. If merges are frequent, re-run with the speaker count set manually.
- Edge-of-turn and interjection errors: fix only where attribution changes the meaning. A mislabeled "right" rarely matters. A mislabeled "I'll approve that" does.
When the stakes are high (legal, medical, research), review the labels against the audio for every quoted passage rather than trusting them blind.
When do you actually need speaker labels?
Not every transcript needs them. Use this as a guide:
| Use case | Speaker labels needed? | Why |
|---|---|---|
| Interview (1-on-1 or panel) | Yes | Attribution is the whole point of the transcript |
| Meeting minutes | Yes | Decisions and action items must be tied to who said them |
| Focus group / user research | Yes | Analysis depends on separating participant voices |
| Podcast show notes | Often | Host vs guest turns improve readability and SEO |
| Legal / deposition | Yes | The record must attribute every statement |
| Lecture or single-speaker talk | No | One speaker means labels add nothing |
| Subtitles / captions for video | No | Captions follow the on-screen speaker, not a label |
If your recording has one voice, skip diarization. If it has two or more and attribution matters, turn it on.
How to get a speaker-labeled transcript with Subanana
In Subanana's transcript mode, diarization is built in. The workflow:
- Upload your audio or video file, or paste a public link. No separate download is needed.
- Set the source language. Subanana supports 95+ languages, and transcript mode takes one source language per file.
- Set the number of speakers: choose a known count for cleaner results, or leave it on auto if you are unsure.
- Generate the transcript. Speaker turns come back labeled as Speaker 1, Speaker 2, and so on, and punctuation and paragraph breaks are applied automatically. That formatting always runs on a transcript; there is no setting to switch it on or off.
- In the editor, rename each speaker to the real participant and fix any turns that were attributed to the wrong person.
- Export to your preferred format.
If you want a summary as well as the labeled transcript, choose meeting mode when you create the project instead. Meeting mode returns the tidied transcript plus a summary that pulls out key discussion points, decisions, and action items. Subanana does not set due dates on those items and does not push them to a task tracker, so if you run on Asana or Linear, moving them across is still a manual step.
For recurring use cases, the same diarization powers audio transcription and podcast transcripts, so interviews, panels, and multi-host shows all come back attributed.
Frequently asked questions
Can AI transcribe multiple speakers?
Yes. AI transcription tools handle multi-speaker audio by pairing transcription with diarization. The audio is split by speaker so each line of text is attributed to whoever said it, as covered above in "What speaker labels (diarization) actually do."
Results depend heavily on the recording, so a few things help before you hit record: give each speaker their own microphone where you can, ask people to name themselves in the first minute, and enter a known speaker count instead of leaving it on auto.
In Subanana's transcript mode, this is the number of speakers setting: choose a known count for cleaner results, or leave it on auto if you are unsure, alongside the source language you set for the file.
What is the difference between diarization and transcription?
Transcription turns speech into text: what was said. Diarization adds who said it by splitting the audio into speaker turns. Most tools run both together, but they are separate steps. You can transcribe without speaker labels, and diarization is the layer that adds them.
Should I set the number of speakers manually or use auto?
If you know exactly how many people are in the recording, set that number. It constrains the clustering and usually produces cleaner labels. Use auto when the count is uncertain (an open meeting, a call with drop-ins), then correct any merges or splits in the editor.
Why does my transcript show more speakers than people in the room?
This is over-segmentation, and it is one of the most common diarization complaints. A single speaker's tone or volume shifts enough mid-recording that the clustering step treats the second half as a different voiceprint, or a short interjection does not carry enough audio to cluster correctly and gets split off as its own label. Entering a known speaker count instead of auto reduces this, because it limits how many clusters the system is allowed to create. After generation, rename both split labels to the same person in the editor and every turn under either label updates to match.
Why are two people labeled as the same speaker?
This is the opposite error, a merge. It is most likely when the voices are similar, when one person speaks very little, or when everyone shares a single microphone. A known speaker count helps here too, since it usually pushes the system to look for the second voice instead of folding it into the first. If it still merges, fix the turns that matter for your use and separate microphones for the next recording.
Why is a speaker's name wrong on the first or last words of a turn?
Words are matched to speaker segments by timestamp, and the boundary between two segments can land a few words early or late. The edge words then sit under the neighboring speaker. It is a boundary error, not a mix-up of who the speaker was, and in the editor it is a one-turn fix.
Does diarization work on multilingual recordings?
The speaker-splitting part does. Diarization separates voices by acoustic characteristics, not by language, so a conversation that switches between languages still gets split by speaker.
What language-switching affects is the transcription underneath. In Subanana's transcript mode you pick one source language for the file, so a recording that moves between two languages is transcribed against the one you chose. If the recording splits cleanly (the first half in English, the second half in Cantonese), the usual answer is to upload each half as its own file with its own source language. Automatic detection of mid-sentence language switching is a feature of Subanana's live caption mode for events, not of file-based transcript mode.
Can I rename "Speaker 1" to a real name?
Yes, and it is mostly a manual step. Diarization groups voices but does not know identities, so you assign the real names once in the editor, and the label updates across every turn for that speaker.