The biggest gains in podcast transcription accuracy come before you transcribe. Record each voice close to its own microphone, keep music out from under speech, and export a clean file. Then use the largest model your device handles, tell it the names and terms it will hear, set the language instead of relying on auto-detect, and finish with an editing pass that targets names and numbers. That order matters. No model setting rescues a guest recorded from across a room.
Why podcasts are hard for speech recognition #
A studio audiobook is easy for a speech model. A podcast rarely is. The usual troublemakers:
- Music beds and stings under or between speech.
- Crosstalk, when hosts laugh or talk over each other.
- Remote guests recorded through compressed call audio or a laptop mic.
- Room echo from untreated spaces.
- Names and jargon the model has rarely seen: guests, brands, niche terms.
- Long silences or music-only stretches, where Whisper-family models sometimes invent or repeat text.
Each fix below targets one of these.
1. Record for the transcript, not just the listener #
- Close-mic every speaker. Distance is the single biggest factor. A mic a hand’s width from the mouth captures far more voice and far less room than one on the other side of a table. Our microphone guide covers types and placement.
- Record separate tracks if your setup allows. Later you can transcribe each track on its own, which removes crosstalk and gives you perfect speaker attribution.
- Have remote guests record locally on their own device as a backup, instead of relying on call audio. The local file is almost always cleaner.
- Treat the room a little. Soft furnishings, rugs and a closed door cut echo. You don’t need a studio.
2. Export a speech-friendly file #
If you can, transcribe a version without music and sound effects. Many editors let you mute the music track and export dialogue only. Keep the timing identical so the transcript still matches the published episode.
Level out the loudness so quiet speakers aren’t lost. Go easy on noise reduction. Moderate cleanup helps, but aggressive processing leaves watery artifacts that confuse speech models as much as the original noise did. There’s no benefit in exporting at a very high sample rate either: Whisper works on 16 kHz audio internally.
For more on noise, see how to minimize background noise for accurate transcription.
3. Use a bigger model for episodes that matter #
Model size is a straight trade between speed and accuracy. On a computer, Whisper comes in tiny, base, small, medium and large sizes, plus turbo, a faster version of the large model. Per OpenAI’s README, memory needs run from about 1 GB for tiny to about 10 GB for large. English-only versions of the smaller sizes are faster when your show is in English.
On a phone, Private Transcribe groups the same models into tiers. Professional (539 MB) is intended for interviews and recordings that matter. Best (574 MB) is Whisper large-v3-turbo and needs a recent phone with about 6 GB of RAM. Both run on the phone, and neither uploads anything.
A practical approach: transcribe a five-minute sample on two model sizes, compare the errors, and use the smallest model that’s good enough.
4. Give the model your vocabulary #
Whisper guesses spellings for names it hasn’t seen, and it guesses consistently wrong. Two ways to help:
- In apps, use custom vocabulary if it’s offered. Private Transcribe lets you list names and jargon so they come out spelled your way.
- On the command line, Whisper’s
--initial_promptoption takes a sentence of context, such as “Hosts Priya and Tom talk to Dr. Okonkwo about Kubernetes.” The model leans toward those spellings.
Keep the list to terms that actually appear in the episode. A huge list can nudge the model toward words nobody said.
5. Set the language instead of auto-detecting #
Auto-detect listens to the first seconds of audio and picks a language. A cold open with music, or a guest with a strong accent, can send it the wrong way. If your show is in one language, set it explicitly. For bilingual shows, use a multilingual model and consider transcribing segments separately.
6. Handle multiple speakers #
Plain Whisper doesn’t label speakers. If you recorded separate tracks, transcribe each one and merge by timestamp. If you only have the mix, use a tool with speaker diarization. Buzz, Vibe and MacWhisper Pro do it in their apps, and WhisperX adds it for scripted workflows. Expect to fix labels where people overlap.
7. Watch for invented text in silence and music #
Whisper-based models occasionally produce text during long silences or music: a repeated sentence, a stray “Thank you for watching.” Trim long dead air and music-only intros before transcribing. When you proofread, be suspicious of any line during the music.
8. Edit where the errors are #
You don’t need to reread every word with the same care. Errors cluster in predictable places:
| Where errors appear | What to do |
|---|---|
| Guest and brand names | Search the transcript for each name and fix them all at once |
| Numbers, dates, prices | Check every one against the audio |
| Crosstalk and laughter | Listen and rewrite, or mark it as crosstalk |
| Music and silence | Delete invented lines |
| Negatives (“can” vs “can’t”) | Verify any line you plan to quote |
How to measure whether it’s getting better #
Word error rate (WER) is the standard measure: substitutions plus deletions plus insertions, divided by the number of words in a correct reference transcript. You don’t need software for a rough check. Hand-correct a five-minute sample once, then compare each new setup against it. If a new mic or model doesn’t lower the count, it isn’t helping.
For why offline models react to noise the way they do, see why offline speech to text accuracy drops with noise. For tool choices, see best offline speech to text for podcast creators.
Frequently asked questions #
Does noise reduction improve transcription accuracy? #
A little noise reduction usually helps, especially on steady hum or hiss. Heavy processing can make things worse, because the artifacts it leaves are unfamiliar to the model. Fix noise at the recording stage where you can, and apply only light cleanup afterward.
Should I transcribe the raw recording or the finished episode? #
Transcribe the finished episode when you need timestamps that match what listeners hear, for captions and chapter markers. Transcribe the raw separate tracks when accurate speaker attribution matters more than timing.
Which model gives the best accuracy on a phone? #
Whisper large-v3-turbo, where your phone can run it. In Private Transcribe it’s the Best tier (574 MB) and needs about 6 GB of RAM. The Professional tier is a good fallback that runs on more phones.
Can custom vocabulary fix every misspelled name? #
Most, not all. Vocabulary strongly nudges the model toward your spellings, but unusual names in noisy passages can still slip through. A search-and-replace pass for each name catches the rest.