STT model recommendations

Models
1

Hi all, I am looking for a decent STT model that can transcribe audio from a podcast. I’ve listed some details about the content of the audio file, in case that helps:

  • Straightforward, English-speaking, two-person-conversation audio that is approximately 1 hour in length.
  • The conversation between the podcast speakers is casual, so it does contain typical filled pauses, discourse markers, disfluencies, etc.
  • Timestamps aren’t necessary.
  • Identifying the active speaker and what they have (generally) said is necessary, but the precision with which the transcript reports this is not.
    • For example, suppose Jack and Jill are the speakers. Jill is explaining something, and while she is speaking, Jack interjects with a filler line like “Mmm, right, that makes sen-”, only for Jack to pause so that Jill’s initial speaking line can continue uninterrupted. If the STT were to transcribe this precisely, it would need to very suddenly realise that Jill is no longer speaking, create a new speaker line for Jack, only to return back to Jill again. I can see lots of issues with that in a more casual and non-turn-based conversation, such as inaccuracies of who said what exactly, and in distinguishing a speaker’s intent with what is literally said.
2

WhisperX is the fit for what you’re describing, it bolts word-level alignment plus diarization (via Pyannote) onto Whisper, and critically it does forced-alignment after transcription rather than transcribing interruptions in strict real-time order, so those messy overlapping-speaker moments (“Mmm, right, that makes sen-”) get attributed correctly without needing frame-perfect turn-taking detection. Since you don’t need precise timestamps, you can skip the alignment step and just use WhisperX’s diarization pipeline on top of Whisper large-v3 or large-v3-turbo (turbo is 4x faster with barely worse accuracy, worth it for a 1hr file).

Plain Whisper alone (no WhisperX) has no built-in speaker separation, so you’d get a wall of text with no idea who said what, that’s the actual gap you’d hit for a two-person podcast without adding diarization on top. Pyannote 3.1 is the diarization backbone WhisperX uses and is currently the best-supported open option for exactly this “good enough, not perfect” speaker separation on casual conversational audio.