LesskeysWhich Whisper model should you run on a Mac?
Run Large V3 Turbo Q5. In our test it tied for the best accuracy of six Whisper models (within 13 words of the best, out of 6,086), and it is the smallest download of that group at 574 MB, running 12 times real time on an Apple M1 Max.
macOS 13+ on Apple Silicon · No subscription
Last reviewed
This guide is for Mac. For Windows, see Running Whisper on Windows.
The answer
Get Large V3 Turbo Q5
It tied for the best accuracy of six Whisper models, and it is the smallest download of the tied group.
- 4 tiedmodels, equal on accuracy
- 574 MBdownload (547 MB in the app)
- 12times real time
Within 13 words of the best, out of 6,086. An hour of audio takes roughly five minutes on an Apple M1 Max. Sizes here are measured file sizes in decimal units; the app labels the same file 547 MB. Outside English a few languages differ: Whisper accuracy in 29 languages.
Tied
Bigger costs more and returned nothing
The same accuracy to within 13 words costs 5.4 times the download and runs 1.6 times slower.
The largest tied model is Large V3 at 3.1 GB. Going smaller costs real accuracy: Small makes 75 more errors than the best, and Tiny makes 3 times as many.
Real differences
Where the differences are real
Tiny makes 3 times as many errors as the best: 418 against 141.
That is a different kind of transcript, not a slightly worse one. It still suits a rough, searchable record made in a hurry from a small download, and it is the wrong choice when the transcript is the record.
The models below the tied group
- Small: 3.55%, 75 more errors than the best; 35 times real time against 12 for Large V3 Turbo Q5, at 488 MB
- Tiny: 6.87%, 277 more errors than the best; 89 times real time against 12 for Large V3 Turbo Q5, at 78 MB
Smaller file
Does a smaller file cost accuracy?
Not that we could measure.
- Large V3 Turbo Q5 against Large V3 Turbo: 574 MB against 1.6 GB (2.8× smaller), and 4 more errors out of 6,086 words (154 against 150)
- Large V3 Q5 against Large V3: 1.1 GB against 3.1 GB (2.9× smaller), and 2 fewer errors out of 6,086 words (141 against 143)
It is a reason to test quantisation on your own audio, not a general law.
Not tested
What this does not tell you
LibriSpeech test-clean is the easy case: read audiobook speech.
- No meetings or phone calls
- No crosstalk or noise
- No other Mac: just the Apple M1 Max
- No speaker labels
Real meetings are harder, and every number would be worse on them: a floor, not a promise about your next stand-up. A model can also stop early on a long file in some languages and still report success: when Whisper stops early.
The data
The full table and how to reproduce it
Errors, speed and download for every model, plus the method, one click away.
Full results table
Six of the Whisper models Lesskeys offers ran over the same 300 utterances (37 minutes of audio) from LibriSpeech test-clean, the standard public benchmark set, which ships its own human transcripts. We scored word error rate against those transcripts, timed each run and recorded the download size. Lower error is better; a bar is drawn in proportion to the largest value in its column.
Six Whisper models on 6,086 words of LibriSpeech test-clean (Apple M1 Max)
| Model | Word error rate | Errors | Speed | Download |
|---|---|---|---|---|
| Large V3 Q5 ● | 2.32% | 141 / 6086 words | 7 times real time | 1.1 GB |
| Large V3 ● | 2.35% | 143 / 6086 words | 7 times real time | 3.1 GB |
| Large V3 Turbo ● | 2.46% | 150 / 6086 words | 12 times real time | 1.6 GB |
| Large V3 Turbo Q5 ● | 2.53% | 154 / 6086 words | 12 times real time | 574 MB |
| Small | 3.55% | 216 / 6086 words | 35 times real time | 488 MB |
| Tiny | 6.87% | 418 / 6086 words | 89 times real time | 78 MB |
Measured on an Apple M1 Max with whisper.cpp, Metal GPU and flash attention on, whisper.cpp defaults (beam size 5), language forced to en, 8 threads. ● marks the models we treat as tied for best. “89 times real time” means one hour of audio is transcribed in about 1 minute. Sizes in this table are measured file sizes in decimal units, so they differ from the labels in the app and on the other pages: the 574 MB file of Large V3 Turbo Q5 is 547 MB there, Small is 466 MB (here 488 MB), Tiny is 75 MB (here 78 MB), Large V3 Q5 is 1.0 GB and Large V3 is 3 GB.
Are the top 4 models really tied?
Large V3 Q5, Large V3, Large V3 Turbo, Large V3 Turbo Q5 finished within 13 words of each other across 6,086 words of reference text. That is the noise floor of the measurement, not a difference. We treat any model within 0.5 percentage points of the best as tied, because ranking noise would make this page look more decisive and be less true.
How to reproduce this benchmark
The dataset is public (OpenSLR), the models are the standard ggml builds of Whisper, and the harness is a single Python file. The sample is chosen deterministically (sorted utterance ids, evenly strided, no random seed), so an independent run picks the same utterances.
The scoring normaliser is the part worth checking, because a word error rate means nothing without it. Ours ignores differences that are formatting rather than recognition (case, punctuation, hyphenation, 23 against twenty three, Mr. against mister) and still counts every real substitution, deletion and insertion. It is locked by eleven test cases, including ones that assert real errors are not normalised away. Without that step models are punished for punctuation style instead of accuracy.
The speed figures come from one machine; a different Apple Silicon chip will land somewhere else: the order should hold, the absolute numbers will not. This measures the transcription models only, not speaker labelling, which is a separate stage with its own error rate.
Questions
Questions about choosing a Whisper model
Which Whisper model should I use on my Mac?
Large V3 Turbo Q5. In this benchmark the 4 large variants finished within 13 words of each other across 6,086 words, so they are tied on accuracy, and of those, Large V3 Turbo Q5 is the smallest download at 574 MB and runs at 12 times real time.
Is the biggest Whisper model the most accurate?
Not measurably. Large V3 is 3.1 GB, 5.4 times the download of Large V3 Turbo Q5, and scored within 13 words of it across 6,086 words, while running 1.6 times slower.
Is a quantised Whisper model worse?
Not measurably here. Large V3 Turbo Q5 is 2.8 times smaller than Large V3 Turbo and made 4 more errors out of 6,086 words. That is within the noise of this measurement, so it is a reason to test quantisation on your own audio rather than a guarantee that it never costs accuracy.
Which Whisper transcription app is the best?
No benchmark can crown one. The 4 leading models finish within 13 words of each other across 6,086 words, so the model underneath barely decides it. What separates the apps is whether they run on-device, transcribe existing recordings, and label who spoke.
More