LesskeysWhisper accuracy in 29 languages
Run the 547 MB Large V3 Turbo Q5 model: across 29 languages no larger model was significantly better in 23 of them. The Hindi, Arabic and Hebrew rows overstate their error rates and are being re-measured, so read them with the correction at the top.
macOS 13+ on Apple Silicon · No subscription
Last reviewed
This guide is for Mac. For Windows, see Running Whisper on Windows.
Read this first
Rows under re-measurement
Correction, dated 2026-08-28
The scorer behind this table normalised text with [^\w\s], and Python's \w does not match Unicode combining marks. Every Devanagari vowel sign, Bengali nukta and Arabic diacritic was therefore replaced by a space, splitting one word into several. Measured on the same references this page used: Tamil scored 1,520 units against 437 real words, Hindi 1,237 against 683, Assamese 1,220 against 498. Latin, Cyrillic and CJK rows are unaffected — those scripts carry no combining marks — and so are the conclusions that rest on them.
This inflates the reference and, because fragments cannot align, the error count with it. So the rows for Hindi, Arabic, Hebrew and the other combining-mark scripts overstate the error rate by an amount that varies per language, and the “worst language” claim further down is not safe for those rows. A corrected sweep across 83 languages is running now and this page will be regenerated from it.
It is recorded here rather than quietly deleted, for the same reason as the retraction below: a normaliser that silently mangles non-Latin script is exactly the failure a reader of a multilingual benchmark should be told about, and it is not visible in any published number.
Status, 15 October 2026
The new measurement covers 83 languages and six models, for both file transcription and dictation. It is not on this page yet, because it is not finished: 89 of its 996 test runs failed, all of them on the largest models and in 17 languages, and the runs did not all use the same amount of computer memory or the same version of the test tool. Until it is finished, the table below is still the 29-language table described above, so the rows marked † (Hindi, Arabic and Hebrew, named in the correction) still carry the inflated error rates, and so may other combining-mark scripts. The comparisons on Latin, Cyrillic and CJK rows are not affected.
The answer
The 547 MB model holds up
In 23 of 29 languages no model in the set is significantly better than the 547 MB default.
- 23 of 29languages: no bigger model clearly better
- 6languages where a bigger model earns its size
- 9 of 29languages where the 1.6 GB turbo model scores worse
Significantly means a two-proportion z-test at z ≤ −2 against the default. Read the other 23 as not separated, not as proven equal.
Exceptions
Where a bigger model earns its size
If you work in Thai, Japanese, Finnish, French, Dutch, Korean, the larger download is worth it.
| What | Default (547 MB) | Better model | Gain |
|---|---|---|---|
| Thai | 20.8% | v3: 12.6% | −8.2 pp |
| Japanese | 18.5% | v3-q5: 14.2% | −4.3 pp |
| Finnish | 10.8% | v3-q5: 7.7% | −3.1 pp |
| French | 10.0% | v3: 7.5% | −2.5 pp |
| Dutch | 10.8% | v3-q5: 8.3% | −2.5 pp |
| Korean | 11.3% | v3-q5: 9.4% | −1.9 pp |
v3-q5 (1.0 GB) captures nearly all of the gain: the 3 GB v3 never improves on it by more than 1.2 percentage points.
Not always bigger
Bigger models are not always more accurate
The 1.6 GB turbo model scores numerically worse than the 547 MB default in 9 of 29 languages.
The 75 MB model is not usable outside English: Hindi† 100.0%, Danish 87.2%, Hungarian 84.8%, Czech 84.0%, Hebrew† 81.9%.
Which languages, and by how much
Hindi† (42.1% against 34.4%), Thai (24.6% against 20.8%), Chinese (Simplified) (16.8% against 13.7%), Malay (9.8% against 7.1%), Vietnamese (10.3% against 9.8%), Indonesian (7.8% against 7.4%), Turkish (15.5% against 15.2%), Greek (11.3% against 11.2%), French (10.1% against 10.0%). Margins that small are not all individually significant at this sample size; the point is the direction, which size alone would not predict. Reaching for the next model up is not a safe default.
Extremes
Easiest and hardest
Best result: Spanish, 2.8%.
The highest error rate that no model improves on is Hebrew†, which does not fall below 48.3% in any configuration tested. The correction says this reading is not safe for combining-mark scripts such as Hebrew, so take it as unconfirmed until the corrected sweep is on this page.
Retraction
A retraction, kept in place
Retraction
An earlier version of this sweep reported large-v3 as significantly better than the default for Ukrainian. That was wrong. It was measured under beam 5, which is whisper-cli's default but not what the app ships. Re-run greedy — the real setting — the default and v3 land in a dead tie. The gap was the decoder, not the model. It is recorded here rather than quietly deleted, because the decoder setting is exactly the variable most local-Whisper comparisons leave unstated.
The data
The full table, the method and the limits
The full table, the method and the limits, collapsed: open any of them.
Error rate by language and model (29 languages, 6 models)
Error rate by language and model (29 languages, 6 models)
| Language | Metric | tiny 75 MB | small 466 MB | turbo-q5 547 MB | turbo 1.6 GB | v3-q5 1.0 GB | v3 3 GB |
|---|---|---|---|---|---|---|---|
| Arabic† ar-SA | WER | 67.0% | 36.9% | 22.9% | 22.4% | 22.3% | 22.2% ● |
| Chinese (Simplified) zh-Hans | CER | 55.0% | 24.8% | 13.7% | 16.8% | 13.1% | 12.6% ● |
| Czech cs | WER | 84.0% | 40.4% | 13.2% | 12.8% | 13.1% | 11.9% ● |
| Danish da | WER | 87.2% | 35.0% | 13.4% | 13.0% | 12.6% ● | 13.3% |
| Dutch nl-NL | WER | 74.8% | 28.8% | 10.8% | 9.8% | 8.3% ● | 8.4% |
| English en-US | WER | 31.9% | 10.7% | 9.1% | 8.9% | 7.0% ● | 8.9% |
| Finnish fi | WER | 64.4% | 31.8% | 10.8% | 10.1% | 7.7% ● | 7.7% ● |
| French fr-FR | WER | 50.1% | 26.1% | 10.0% | 10.1% | 7.7% | 7.5% ● |
| German de-DE | WER | 35.0% | 11.9% | 5.2% | 4.7% ● | 5.2% | 5.2% |
| Greek el | WER | 74.7% | 32.9% | 11.2% ● | 11.3% | 11.8% | 12.3% |
| Hebrew† he | WER | 81.9% | 63.2% | 51.0% | 48.4% | 48.3% ● | 48.3% ● |
| Hindi† hi | WER | 100.0% | 68.3% | 34.4% | 42.1% | 33.4% | 32.4% ● |
| Hungarian hu | WER | 84.8% | 48.7% | 17.9% | 17.9% | 17.0% | 16.3% ● |
| Indonesian id | WER | 59.2% | 21.8% | 7.4% ● | 7.8% | 8.1% | 9.4% |
| Italian it | WER | 31.8% | 8.2% | 4.0% | 2.9% ● | 4.6% | 4.5% |
| Japanese ja | CER | 54.3% | 25.4% | 18.5% | 15.6% | 14.2% ● | 17.0% |
| Korean ko | CER | 33.9% | 24.6% | 11.3% | 11.3% | 9.4% ● | 9.5% |
| Malay ms | WER | 62.5% | 25.1% | 7.1% | 9.8% | 6.0% | 5.9% ● |
| Norwegian no | WER | 64.0% | 24.2% | 10.0% | 9.8% ● | 10.0% | 9.8% ● |
| Polish pl | WER | 57.0% | 17.7% | 6.7% | 6.4% ● | 7.4% | 7.6% |
| Portuguese (BR) pt-BR | WER | 25.4% | 7.9% | 5.2% | 5.1% | 5.0% | 4.9% ● |
| Romanian ro | WER | 79.1% | 33.4% | 10.2% | 9.9% | 9.5% | 9.2% ● |
| Russian ru | WER | 38.7% | 12.9% | 6.4% | 6.3% ● | 6.8% | 7.3% |
| Spanish es-ES | WER | 19.3% | 7.0% | 3.5% | 3.5% | 3.0% | 2.8% ● |
| Swedish sv | WER | 57.2% | 21.4% | 8.3% | 8.1% | 7.1% ● | 7.3% |
| Thai th | CER | 62.4% | 27.5% | 20.8% | 24.6% | 13.2% | 12.6% ● |
| Turkish tr | WER | 62.1% | 30.5% | 15.2% | 15.5% | 13.1% ● | 13.5% |
| Ukrainian uk | WER | 68.3% | 39.2% | 23.1% | 21.0% ● | 23.3% | 23.1% |
| Vietnamese vi | WER | 64.1% | 26.6% | 9.8% ● | 10.3% | 10.2% | 10.2% |
How it was measured
Six Whisper model variants transcribed the same read-speech corpus in 29 languages, every language the Lesskeys App Store listing ships in, on Apple Silicon, through the same decoder settings the app itself uses (greedy, no beam search). Rates are word error rate, except for Japanese, Korean, Thai and Chinese, where words are not space-delimited and character error rate is used instead. Lower is better.
● marks the best model for that language. The highlighted column is the 547 MB model, the default in Lesskeys 3.1.4, and the baseline for everything above. † marks a row that is under re-measurement. Scroll the table sideways on a phone; the language stays in view.
The limits of this test
- One chunk per language (about 1,000 words, or about 2,000 characters for the character-error languages). Enough to separate large differences, not enough to call small ones. Where this page says a model “holds”, read that as not separated at this sample size, not as proven identical.
- whisper-cli with the app's context and decode settings, not the full Lesskeys pipeline, which adds voice-activity detection, a hallucination filter, loudness normalization and a language clamp. This table does not measure that pipeline.
- Read speech, not meetings. Conversational and overlapping audio is materially harder.
- Absolute rates carry some inflation from number and punctuation formatting differences that a human reader would not count as errors. Comparisons between models on the same language are the trustworthy part; comparisons between languages are noisier.
Questions
Questions about Whisper accuracy by language
Which Whisper model should I run on a Mac?
In 23 of the 29 languages measured, the 547 MB quantized Large V3 Turbo model is as accurate as anything larger, so it is the sensible default. The exceptions are listed on this page; in those languages the 1.0 GB v3-q5 model is worth the extra download.
Does quantizing a Whisper model hurt accuracy?
Barely. Across all 29 languages the full-precision 3 GB model never beats the 1.0 GB quantized one by more than 1.2 percentage points, and the 547 MB quantized default matches much larger models in most languages.
Is a bigger Whisper model always more accurate?
No. The 1.6 GB turbo model scores numerically worse than the 547 MB default in 9 of the 29 languages measured. Model size is not a reliable proxy for accuracy in a given language.
Was this measured in the cloud?
No. Every number was produced locally on Apple Silicon, which is also how Lesskeys runs transcription: the audio never leaves the machine.
More