MMLU
General knowledge% accuracyMassive Multitask Language Understanding — 57-subject multiple-choice exam.
Updated Jun 1, 2025Latest measured Jun 1, 20252 verified · 11 self-reported
Verified results come from third-party or public leaderboard sources. Self-reported results come from provider papers, blogs, or vendor disclosures and should be compared with extra caution.
At a glance
Total results
13
Models tested
13
Providers
9
Verified · Self-reported
2 · 11
Average
85.14 % accuracy
Median
85.2 % accuracy
Range
75.7 – 91 % accuracy
Latest result
Jun 1, 2025
Score distribution
1
0
0
1
3
1
1
2
Methodology
5-shot evaluation across 57 subjects (humanities, STEM, social sciences, law, medicine). Score is percentage of correct answers.
Limitations
Widely used and partially contaminated; ceilings are compressed at the top. Does not measure open-ended reasoning or tool use.
By provider
- Average: 87 % accuracyBest: 91 % accuracy
- DeepSeek· 2 models90.8 % accuracyDeepSeek R1Average: 89.65 % accuracyBest: 90.8 % accuracy
- Anthropic· 2 models87.5 % accuracyClaude Opus 4
Full leaderboard
Showing 13 of 13| # | Model | Provider | Score (% accuracy) |
|---|---|---|---|
| 1 | o3 | OpenAI | 91 |
| 2 | DeepSeek R1 | DeepSeek | 90.8 |
| 3 | DeepSeek V3 | DeepSeek | 88.5 |
| 4 | GPT-5 | OpenAI | 88 |
| 5 | Claude Opus 4 | Anthropic | 87.5 |
| 6 | Grok 3 | xAI | 87 |
| 7 | Gemini 2 Pro | 85.2 | |
| 8 | Phi-4 | Microsoft | 84.8 |
| 9 | Claude Sonnet 4 | Anthropic | 83.1 |
| 10 | Llama 3 70B | Meta | 82 |
| 11 | GPT-4o mini | OpenAI | 82 |
| 12 | Mistral Large | Mistral AI | 81.2 |
| 13 | Command R+ | Cohere | 75.7 |
Community ratings
No ratings yet. Be the first to rate MMLU.
Rate MMLU
Sign in to rate and review.
Comments
Sign in to leave a comment.