Flow - 4x faster than typing
Mini toolsMATH
MATH (Hendrycks)
12.5k competition mathematics problems (AMC, AIME, USAMO style). Reported as overall % or split by Level 1-5 difficulty. The "easy" levels are now saturated; Level 5 still discriminates.
Math
Text
Accuracy
Max 100.0%
Released Mar 2021
Saturated
Possibly contaminated
17
Results
16
Models scored
88.6%
Top: Seed 1.5
69.2%
Median
Best results
Top primary scores; one row per model.
Frontier over time
Each dot is one model result; the line traces the running best score.
All results
Showing all configurations including non-primary alternates. · Show only primary
| # | Model | Score | Conditions | Eval date | Source | Flags |
|---|---|---|---|---|---|---|
| 1 | Gemini 2.0 Flash | 90.9% | 0-shot · standard | 05 Feb 2025 | Self-reported | |
| 2 | Seed 1.5 | 88.6% | — | 22 Jan 2025 | Self-reported | Primary |
| 3 | Nemotron 3 Super | 84.8% | 4-shot | 03 Apr 2026 | Self-reported | Primary |
| 4 | Command A | 80.0% | — | 07 Apr 2025 | Self-reported | Primary |
| 5 | Llama 3.3 | 77.0% | 0-shot · CoT | 06 Dec 2024 | Self-reported | Primary |
| 6 | Nova Pro | 76.6% | 0-shot · CoT | 03 Dec 2024 | Self-reported | Primary |
| 7 | GPT-4o | 76.6% | — | 16 Apr 2025 | Self-reported | Primary |
| 8 | Nova Lite | 73.3% | 0-shot · CoT | 03 Dec 2024 | Self-reported | Primary |
| 9 | Claude Sonnet 3.5 | 71.1% | 0-shot · CoT · standard | 20 Jun 2024 | Self-reported | |
| 10 | Nova Micro | 69.3% | 0-shot · CoT | 03 Dec 2024 | Self-reported | Primary |
| 11 | Claude Haiku 3.5 | 69.2% | 0-shot · CoT | 22 Oct 2024 | Self-reported | Primary |
| 12 | Gemini 1.5 | 67.7% | 4-shot · standard | 01 May 2024 | Self-reported | |
| 13 | Claude Opus 3 | 60.1% | 0-shot · CoT | 22 Oct 2024 | Self-reported | Primary |
| 14 | Gemini 1.5 Flash | 54.9% | 4-shot · standard | 01 May 2024 | Self-reported | |
| 15 | Gemini Ultra | 53.2% | 4-shot · standard | 06 Dec 2023 | Self-reported | |
| 16 | Pixtral 12B | 48.1% | Maj@1 | 10 Oct 2024 | Self-reported | Primary |
| 17 | Llama 3.2 | 48.0% | 0-shot · CoT | 25 Oct 2024 | Self-reported | Primary |
| 18 | Claude Haiku 3 | 38.9% | 0-shot · CoT · standard | 04 Mar 2024 | Self-reported | |
| 19 | Mixtral 8x22B | 28.4% | — | 08 Jan 2024 | Paper | Primary |
| 20 | Mixtral 8x7B | 28.4% | — | 01 Dec 2023 | Self-reported | Primary |
| 21 | Gemma 2 | 15.0% | — | 25 Feb 2025 | Self-reported | Primary |
| 22 | Gemma 2 | 15.0% | 4-shot | 25 Feb 2025 | Self-reported | Primary |
| 23 | Mistral 7B | 13.1% | — | 01 Sep 2023 | Self-reported | Primary |
