MMODELYST
Evals/MATH
EVL

MATH

Competition math

Source
benchmarkMathAccuracy (exact-match on final answer)12.5K items
What this measures

Competition mathematics — algebra, geometry, number theory, calculus — at the level of high-school math olympiads. Much harder than grade-school math.

Sample
If $x^2 + y^2 = 1$ and $x + y = \tfrac{1}{2}$, find $xy$. → $-\tfrac{3}{8}$
Leaderboard · 98 models
Accuracy (exact-match on final answer) — higher is better
1GPT-5
99.4
2Grok 3 mini Reasoning
99.2
3o3
99.2
4Claude 4 Sonnet
99.1
5Grok 4
99
6o4-mini
98.9
7Gemini 2.5 Pro Preview (May' 25)
98.6
8Qwen3 235B A22B 2507
98.4
9DeepSeek R1 0528 (May '25)
98.3
10Llama Nemotron Super 49B v1.5
98.3
11Claude 4 Opus
98.2
12Gemini 2.5 Flash
98.1
13Gemini 2.5 Flash Preview
98.1
14Qwen3 235B A22B 2507 Instruct
98
15MiniMax M1 80k
98
16Gemini 2.5 Pro Preview (Mar' 25)
98
17GLM-4.5
97.9
18EXAONE 4.0 32B
97.7
19Qwen3 30B A3B 2507
97.6
20Qwen3 30B A3B 2507 Instruct
97.5
21o3-mini
97.3
22MiniMax M1 40k
97.2
23Kimi K2
97.1
24o1
97
25Gemini 2.5 Flash-Lite
96.9
26Solar Pro 2
96.7
27Gemini 2.5 Pro
96.7
28DeepSeek R1 (Jan '25)
96.6
29GLM-4.5-Air
96.5
30Magistral Small 1
96.3
31Qwen3 14B
96.1
32Qwen3 32B
96.1
33Qwen3 30B A3B
95.9
34Llama-3_3-Nemotron-Super-49B-v1
95.9
35QwQ 32B
95.7
36Sonar Reasoning Pro
95.7
37R1 1776
95.4
38Llama 3.1 Nemotron Ultra 253B v1
95.2
39DeepSeek R1 Distill Qwen 14B
94.9
40Llama 3.1 Nemotron Nano 4B v1.1
94.7
41Claude 3.7 Sonnet
94.7
42Gemini 2.0 Flash Thinking Experimental (Jan '25)
94.4
43o1-mini
94.4
44Qwen3 Coder 480B A35B Instruct
94.2
45DeepSeek V3 0324
94.2
46DeepSeek R1 Distill Qwen 32B
94.1
47DeepSeek R1 Distill Llama 70B
93.5
48Qwen3 4B
93.3
49DeepSeek R1 0528 Qwen3 8B
93.2
50ERNIE 4.5 300B A47B
93.1
51Qwen3 235B A22B
93
52Gemini 2.0 Flash (Feb '25)
93
53GPT-4.1 mini
92.5
54o1-preview
92.4
55Gemini 2.0 Pro Experimental (Feb '25)
92.3
56Sonar Reasoning
92.1
57Magistral Medium 1
91.7
58GPT-4.1
91.3
59Gemini 2.0 Flash
91.1
60QwQ 32B-Preview
91
61Mistral Medium 3
90.7
62Qwen3 8B
90.4
63DeepSeek-V3
90.2
64Solar Pro 2 (Preview)
90
65Qwen3 1.7B
89.4
66Qwen3 Coder 30B A3B Instruct
89.3
67GPT-4o (March 2025, chatgpt-4o-latest)
89.3
68Reka Flash 3
89.3
69Llama 4 Maverick
88.9
70DeepSeek V3 (Dec '24)
88.7
71Mistral Small 3.2
88.3
72Gemma 3 27B Instruct
88.3
73Gemini 1.5 Pro
87.6
74Gemini 2.0 Flash-Lite (Feb '25)
87.3
75Gemini 2.0 Flash-Lite (Preview)
87.3
76Grok 3
87
77Qwen2.5 Instruct 72B
85.8
78DeepSeek R1 Distill Llama 8B
85.3
79Gemma 3 12B Instruct
85.3
80GPT-4.1 nano
84.8
81Sarvam M
84.7
82Llama 4 Scout
84.4
83Nova Premier
83.9
84Qwen2.5 Max
83.5
85Qwen2.5-72B-Instruct
83.1
86Gemini 1.5 Flash (Sep '24)
82.7
87Sonar
81.7
88phi-4
81
89Qwen2.5 Instruct 32B
80.5
90Qwen2.5 Turbo
80.5
91GPT-4o (ChatGPT)
79.7
92GPT-4o
79.5
93GPT-4o (May '24)
79.1
94GPT-4o mini
78.9
95Nova Pro
78.6
96Claude 3.5 Sonnet
78.3
97Llama 3.1 Tulu3 405B
77.8
98Grok-2
77.8
Benchmark health
saturating

Top models cluster near the ceiling — differences here are getting too small to mean much. Weight newer, harder benchmarks more.

98.8
top-10 mean
99.4
best
98
models
What improves this score

Datasets & environments shown to raise it.