EVL
GPQA Diamond
Graduate-level QA
What this measures
Genuinely hard science questions written by PhDs — biology, physics, chemistry — that are 'Google-proof' (you can't just look up the answer). Tests deep reasoning, not recall.
Sample
A graduate-level question like: 'Which quantum number determines the shape of an atomic orbital?' but at a difficulty where even domain experts score ~65%.
Leaderboard · 99 models
Accuracy on the hardest 'Diamond' subset — higher is betterBenchmark health
saturatingTop models cluster near the ceiling — differences here are getting too small to mean much. Weight newer, harder benchmarks more.
93.0
top-10 mean
94.1
best
99
models
Recent changes
Research using this benchmark
all →Deeper is Not Always Better: Mitigating the Alignment Tax via Confident Layer DecodingWhen Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier ModelsReasoning with Sampling: Cutting at Decision PointsTRACER: Turn-level Regret Matching with Inner Reinforcement Credit for Cooperative Multi-LLM ReasoningDetection Without Correction: A Two-Parameter Decomposition of Multi-Stage LLM Pipelines
Matched by name in title/abstract.