EVL
HumanEval
Python code synthesis
What this measures
Can the model write correct Python? Given a function description, it must produce code that passes hidden unit tests. The standard first check of coding ability.
Sample
def has_close_elements(numbers, threshold): '''Return True if any two numbers are closer than threshold.''' — the model writes the body; it's run against tests.
Leaderboard · 16 models
pass@1 (% solved correctly on the first try) — higher is betterBenchmark health
activeDiscriminating but climbing — the frontier is moving up this benchmark.
87.3
top-10 mean
92
best
16
models
What improves this score
Datasets & environments shown to raise it.
Research using this benchmark
all →KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning TasksImproved Large Language Diffusion ModelsAC-ODM: Actor--Critic Online Data Mixing for Sample-Efficient LLM PretrainingHonest Lying: Understanding Memory Confabulation in Reflexive AgentsEvaluating Large Language Models Trained on Code
Matched by name in title/abstract.