MMODELYST
Evals/MBPP
EVL

MBPP

Basic Python problems

Source ↗
benchmarkCodepass@1 (% solved correctly)974 items
What this measures

Mostly Basic Python Problems — short, entry-level programming tasks. A broad, easier companion to HumanEval.

Sample
'Write a function to find the shared elements from two lists.' — graded by running test cases.
Leaderboard · 0 models
pass@1 (% solved correctly) — higher is better
No scores recorded yet.
What improves this score

Datasets & environments shown to raise it.

Code Alpaca+7.0
The Stack v2+6.0
SWE-Gym+5.0
Research using this benchmark
all →
SePO: Self-Evolving Prompt Agent for System Prompt OptimizationCode Llama: Open Foundation Models for CodeWizardCoder: Empowering Code Large Language Models with Evol-InstructProgram Synthesis with Large Language ModelsLiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

Matched by name in title/abstract.