Signals · novel-idea finder
Discover
Benchmark correlation
Pearson r across models scored on both benchmarks. Amber = redundant (measures the same thing); blue = independent signal. Pick benchmarks that disagree to cover more ground.
aa-lcr | aime | bbh | gpqa | gsm8k | hle | humaneval | ifbench | ifeval | livecodebench | math | math-lvl5 | mmlu | mmlu-pro | mmmu | musr | scicode | swe-bench | tau2-bench | terminal-bench-hard | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| aa-lcr | · | 0.7 | 0.8 | 0.7 | 0.8 | 0.7 | 0.4 | 0.6 | 0.8 | 0.8 | 0.8 | 0.8 | 0.8 | |||||||
| aime | 0.7 | · | 0.9 | 0.7 | 0.7 | 0.9 | 0.7 | 0.7 | 0.7 | 0.7 | 0.6 | 0.6 | ||||||||
| bbh | · | 0.7 | 0.7 | 0.6 | -0.1 | 0.2 | 0.4 | 0.9 | 0.9 | 0.5 | ||||||||||
| gpqa | 0.8 | 0.9 | 0.7 | · | 0.8 | 0.7 | 0.8 | 0.8 | -0.2 | 0.9 | 0.8 | 0.3 | 0.9 | 0.9 | 0.8 | 0.6 | 0.9 | 0.8 | 0.7 | 0.8 |
| gsm8k | 0.7 | 0.8 | · | 0.9 | 0.8 | 0.5 | 0.8 | 0.8 | 0.9 | 0.6 | ||||||||||
| hle | 0.7 | 0.7 | 0.7 | · | 0.2 | 0.8 | 0.7 | 0.4 | 0.5 | 0.5 | 0.7 | 0.7 | 0.8 | 0.7 | 0.9 | |||||
| humaneval | 0.6 | 0.8 | 0.9 | 0.2 | · | 0.2 | 0.9 | 0.8 | 0.8 | 0.9 | 0.9 | 0.8 | 0.9 | 0.5 | 0.9 | |||||
| ifbench | 0.8 | 0.7 | 0.8 | 0.8 | 0.2 | · | 0.8 | 0.5 | 0.8 | 0.7 | 0.7 | 0.8 | -0.0 | 0.8 | 0.8 | |||||
| ifeval | -0.1 | -0.2 | 0.8 | 0.9 | · | -0.2 | -0.0 | 0.9 | -0.2 | -0.3 | ||||||||||
| livecodebench | 0.7 | 0.9 | 0.9 | 0.7 | 0.8 | 0.8 | · | 0.8 | 0.8 | 0.8 | 0.8 | 0.8 | 0.3 | 0.7 | 0.7 | |||||
| math | 0.4 | 0.7 | 0.2 | 0.8 | 0.5 | 0.4 | 0.8 | 0.5 | -0.2 | 0.8 | · | 0.6 | 0.7 | 0.9 | 0.8 | 0.4 | 0.7 | 0.7 | 0.5 | 0.5 |
| math-lvl5 | 0.4 | 0.3 | 0.8 | 0.9 | -0.0 | 0.6 | · | 0.7 | 0.6 | 0.3 | ||||||||||
| mmlu | 0.9 | 0.9 | 0.8 | 0.5 | 0.9 | 0.8 | 0.9 | 0.8 | 0.7 | 0.7 | · | 0.9 | 0.9 | 0.8 | 0.8 | |||||
| mmlu-pro | 0.6 | 0.7 | 0.9 | 0.9 | 0.9 | 0.5 | 0.8 | 0.7 | -0.2 | 0.8 | 0.9 | 0.6 | 0.9 | · | 0.9 | 0.6 | 0.9 | 0.8 | 0.6 | 0.6 |
| mmmu | 0.8 | 0.7 | 0.8 | 0.7 | 0.9 | 0.7 | 0.8 | 0.8 | 0.9 | 0.9 | · | 0.9 | 0.5 | 0.7 | 0.8 | |||||
| musr | 0.5 | 0.6 | 0.6 | 0.5 | -0.3 | 0.4 | 0.3 | 0.8 | 0.6 | · | ||||||||||
| scicode | 0.8 | 0.7 | 0.9 | 0.7 | 0.9 | 0.8 | 0.8 | 0.7 | 0.8 | 0.9 | 0.9 | · | 0.9 | 0.7 | 0.8 | |||||
| swe-bench | 0.8 | 0.8 | 0.8 | -0.0 | 0.3 | 0.7 | 0.8 | 0.5 | 0.9 | · | 0.6 | 1.0 | ||||||||
| tau2-bench | 0.8 | 0.6 | 0.7 | 0.7 | 0.8 | 0.7 | 0.5 | 0.6 | 0.7 | 0.7 | 0.6 | · | 0.8 | |||||||
| terminal-bench-hard | 0.8 | 0.6 | 0.8 | 0.9 | 0.8 | 0.7 | 0.5 | 0.6 | 0.8 | 0.8 | 1.0 | 0.8 | · |
Most redundant pairs
gpqa × livecodebenchr=0.93 · n=276
aime × livecodebenchr=0.91 · n=207
gpqa × scicoder=0.91 · n=401
Most independent pairs
ifeval × musrr=-0.30 · n=1480
ifeval × mmlu-pror=-0.22 · n=1480
gpqa × ifevalr=-0.20 · n=1482
Capability coverage — white space
How well each capability is measured (scored model-benchmark pairs). Thin bars = under-explored — opportunity.
Efficiency frontier
Best average benchmark score achievable at each model size (Pareto frontier) — models that punch above their weight.
Price vs performance — best value
Average benchmark score per dollar of output (per 1M tokens), among models with published pricing. Higher value = more quality for your money. Prices are vendor-reported — verify before relying.
ModelAvg$/M in·outValue
Model release velocity
Models in the catalog by release quarter (last 3 years).
23 Q3
23 Q4
24 Q1
24 Q2
24 Q3
24 Q4
25 Q1
25 Q2
25 Q3
25 Q4
26 Q1
26 Q2
26 Q3