New results: Gemini 4 Carbon and Gemini 4 Hydrogen
Added Gemini 4 Carbon and Gemini 4 Hydrogen. Best of them: Gemini 4 Carbon medium, 78.3%. Fable 5.2 and Opus 5.5 were run again.
Added
| Model · effort | Score | Tokens per task | Seconds per task | Cost per test |
|---|---|---|---|---|
| Gemini 4 Carbon · medium | 78.3% | 1,874 | 38.4 | $0.0195 |
| Gemini 4 Hydrogen · high | 76.1% | 4,406 | 27.6 | $0.0337 |
| Gemini 4 Carbon · high | 75.5% | 10,067 | 175.9 | $0.1014 |
| Gemini 4 Carbon · low | 72.5% | 254 | 8.4 | $0.0034 |
| Gemini 4 Hydrogen · medium | 72.5% | 2,236 | 14.4 | $0.0174 |
| Gemini 4 Hydrogen · low | 66.4% | 875 | 5.7 | $0.0072 |
Run again
The page shows the average of all runs; these are the newest run against the average before it.
| Model · effort | Score | Tokens per task |
|---|---|---|
| Fable 5.2 · high | 75.8% → 73.0% | 798 → 753 |
| Fable 5.2 · low | 72.5% → 73.8% | 474 → 424 |
| Fable 5.2 · medium | 73.3% → 72.5% | 509 → 457 |
| Opus 5.5 · high | 70.3% → 68.6% | 608 → 568 |
| Opus 5.5 · low | 65.1% → 62.2% | 321 → 308 |
| Opus 5.5 · medium | 68.0% → 67.5% | 492 → 465 |