Changelog
Releases and progress. Subscribe to Some Bench (RSS)
New results: Gemini 4 Neon
Added Gemini 4 Neon. Best of them: Gemini 4 Neon high, 78.3%.
Added
| Model · effort | Score | Tokens per task | Seconds per task | Cost per test |
|---|---|---|---|---|
| Gemini 4 Neon · high | 78.3% | 7,394 | 62.0 | $0.1121 |
| Gemini 4 Neon · medium | 78.3% | 2,455 | 23.6 | $0.0381 |
| Gemini 4 Neon · low | 75.8% | 1,008 | 11.1 | $0.0164 |
New results: GPT-6.2 Astra and GPT-6.2 Sol
Added GPT-6.2 Astra and GPT-6.2 Sol. Best of them: GPT-6.2 Astra high, 76.1%.
Added
| Model · effort | Score | Tokens per task | Seconds per task | Cost per test |
|---|---|---|---|---|
| GPT-6.2 Astra · high | 76.1% | 335 | 9.0 | $0.0164 |
| GPT-6.2 Astra · medium | 73.6% | 97 | 3.8 | $0.0068 |
| GPT-6.2 Astra · low | 70.9% | 42 | 2.5 | $0.0046 |
| GPT-6.2 Sol · high | 68.0% | 137 | 2.8 | $0.0021 |
| GPT-6.2 Sol · medium | 62.8% | 52 | 1.6 | $0.0013 |
| GPT-6.2 Sol · low | 59.4% | 18 | 1.1 | $0.0009 |
New results: Gemini 4 Carbon and Gemini 4 Hydrogen
Added Gemini 4 Carbon and Gemini 4 Hydrogen. Best of them: Gemini 4 Carbon medium, 78.3%. Fable 5.2 and Opus 5.5 were run again.
Added
| Model · effort | Score | Tokens per task | Seconds per task | Cost per test |
|---|---|---|---|---|
| Gemini 4 Carbon · medium | 78.3% | 1,874 | 38.4 | $0.0195 |
| Gemini 4 Hydrogen · high | 76.1% | 4,406 | 27.6 | $0.0337 |
| Gemini 4 Carbon · high | 75.5% | 10,067 | 175.9 | $0.1014 |
| Gemini 4 Carbon · low | 72.5% | 254 | 8.4 | $0.0034 |
| Gemini 4 Hydrogen · medium | 72.5% | 2,236 | 14.4 | $0.0174 |
| Gemini 4 Hydrogen · low | 66.4% | 875 | 5.7 | $0.0072 |
Run again
The page shows the average of all runs; these are the newest run against the average before it.
| Model · effort | Score | Tokens per task |
|---|---|---|
| Fable 5.2 · high | 75.8% → 73.0% | 798 → 753 |
| Fable 5.2 · low | 72.5% → 73.8% | 474 → 424 |
| Fable 5.2 · medium | 73.3% → 72.5% | 509 → 457 |
| Opus 5.5 · high | 70.3% → 68.6% | 608 → 568 |
| Opus 5.5 · low | 65.1% → 62.2% | 321 → 308 |
| Opus 5.5 · medium | 68.0% → 67.5% | 492 → 465 |
New results: Ripple 65B Preview and Inkling
Added Ripple 65B Preview and Inkling. Best of them: Ripple 65B Preview medium, 60.6%.
Added
| Model · effort | Score | Tokens per task | Seconds per task | Cost per test |
|---|---|---|---|---|
| Ripple 65B Preview · medium | 60.6% | 10,813 | 102.1 | $0.0065 |
| Ripple 65B Preview · low | 60.0% | 11,402 | 108.3 | $0.0069 |
| Ripple 65B Preview · high | 57.8% | 10,530 | 96.2 | $0.0064 |
| Inkling · high | 52.8% | 6,903 | 69.5 | $0.0283 |
| Inkling · medium | 39.5% | 2,424 | 32.6 | $0.0101 |
| Inkling · low | 26.7% | 375 | 6.3 | $0.0018 |
Very verbose LLMs
Old trick played again: very very verbose LLMs
New results: Muse Ember and Kimi K3.1 Pro
Added Muse Ember and Kimi K3.1 Pro. Best of them: Muse Ember high, 67.2%.
Added
| Model · effort | Score | Tokens per task | Seconds per task | Cost per test |
|---|---|---|---|---|
| Muse Ember · high | 67.2% | 7,614 | 20.7 | $0.0038 |
| Muse Ember · medium | 62.0% | 5,090 | 10.7 | $0.0026 |
| Muse Ember · low | 60.8% | 3,352 | 7.5 | $0.0017 |
| Kimi K3.1 Pro · high | 60.3% | 1,656 | 42.5 | $0.0175 |
| Kimi K3.1 Pro · low | 42.8% | 494 | 6.3 | $0.0058 |
| Kimi K3.1 Pro · medium | 42.8% | 446 | 5.9 | $0.0053 |
Fewest tokens
71% on just 18 tokens
New results: Fable 5.2, Muse Wonder, Nemotron 3 Ultra and Nemotron 3.5 Super VL
Added Fable 5.2, Muse Wonder, Nemotron 3 Ultra and Nemotron 3.5 Super VL. Best of them: Fable 5.2 high, 75.8%. Opus 5.5 was run again.
Added
| Model · effort | Score | Tokens per task | Seconds per task | Cost per test |
|---|---|---|---|---|
| Fable 5.2 · high | 75.8% | 798 | 9.9 | $0.0372 |
| Muse Wonder · high | 74.8% | 7,284 | 66.1 | $0.1468 |
| Muse Wonder · medium | 73.9% | 5,352 | 47.7 | $0.1082 |
| Fable 5.2 · medium | 73.3% | 509 | 7.1 | $0.0265 |
| Fable 5.2 · low | 72.5% | 474 | 6.8 | $0.0353 |
| Muse Wonder · low | 68.3% | 3,289 | 29.5 | $0.0669 |
| Nemotron 3 Ultra · medium | 50.0% | 8,496 | 77.5 | $0.0206 |
| Nemotron 3 Ultra · high | 47.8% | 8,710 | 77.8 | $0.0211 |
| Nemotron 3 Ultra · low | 47.8% | 8,902 | 79.5 | $0.0215 |
| Nemotron 3.5 Super VL · off | 26.4% | 5,380 | 24.1 | $0.0027 |
Run again
The page shows the average of all runs; these are the newest run against the average before it.
| Model · effort | Score | Tokens per task |
|---|---|---|
| Opus 5.5 · high | 71.4% → 68.0% | 636 → 552 |
| Opus 5.5 · low | 65.1% → 65.0% | 322 → 319 |
| Opus 5.5 · medium | 67.8% → 68.3% | 503 → 472 |
Opus 5.5 and Fable 5.2
Is Opus 5.5 nerfed after Fable 5.2's release?
Best value
The top score isn't the best buy