SWE-bench Pro
Created by Scale AI, 19 Sep 2025 · Higher is better · Website
- OpenAI
- Anthropic
- Other labs
- Reported in a later post or card
Results
| Model | Score | Source |
|---|---|---|
| Claude Opus 5.522 Sep 2026 · Anthropic | 89.9% | Claude Opus 5.5 System CardChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1adaptive thinking, max effort; average of 5 trialsOur capture |
| Claude Sonnet 5.528 Sep 2026 · Anthropic | 81.3% | Claude Sonnet 5.5 System CardChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1adaptive thinking, max effort; average of 5 trialsOur capture |
| Claude Fable 5.11 Sep 2026 · Anthropic | 81.2% | Claude Fable 5.1 and Claude Mythos 5.1 System CardChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1average of 5 trials |
| Claude Mythos 59 Jun 2026 · Anthropic | 80.3% | Claude Fable 5 and Claude Mythos 5 System CardChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1adaptive thinking, max effort; average of 5 trialsOur capture |
| Claude Fable 59 Jun 2026 · Anthropic | 80% | Claude Fable 5 and Claude Mythos 5 System CardChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 4 other lab documents. The first 3 are linked: 1 2 3adaptive thinking, max effort; average of 5 trials; with production safeguards, including fallback to Opus 4.8Our capture |
| Claude Opus 524 Jul 2026 · Anthropic | 79.2% | Claude Opus 5 System CardChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 2 other lab documents: 1 2adaptive thinking, max effort; average of 5 trialsOur capture |
| Claude Mythos Preview7 Apr 2026 · Anthropic | 77.8% | Introducing Claude Opus 4.7Checked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 2 other lab documents: 1 2reported laterSource image |
| Claude Opus 4.828 May 2026 · Anthropic | 69.2% | Introducing Claude Opus 4.8Checked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 7 other lab documents. The first 3 are linked: 1 2 3Source image |
| Hy4 preview28 Aug 2026 · Tencent | 65.7% | Introducing Hy4 previewChecked A person checked this number against its source on 9 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.SWE-agent scaffold; highest reasoning settingSource image |
| Claude Haiku 5.57 Oct 2026 · Anthropic | 64.8% | Claude Haiku 5.5 System CardChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.adaptive thinking, max effort; average of 5 trialsOur capture |
| GPT-5.6 Sol9 Jul 2026 · OpenAI | 64.6% | GPT-5.6: Frontier intelligence that scales with your ambitionChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 4 other lab documents. The first 3 are linked: 1 2 3Our capture |
| Claude Opus 4.716 Apr 2026 · Anthropic | 64.3% | Introducing Claude Opus 4.7Checked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 6 other lab documents. The first 3 are linked: 1 2 3memorization screens flag a subset of problemsSource image |
| GPT-5.6 Terra9 Jul 2026 · OpenAI | 63.4% | GPT-5.6: Frontier intelligence that scales with your ambitionChecked A person checked this number against its source on 26 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Our capture |
| Claude Sonnet 530 Jun 2026 · Anthropic | 63.2% | Introducing Claude Sonnet 5Checked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 3 other lab documents: 1 2 3Source image |
| GPT-5.6 Luna9 Jul 2026 · OpenAI | 62.7% | GPT-5.6: Frontier intelligence that scales with your ambitionChecked A person checked this number against its source on 26 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Our capture |
| GLM-5.216 Jun 2026 · Z.ai | 62.1% | GLM-5.2: Built for Long-Horizon TasksChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1OpenHands with a tailored prompt; 400K contextOur capture |
| Ripple-65B Preview4 Nov 2026 · Ricursive Intelligence | 61.42% | ricursiveAI/Ripple-65B-Preview model cardChecked A person checked this number against its source on 7 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.v1.0; OpenCode harness; maximum effortOur capture |
| GPT-5.523 Apr 2026 · OpenAI | 59.4% | GPT-5.6: Frontier intelligence that scales with your ambitionChecked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.reported laterOur capture |
| MiniMax M31 Jun 2026 · MiniMax | 59.0% | MiniMax M3: Frontier Coding, 1M Context, Native Multimodality — All in One ModelChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1Claude Code harnessSource image |
| Kimi K2.620 Apr 2026 · Moonshot AI | 58.6% | Kimi K2.6: Advancing Open-Source CodingChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 3 other lab documents: 1 2 3in-house harness adapted from SWE-agent; thinking mode; average of 10 runsOur capture |
| GLM-5.17 Apr 2026 · Z.ai | 58.4% | GLM-5.1: Towards Long-Horizon TasksChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 4 other lab documents. The first 3 are linked: 1 2 3OpenHands with a tailored prompt; 200K contextOur capture |
| Claude Sonnet 4.617 Feb 2026 · Anthropic | 58.1% | Introducing Claude Sonnet 5Checked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1reported laterSource image |
| Hy36 Jul 2026 · Tencent | 57.9% | Introducing Hy3Corroborated The same number is printed in 2 other lab documents: 1 2SWE-agent scaffold; highest reasoning settingSource image |
| Hy36 Jul 2026 · Tencent | 46.0% | Introducing Hy3Corroborated The same number is printed in 2 other lab documents: 1 2Hy3 preview, re-evaluated at Hy3 launch; SWE-agent scaffold; highest reasoning setting; another report, not chartedSource image |
| Qwen3.6-Max-Preview18 Apr 2026 · Alibaba | 57.3% | Qwen3.6-Max-Preview: Smarter, Sharper, Still EvolvingChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Source image |
| MiniMax M2.718 Mar 2026 · MiniMax | 56.22% | MiniMax M2.7: Early Echoes of Self-EvolutionChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 2 other lab documents: 1 2 |
| MiniMax M2.718 Mar 2026 · MiniMax | 56.2% | MiniMax M3: Frontier Coding, 1M Context, Native Multimodality — All in One ModelChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 2 other lab documents: 1 2reported later; another report, not chartedSource image |
| DeepSeek-V4-Pro24 Apr 2026 · DeepSeek | 55.4% | DeepSeek V4 Preview ReleaseChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 3 other lab documents: 1 2 3max reasoning effortSource image |
| MiniMax M2.512 Feb 2026 · MiniMax | 55.4% | MiniMax M2.5: Built for Real-World Productivity.Checked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1Claude Code harness; average of 4 runsSource image |
| GLM-512 Feb 2026 · Z.ai | 55.1% | GLM-5.1: Towards Long-Horizon TasksChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1OpenHands with a tailored prompt; reported laterOur capture |
| Claude Opus 4.65 Feb 2026 · Anthropic | 53.4% | Introducing Claude Opus 4.7Checked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1memorization screens flag a subset of problems; reported laterSource image |
| Claude Opus 4.524 Nov 2025 · Anthropic | 52.0% | Claude Opus 4.5 System CardChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.no thinking; average of 5 trialsOur capture |
| Claude Opus 4.524 Nov 2025 · Anthropic | 51.60% | Claude Opus 4.5 System CardChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.64K thinking budget; average of 5 trials; another report, not chartedOur capture |
| Nemotron 3.5 Super VL 120B-A12B15 Oct 2026 · NVIDIA | 51.78% | nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 model card (launch-day revision)Corroborated The same number is printed in 3 other lab documents: 1 2 3Our capture |
| Kimi K2.527 Jan 2026 · Moonshot AI | 50.7% | Kimi K2.5: Visual Agentic IntelligenceChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 2 other lab documents: 1 2in-house minimal-tool harness; non-thinking mode; average of 5 runsOur capture |
| MiniMax M2.123 Dec 2025 · MiniMax | 49.7% | MiniMax M2.5: Built for Real-World Productivity.Checked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Claude Code harness; average of 4 runs; reported laterSource image |
| Nemotron 3 Ultra 550B-A55B4 Jun 2026 · NVIDIA | 49.66% | Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AICorroborated The same number is printed in 2 other lab documents: 1 2OpenCode harness; web search and fetch disabled; average of 3 runs; reported laterOur capture |
| Laguna XS 2.12 Jul 2026 · Poolside | 47.6% | Introducing Laguna XS 2.1Checked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.poolside harness on Harbor; pass@1 averaged over 2 attempts; some task images patchedOur capture |
| Laguna M.128 Apr 2026 · Poolside | 46.9% | Introducing Laguna XS.2 and Laguna M.1Corroborated The same number is printed in 1 other lab document: 1poolside harness on Harbor; pass@1 averaged over 3 runs; some task images patchedOur capture |
| Laguna M.128 Apr 2026 · Poolside | 49.2% | Long context update: Laguna XS.2 and M.1Corroborated The same number is printed in 1 other lab document: 1poolside harness on Harbor; pass@1 averaged over 3 runs; some task images patched; reported later; another report, not chartedOur capture |
| Laguna XS.228 Apr 2026 · Poolside | 44.5% | Introducing Laguna XS.2 and Laguna M.1Corroborated The same number is printed in 1 other lab document: 1poolside harness on Harbor; pass@1 averaged over 3 runs; some task images patchedOur capture |
| Laguna XS.228 Apr 2026 · Poolside | 46.3% | Long context update: Laguna XS.2 and M.1Corroborated The same number is printed in 2 other lab documents: 1 2poolside harness on Harbor; pass@1 averaged over 3 runs; some task images patched; reported later; another report, not chartedOur capture |
| Qwen3-Coder-Next3 Feb 2026 · Alibaba | 44.3% | Qwen3-Coder-Next: Pushing Small Hybrid Models on Agentic CodingChecked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.SWE-Agent harnessSource image |
| SWE-1.529 Oct 2025 · Cognition | 34.47% | Introducing SWE-1.5: Our Fast Agent ModelChecked A person checked this number against its source on 7 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.SWE-agent harnessSource image |
| SWE-1.529 Oct 2025 · Cognition | 29.00% | Introducing SWE-1.5: Our Fast Agent ModelChecked A person checked this number against its source on 7 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Claude Code harness; another report, not chartedSource image |
