SWE-bench Pro V2
Created by Scale AI, 22 Sep 2026 · Higher is better · Website
Versions
| Version | Released | Status | Models |
|---|---|---|---|
| SWE-bench Verified | 13 Aug 2024 | Inactive | 100 |
| SWE-bench Pro | 19 Sep 2025 | Active | 69 |
| SWE-bench Pro V2 | 22 Sep 2026 | Active | 19 |
Settings
| Task set | Reported by |
|---|---|
| Full set (642 tasks) | |
| HARD-51 (51 tasks) |
Bold: Benchmark default based on README · Download image
Results
Show
- OpenAI
- Anthropic
- Other labs
- On the creator’s leaderboard
- OpenAI
- Anthropic
- Other labs
- On the creator’s leaderboard
| Model | Harness and effort | Score | Source |
|---|---|---|---|
| Claude Opus 524 Jul 2026 | Claude Codexhigh effort | 98.00% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Claude Fable 5.11 Sep 2026 | Claude Codehigh effort | 92.20% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Gemini 4 Argon30 Sep 2026 | mini-SWE-agenthigh effort | 92.16% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| GPT-6 Astra3 Sep 2026 | Codexhigh effort | 90.20% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| SWE-2 Flash4 Nov 2026 | Harness not specifiedEffort not specified | 89.9% | Introducing SWE-2 FlashChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Our capture |
| Kimi K316 Jul 2026 | mini-SWE-agentmax effort | 88.20% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Claude Sonnet 530 Jun 2026 | Claude Codexhigh effort | 88.20% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Inkling 2 Small5 Nov 2026 | mini-SWE-agenteffort 0.99 (xhigh) | 86.3% | Introducing Inkling 2 SmallCorroborated The same number is printed in 2 other documents: 1 2Run by Scale AI · 1M-token trajectory limitOur capture |
| Inkling 2 Small5 Nov 2026 | Harness not specifiedEffort not specified | 86.27% | SWE-Bench Pro (Public Dataset) V2 LeaderboardCorroborated The same number is printed in 1 other document: 1Scale AI’s leaderboard, 18 Nov 2026 · another report |
| GPT-5.6 Terra9 Jul 2026 | Codexxhigh effort | 86.30% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| MiniMax M3.1 Flash19 Oct 2026 | mini-SWE-agentxhigh effort | 84.31% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| GLM-5.314 Aug 2026 | mini-SWE-agentmax effort | 84.30% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| GPT-5.6 Sol9 Jul 2026 | Codexxhigh effort | 82.40% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Ripple-65B Preview4 Nov 2026 | Terminus 2maximum effort per the model card | 79.1% | Introducing Ripple-65B Preview, a Frontier Open-Weight ModelCorroborated The same number is printed in 1 other document: 1Our capture |
| Claude Sonnet 5.528 Sep 2026 | mini-SWE-agentxhigh effort | 70.59% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Gemini 3.8 Flash2 Sep 2026 | mini-SWE-agenthigh effort | 58.80% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Inkling15 Jul 2026 | mini-SWE-agentxhigh effort | 56.90% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Claude Haiku 5.57 Oct 2026 | mini-SWE-agenthigh effort | 52.94% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Claude Haiku 4.515 Oct 2025 | Claude Codexhigh effort | 25.50% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |