SWE-bench Pro V2
Created by Scale AI, 22 Sep 2026 · Higher is better · Website
Versions
| Version | Released | Status | Models |
|---|---|---|---|
| SWE-bench Verified | 13 Aug 2024 | Inactive | 100 |
| SWE-bench Pro | 19 Sep 2025 | Active | 69 |
| SWE-bench Pro V2 | 22 Sep 2026 | Active | 19 |
Settings
| Task set | Reported by |
|---|---|
| Full set (642 tasks) | |
| HARD-51 (51 tasks) |
Bold: Benchmark default based on README · Download image
Results
Show
- OpenAI
- Anthropic
- Other labs
- On the creator’s leaderboard
- OpenAI
- Anthropic
- Other labs
- On the creator’s leaderboard
| Model | Harness and effort | Score | Source |
|---|---|---|---|
| Claude Opus 524 Jul 2026 | Claude Codexhigh effort | 99.40% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Gemini 4 Argon30 Sep 2026 | mini-SWE-agenthigh effort | 99.22% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Claude Fable 5.11 Sep 2026 | Claude Codehigh effort | 99.10% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Inkling 2 Small5 Nov 2026 | mini-SWE-agenteffort 0.99 (xhigh) | 97.7% | Introducing Inkling 2 SmallCorroborated The same number is printed in 2 other documents: 1 2Run by Scale AI · 1M-token trajectory limitOur capture |
| Inkling 2 Small5 Nov 2026 | Harness not specifiedEffort not specified | 97.66% | SWE-Bench Pro (Public Dataset) V2 LeaderboardCorroborated The same number is printed in 1 other document: 1Scale AI’s leaderboard, 18 Nov 2026 · another report |
| Kimi K316 Jul 2026 | mini-SWE-agentmax effort | 97.70% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| GPT-6 Astra3 Sep 2026 | Codexhigh effort | 96.90% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| MiniMax M3.1 Flash19 Oct 2026 | mini-SWE-agentxhigh effort | 96.57% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| GPT-6.1 Sol29 Sep 2026 | mini-SWE-agentxhigh effort | 96.11% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Claude Sonnet 5.528 Sep 2026 | mini-SWE-agentxhigh effort | 95.79% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| GLM-5.314 Aug 2026 | mini-SWE-agentmax effort | 95.60% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| GPT-5.6 Sol9 Jul 2026 | Codexxhigh effort | 95.50% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Gemini 3.8 Flash2 Sep 2026 | mini-SWE-agenthigh effort | 94.86% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Claude Sonnet 530 Jun 2026 | Claude Codexhigh effort | 93.15% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| GPT-5.6 Terra9 Jul 2026 | Codexxhigh effort | 92.37% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Inkling15 Jul 2026 | mini-SWE-agentxhigh effort | 89.88% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |
| Claude Haiku 5.57 Oct 2026 | mini-SWE-agenthigh effort | 87.23% | SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026 |