Frontier Log

Benchmark · Active

SWE-bench Pro V2

Created by Scale AI, 22 Sep 2026 · Higher is better · Website

Versions

VersionReleasedStatusModels
SWE-bench Verified13 Aug 2024Inactive100
SWE-bench Pro19 Sep 2025Active69
SWE-bench Pro V222 Sep 2026Active19

Settings

Task setReported by
Full set (642 tasks)Thinking MachinesScale AI’s leaderboard
HARD-51 (51 tasks)CognitionRicursive IntelligenceThinking MachinesScale AI’s leaderboard

Bold: Benchmark default based on README · Download image

Results

Show
020406080100Jul 2026Oct 202699.40%Opus 597.7%Inkling 2 Small99.22%Gemini 4 Argon96.90%GPT-6 Astra93.15%Sonnet 5020406080100Jul 2026Oct 202699.40%Opus 597.7%Inkling 2 Small96.90%GPT-6 Astra
  • OpenAI
  • Anthropic
  • Google
  • Other labs
  • On the creator’s leaderboard
ModelHarness and effortScoreSource
Claude Opus 524 Jul 2026Claude Codexhigh effort99.40%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Gemini 4 Argon30 Sep 2026mini-SWE-agenthigh effort99.22%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Claude Fable 5.11 Sep 2026Claude Codehigh effort99.10%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Inkling 2 Small5 Nov 2026mini-SWE-agenteffort 0.99 (xhigh)97.7%Introducing Inkling 2 SmallCorroborated The same number is printed in 2 other documents: 1 2Run by Scale AI · 1M-token trajectory limitOur capture
Inkling 2 Small5 Nov 2026Harness not specifiedEffort not specified97.66%SWE-Bench Pro (Public Dataset) V2 LeaderboardCorroborated The same number is printed in 1 other document: 1Scale AI’s leaderboard, 18 Nov 2026 · another report
Kimi K316 Jul 2026mini-SWE-agentmax effort97.70%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
GPT-6 Astra3 Sep 2026Codexhigh effort96.90%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
MiniMax M3.1 Flash19 Oct 2026mini-SWE-agentxhigh effort96.57%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
GPT-6.1 Sol29 Sep 2026mini-SWE-agentxhigh effort96.11%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Claude Sonnet 5.528 Sep 2026mini-SWE-agentxhigh effort95.79%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
GLM-5.314 Aug 2026mini-SWE-agentmax effort95.60%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
GPT-5.6 Sol9 Jul 2026Codexxhigh effort95.50%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Gemini 3.8 Flash2 Sep 2026mini-SWE-agenthigh effort94.86%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Claude Sonnet 530 Jun 2026Claude Codexhigh effort93.15%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
GPT-5.6 Terra9 Jul 2026Codexxhigh effort92.37%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Inkling15 Jul 2026mini-SWE-agentxhigh effort89.88%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Claude Haiku 5.57 Oct 2026mini-SWE-agenthigh effort87.23%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026

All benchmarks