Frontier Log

Benchmark · Active

SWE-bench Pro V2

Created by Scale AI, 22 Sep 2026 · Higher is better · Website

Versions

VersionReleasedStatusModels
SWE-bench Verified13 Aug 2024Inactive100
SWE-bench Pro19 Sep 2025Active69
SWE-bench Pro V222 Sep 2026Active19

Settings

Task setReported by
Full set (642 tasks)Thinking MachinesScale AI’s leaderboard
HARD-51 (51 tasks)CognitionRicursive IntelligenceThinking MachinesScale AI’s leaderboard

Bold: Benchmark default based on README · Download image

Results

Show
020406080100Oct 2025Jan 2026Apr 2026Jul 2026Oct 202698.00%Opus 586.3%Inkling 2 Small90.20%GPT-6 Astra25.50%Haiku 4.5020406080100Oct 2025Jan 2026Apr 2026Jul 2026Oct 202698.00%Opus 586.3%Inkling 2 Small25.50%Haiku 4.5
  • OpenAI
  • Anthropic
  • Google
  • Other labs
  • On the creator’s leaderboard
ModelHarness and effortScoreSource
Claude Opus 524 Jul 2026Claude Codexhigh effort98.00%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Claude Fable 5.11 Sep 2026Claude Codehigh effort92.20%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Gemini 4 Argon30 Sep 2026mini-SWE-agenthigh effort92.16%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
GPT-6 Astra3 Sep 2026Codexhigh effort90.20%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
SWE-2 Flash4 Nov 2026Harness not specifiedEffort not specified89.9%Introducing SWE-2 FlashChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Our capture
Kimi K316 Jul 2026mini-SWE-agentmax effort88.20%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Claude Sonnet 530 Jun 2026Claude Codexhigh effort88.20%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Inkling 2 Small5 Nov 2026mini-SWE-agenteffort 0.99 (xhigh)86.3%Introducing Inkling 2 SmallCorroborated The same number is printed in 2 other documents: 1 2Run by Scale AI · 1M-token trajectory limitOur capture
Inkling 2 Small5 Nov 2026Harness not specifiedEffort not specified86.27%SWE-Bench Pro (Public Dataset) V2 LeaderboardCorroborated The same number is printed in 1 other document: 1Scale AI’s leaderboard, 18 Nov 2026 · another report
GPT-5.6 Terra9 Jul 2026Codexxhigh effort86.30%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
MiniMax M3.1 Flash19 Oct 2026mini-SWE-agentxhigh effort84.31%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
GLM-5.314 Aug 2026mini-SWE-agentmax effort84.30%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
GPT-5.6 Sol9 Jul 2026Codexxhigh effort82.40%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Ripple-65B Preview4 Nov 2026Terminus 2maximum effort per the model card79.1%Introducing Ripple-65B Preview, a Frontier Open-Weight ModelCorroborated The same number is printed in 1 other document: 1Our capture
Claude Sonnet 5.528 Sep 2026mini-SWE-agentxhigh effort70.59%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Gemini 3.8 Flash2 Sep 2026mini-SWE-agenthigh effort58.80%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Inkling15 Jul 2026mini-SWE-agentxhigh effort56.90%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Claude Haiku 5.57 Oct 2026mini-SWE-agenthigh effort52.94%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026
Claude Haiku 4.515 Oct 2025Claude Codexhigh effort25.50%SWE-Bench Pro (Public Dataset) V2 LeaderboardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Scale AI’s leaderboard, 18 Nov 2026

All benchmarks