SWE-bench Pro
Created by Scale AI, 19 Sep 2025 · Higher is better · Website
- Other labs
- Reported in a later post or card
Results
| Model | Score | Source |
|---|---|---|
| Qwen3.8-Max3 Aug 2026 · Alibaba | 67.7% | Qwen3.8-Max: A New Bar for Coding and CoworkChecked A person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Claude Code harnessOur capture |
| Qwen3.8-Omni-Flash18 Sep 2026 · Alibaba | 63.3% | Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.Checked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Claude Code harness; 256K contextOur capture |
| Qwen3.8-Flash-Next26 Aug 2026 · Alibaba | 62.5% | Qwen3.8-Flash-Next: A New Architecture, Towards Ultimate Cost-EfficiencyCorroborated The same number is printed in 1 other lab document: 1Claude Code harness; 256K contextOur capture |
| Qwen3.7-Max20 May 2026 · Alibaba | 60.6% | Qwen3.7: The Agent FrontierChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1internal harness with bash and file-edit tools; 200K contextOur capture |
| LongCat-2.030 Jun 2026 · Meituan | 59.5% | Introducing LongCat-2.0Checked A person checked this number against its source on 23 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 2 other lab documents: 1 2Claude Code harnessOur capture |
| Qwen3.7-Plus1 Jun 2026 · Alibaba | 57.6% | Qwen3.7-Plus: Multimodal Agent IntelligenceChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.internal harness with bash and file-edit tools; 200K contextOur capture |
| Qwen3.7-Plus1 Jun 2026 · Alibaba | 55.8% | Qwen3.8-Flash-Next: A New Architecture, Towards Ultimate Cost-EfficiencyCorroborated The same number is printed in 1 other lab document: 1Claude Code harness; 256K context; reported later; another report, not chartedOur capture |
| Qwen3.6-Plus2 Apr 2026 · Alibaba | 56.6% | Qwen3.6-Plus: Towards Real World AgentsChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1internal harness with bash and file-edit tools; 200K contextOur capture |
| Qwen3.5-397B-A17B16 Feb 2026 · Alibaba | 50.9% | Qwen3.6-Plus: Towards Real World AgentsChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.internal harness with bash and file-edit tools; 200K context; reported laterOur capture |
| Qwen3.6-35B-A3B15 Apr 2026 · Alibaba | 49.5% | Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to AllChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 1 other lab document: 1internal harness with bash and file-edit tools; 200K contextOur capture |
