Claude 3.5 Sonnet (upgraded)
Announced on anthropic.com
Benchmark results
| Benchmark | Score | Source check |
|---|---|---|
| SWE-bench Verified | 49.0% | |
| GPQA Diamond0-shot chain of thought | 65.0% | |
| AIME 2024No tools · 0-shot chain of thought | 16.0% | |
| majority vote over 64 samples; 0-shot chain of thought · parallel test-time compute · in Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet | 27.6% | CheckedA person checked this number against its source on 26 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced. |
| MMLU-Pro0-shot chain of thought | 78.0% | CorroboratedThe same number is printed in 1 other lab document: 1 |
Sources
Reported by Anthropic
- Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnetassets.anthropic.com · 22 Oct 2024
- Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haikuanthropic.com · 22 Oct 2024
Original image
