Frontier Log

AnthropicModel

Claude Opus 4.8

Made by
Anthropic
Released
Announced in
Introducing Claude Opus 4.8anthropic.com
Event
Claude Opus 4.8
API record
/api/v1/models/claude-opus-4-8

Benchmark results

BenchmarkScoreSource check
Humanity's Last ExamNo tools49.8%
CheckedA person checked this number against its source on 16 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 7 other lab documents, the first 3 linked: 1 2 3
Humanity's Last ExamWith tools57.9%
CheckedA person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 8 other lab documents, the first 3 linked: 1 2 3
SWE-bench Verified88.6%
CheckedA person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 1 other lab document: 1
SWE-bench ProThe source doesn't say whether this is the public or the commercial set69.2%
CheckedA person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 7 other lab documents, the first 3 linked: 1 2 3
GPQA Diamondaveraged over 25 trials93.6%
CheckedA person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 1 other lab document: 1
BrowseCompsingle agent; adaptive thinking, max effort; 10M-token limit, compaction at 200K tokens84.3%
CheckedA person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 5 other lab documents, the first 3 linked: 1 2 3
ARC-AGI-2reported later, in Claude Opus 5 System Card72.1%
CheckedA person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
AutomationBenchZapier leaderboard, private held-out set; max effort15.5%
CorroboratedThe same number is printed in 2 other lab documents: 1 2
Zapier leaderboard, private held-out set; max effort · reported later, in Introducing Claude Opus 517.0%
CorroboratedThe same number is printed in 1 other lab document: 1
DeepSWE v1.1reported later, in Introducing Claude Opus 559.0%
CheckedA person checked this number against its source on 17 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 3 other lab documents: 1 2 3
Terminal-Bench 2.1Terminus 2 harness74.6%
CheckedA person checked this number against its source on 18 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 1 other lab document: 1
reported later, in Introducing Claude Sonnet 582.7%
CheckedA person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
OSWorld-Verifiedrun method changed; may not be comparable with earlier Anthropic OSWorld-Verified scores83.4%
CheckedA person checked this number against its source on 18 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
CorroboratedThe same number is printed in 5 other lab documents, the first 3 linked: 1 2 3
ARC-AGI-3verified by the ARC Prize Foundation; high effort; semi-private set · reported later, in Introducing Claude Opus 51.5%
CorroboratedThe same number is printed in 1 other lab document: 1
Sources

Reported by Anthropic

Events

Stories

All models