OpenAI12 Sep 2024Model

o1-preview

Benchmark results

Benchmark	Score	Source check
SWE-bench VerifiedAgentless scaffold; 5 tries to produce a patch; 3-trial average; 477 of the 500 tasks	41.4%	Checked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
reported later, in OpenAI o1 and new tools for developers	41.3%	Checked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
GPQA Diamondreported later, in OpenAI o1 and new tools for developers	73.3%	Corroborated The same number is printed in 3 other lab documents: 1 2 3
AIME 2024No tools · AIME year and pass@1 not stated · reported later, in OpenAI o1-mini	44.6%	Checked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
pass@1 · reported later, in OpenAI o1 and new tools for developers	42.0%	Checked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.

Sources

Reported by OpenAI