Frontier Log

Benchmark · Active

AutomationBench 1.0.6

Created by Zapier, 31 Jul 2026 · Higher is better · Website

0204060Jul 2026Oct 202653.9%Gemini 4 Carbon44.7%Sonnet 5.541.4%GPT-6 Astra51.3%Gemini 4 Argon36.1%GPT-6.1 Sol33.2%GPT-6 Sol30.4%Gemini 3.7 Flash26.9%Opus 50204060Jul 2026Oct 202653.9%Gemini 4 Carbon44.7%Sonnet 5.541.4%GPT-6 Astra51.3%Gemini 4 Argon36.1%GPT-6.1 Sol

Results

ModelScoreSource
Gemini 4 Carbon28 Oct 2026 · Google53.9%Gemini 4 Carbon: our next frontier modelCorroborated The same number is printed in 1 other lab document: 1Run by Zapier · set named in the evaluation methodology the post citesSource image
Gemini 4 Argon30 Sep 2026 · Google51.3%Gemini 4 Argon: our next era of frontier intelligenceChecked A person checked this number against its source on 16 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · set named in the evaluation methodology the post citesSource image
Claude Sonnet 5.528 Sep 2026 · Anthropic44.7%Claude Sonnet 5.5 System CardChecked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · release 1.0.6; max effort; default fallbacks onOur capture
GPT-6 Astra3 Sep 2026 · OpenAI41.4%GPT-6 Astra: A new generation of intelligenceChecked A person checked this number against its source on 16 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 3 other lab documents: 1 2 3Run by Zapier · highest score at any effort; set named in the GPT-6.1 Sol post, which prints the same figureOur capture
GPT-6 Astra3 Sep 2026 · OpenAI30.3%Introducing GPT-6 Sol and LunaChecked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · low effort; AutomationBench 1.0.6; reported later; another report, not chartedOur capture
GPT-6 Astra3 Sep 2026 · OpenAI34.1%Introducing GPT-6.1 SolCorroborated The same number is printed in 1 other lab document: 1Run by Zapier · medium effort; AutomationBench 1.0.6; reported later; another report, not chartedOur capture
GPT-6 Astra3 Sep 2026 · OpenAI37.1%Introducing GPT-6.1 SolCorroborated The same number is printed in 1 other lab document: 1Run by Zapier · high effort; AutomationBench 1.0.6; reported later; another report, not chartedOur capture
GPT-6 Astra3 Sep 2026 · OpenAI39.0%Introducing GPT-6.1 SolCorroborated The same number is printed in 2 other lab documents: 1 2Run by Zapier · xhigh effort; AutomationBench 1.0.6; reported later; another report, not chartedOur capture
Claude Opus 5.522 Sep 2026 · Anthropic40.0%Introducing Claude Opus 5.5Checked A person checked this number against its source on 16 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Corroborated The same number is printed in 2 other lab documents: 1 2Run by Zapier · without fallback models; set and release 1.0.6 named in the Claude Sonnet 5.5 system card, which prints the same figureOur capture
Claude Opus 5.522 Sep 2026 · Anthropic42.5%Claude Sonnet 5.5 System CardChecked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · refused tasks re-run with default fallbacks; release 1.0.6; reported later; another report, not chartedOur capture
GPT-6.1 Sol29 Sep 2026 · OpenAI36.1%Introducing GPT-6.1 SolChecked A person checked this number against its source on 8 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · max effort; AutomationBench 1.0.6Our capture
GPT-6.1 Sol29 Sep 2026 · OpenAI24.7%Introducing GPT-6.1 SolChecked A person checked this number against its source on 8 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · low effort; AutomationBench 1.0.6; another report, not chartedOur capture
GPT-6.1 Sol29 Sep 2026 · OpenAI31.7%Introducing GPT-6.1 SolChecked A person checked this number against its source on 8 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · medium effort; AutomationBench 1.0.6; another report, not chartedOur capture
GPT-6.1 Sol29 Sep 2026 · OpenAI33.2%Introducing GPT-6.1 SolChecked A person checked this number against its source on 8 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · high effort; AutomationBench 1.0.6; another report, not chartedOur capture
GPT-6.1 Sol29 Sep 2026 · OpenAI35.5%Introducing GPT-6.1 SolChecked A person checked this number against its source on 8 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · xhigh effort; AutomationBench 1.0.6; another report, not chartedOur capture
GPT-6 Sol22 Sep 2026 · OpenAI33.2%Introducing GPT-6 Sol and LunaChecked A person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.Run by Zapier · xhigh effort; AutomationBench 1.0.6Our capture
Gemini 3.7 Flash13 Aug 2026 · Google30.4%Introducing Gemini 3.7 FlashCorroborated The same number is printed in 1 other lab document: 1Run by ZapierSource image
Claude Opus 524 Jul 2026 · Anthropic26.9%Introducing Claude Fable 5.1 and Claude Mythos 5.1Corroborated The same number is printed in 3 other lab documents: 1 2 3Run by Zapier · max effort; set named in the Claude Fable 5.1 system card, which prints the same figure; reported laterOur capture
Gemini 3.6 Flash21 Jul 2026 · Google17.0%Introducing Gemini 3.7 FlashCorroborated The same number is printed in 1 other lab document: 1Run by Zapier · reported laterSource image
Claude Sonnet 530 Jun 2026 · Anthropic10.7%Claude Sonnet 5.5 System CardCorroborated The same number is printed in 1 other lab document: 1Run by Zapier · release 1.0.6; reported laterOur capture

All benchmarks