AutomationBench 1.0.6
Created by Zapier, 31 Jul 2026 · Higher is better · Website
- Other labs
- Reported in a later post or card
Results
| Model | Score | Source |
|---|---|---|
| MiniMax M3.1 Flash19 Oct 2026 · MiniMax | 61.2% | MiniMax M3.1 Flash: Redefining Agent Performance and CostChecked A person checked this number against its source on 26 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.xhigh effort; AutomationBench v1.0.6; Claude Code harnessSource image |
| MiniMax M3.1 Flash19 Oct 2026 · MiniMax | 52.8% | MiniMax M3.1 Flash: Redefining Agent Performance and CostChecked A person checked this number against its source on 26 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.low effort; AutomationBench v1.0.6; Claude Code harness; another report, not chartedSource image |
| MiniMax M3.1 Flash19 Oct 2026 · MiniMax | 57.7% | MiniMax M3.1 Flash: Redefining Agent Performance and CostChecked A person checked this number against its source on 26 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.medium effort; AutomationBench v1.0.6; Claude Code harness; another report, not chartedSource image |
| MiniMax M3.1 Flash19 Oct 2026 · MiniMax | 59.2% | MiniMax M3.1 Flash: Redefining Agent Performance and CostChecked A person checked this number against its source on 26 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.high effort; AutomationBench v1.0.6; Claude Code harness; another report, not chartedSource image |
| DeepSeek-V4.1-Flash-101717 Oct 2026 · DeepSeek | 60.5% | DeepSeek-V4.1-Flash-1017: Wish-Driven Programming Made RealCorroborated The same number is printed in 1 other lab document: 1AutomationBench v1.0.6Source image |
| GLM-5.3-Flash26 Aug 2026 · Z.ai | 48.8% | GLM-5.3-Flash: Frontier Intelligence, Flash CostCorroborated The same number is printed in 3 other lab documents: 1 2 3AutomationBench v1.0.6 with a fix for null-type handlingOur capture |
| GLM-5.314 Aug 2026 · Z.ai | 48.2% | GLM-5.3: Frontier Coding with Emergent Cyber CapabilitiesChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.AutomationBench v1.0.6 with a fix for null-type handlingOur capture |
| Hy4 preview28 Aug 2026 · Tencent | 32.1% | Introducing Hy4 previewChecked A person checked this number against its source on 8 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.AutomationBench v1.0.6Source image |
| GLM-5.216 Jun 2026 · Z.ai | 26.2% | GLM-5.3: Frontier Coding with Emergent Cyber CapabilitiesChecked A person checked this number against its source on 19 Oct 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.AutomationBench v1.0.6; reported laterOur capture |
| Hy36 Jul 2026 · Tencent | 16.1% | Introducing Hy4 previewChecked A person checked this number against its source on 7 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.AutomationBench v1.0.6; highest reasoning setting; reported laterSource image |
