Benchmark · InactiveSuperseded by Agents' Last Exam v1.1

Agents' Last Exam v1

Name: Agents' Last Exam v1, cli subset, pass rate: lab-reported scores
Creator: Frontier Log

Created by UC Berkeley RDI, 3 Jun 2026 · Higher is better · Website

Other labs
Reported in a later post or card

Results

Model	Score	Source	Source check
DeepSeek-V4.1-Flash-101717 Oct 2026 · DeepSeek	35.4%	DeepSeek-V4.1-Flash-1017: Wish-Driven Programming Made Realharness not statedSource image	Corroborated The same number is printed in 1 other lab document: 1
DeepSeek-V4.1-Flash10 Sep 2026 · DeepSeek	31.8%	DeepSeek-V4.1-Flash: Smarter, Faster, More EfficientCLI subset; pass@1; official ALE scaffold; max reasoning effortSource image	Corroborated The same number is printed in 4 other lab documents, the first 3 linked: 1 2 3
DeepSeek-V4-Flash-Vision-Exp21 Aug 2026 · DeepSeek	27.3%	DeepSeek-V4-Flash-Vision-Exp Release: Multimodal API Now Liveharness not statedSource image	Corroborated The same number is printed in 1 other lab document: 1
GLM-5.314 Aug 2026 · Z.ai	28.5%	GLM-5.3: Frontier Coding with Emergent Cyber Capabilities105-task CLI subset; Claude Code harness; max effort; 1M context; 4h timeout, up to 8h per task	Corroborated The same number is printed in 4 other lab documents, the first 3 linked: 1 2 3
DeepSeek-V4-Pro-081313 Aug 2026 · DeepSeek	25.7%	DeepSeek-V4-Pro GA Releaseharness not statedSource image	Corroborated The same number is printed in 3 other lab documents: 1 2 3
GLM-5.216 Jun 2026 · Z.ai	23.8%	GLM-5.3: Frontier Coding with Emergent Cyber Capabilities105-task CLI subset; Claude Code harness; reported later	Corroborated The same number is printed in 2 other lab documents: 1 2
DeepSeek-V4-Flash24 Apr 2026 · DeepSeek	15.8%	DeepSeek-V4-Pro GA Releaseharness not stated; reported laterSource image
DeepSeek-V4-Pro24 Apr 2026 · DeepSeek	16.5%	DeepSeek-V4-Pro GA Releaseharness not stated; reported laterSource image

All benchmarks