Frontier Log

Benchmark · Active

Humanity's Last Exam

Created by Center for AI Safety and Scale AI, 23 Jan 2025 · Higher is better · Website

010203040Oct 202629.20%Nemotron 3.5 Super VL 120B-A12B010203040Oct 202629.20%Nemotron 3.5 Super VL 120B-A12B

Settings and run setups

Task setToolsProviders
Full set (2,500 questions)No toolsOpenAIAnthropicGoogleMetaMoonshot AI
Full set (2,500 questions)With toolsOpenAIAnthropicGoogleMetaMoonshot AIZ.aiMeituanTencent
Text-only subsetNo toolsOpenAIMetaDeepSeekAlibabaMoonshot AIMiniMaxZ.aiNVIDIATencentThinking Machines
Text-only subsetWith toolsDeepSeekMoonshot AIZ.aiNVIDIATencent
Not statedNo toolsOpenAIGoogleDeepSeekAlibabaMiniMaxZ.aiNVIDIA
Not statedWith toolsOpenAIDeepSeekAlibabaZ.aiNVIDIAThinking Machines
Full set (2,500 questions)Not statedNVIDIA
Text-only subsetNot statedThinking Machines
Not statedNot statedDeepSeekAlibaba

Download image

Results

ModelScoreSource
Nemotron 3.5 Super VL 120B-A12B15 Oct 202629.20%Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AICorroborated The same number is printed in 2 other lab documents: 1 2single sample · tools not named · GPT-5.6 Luna judgeOur capture

All benchmarks