Frontier Log

Benchmark · Active

Humanity's Last Exam

Created by Center for AI Safety and Scale AI, 23 Jan 2025 · Higher is better · Website

010203040506044.1%Inkling 2 Small010203040506044.1%Inkling 2 Small

Settings and run setups

Task setToolsProviders
Full set (2,500 questions)No toolsOpenAIAnthropicGoogleMetaMoonshot AI
Full set (2,500 questions)With toolsOpenAIAnthropicGoogleMetaMoonshot AIZ.aiMeituanTencent
Text-only subsetNo toolsOpenAIMetaDeepSeekAlibabaMoonshot AIMiniMaxZ.aiNVIDIATencentThinking Machines
Text-only subsetWith toolsDeepSeekMoonshot AIZ.aiNVIDIATencent
Not statedNo toolsOpenAIGoogleDeepSeekAlibabaMiniMaxZ.aiNVIDIA
Not statedWith toolsOpenAIDeepSeekAlibabaZ.aiNVIDIAThinking Machines
Full set (2,500 questions)Not statedNVIDIA
Text-only subsetNot statedThinking Machines
Not statedNot statedDeepSeekAlibaba

Download image

Results

ModelScoreSource
Inkling 2 Small5 Nov 202644.1%Introducing Inkling 2 SmallCorroborated The same number is printed in 1 other lab document: 1Run by Artificial Analysis · effort 0.99 · 1M-token trajectory limitOur capture

All benchmarks