Benchmark · Active
Humanity's Last Exam Created by Center for AI Safety and Scale AI , 23 Jan 2025 · Higher is better · Website
Version HLE HLE-Diamond Setting No tools, full set (2,500 questions) With tools, full set (2,500 questions) No tools, text-only subset With tools, text-only subset No tools, set not stated With tools, set not stated Full set (2,500 questions), tools not stated Text-only subset, tools not stated Tools and set not stated 0 10 20 30 40 50 60 44.1% Inkling 2 Small 0 10 20 30 40 50 60 44.1% Inkling 2 Small Settings and run setups Task set Tools Providers Full set (2,500 questions) No tools OpenAI Anthropic Google Meta Moonshot AI Full set (2,500 questions) With tools OpenAI Anthropic Google Meta Moonshot AI Z.ai Meituan Tencent Text-only subset No tools OpenAI Meta DeepSeek Alibaba Moonshot AI MiniMax Z.ai NVIDIA Tencent Thinking Machines Text-only subset With tools DeepSeek Moonshot AI Z.ai NVIDIA Tencent Not stated No tools OpenAI Google DeepSeek Alibaba MiniMax Z.ai NVIDIA Not stated With tools OpenAI DeepSeek Alibaba Z.ai NVIDIA Thinking Machines Full set (2,500 questions) Not stated NVIDIA Text-only subset Not stated Thinking Machines Not stated Not stated DeepSeek Alibaba
Download image
All benchmarks