Frontier Log

NVIDIAModel

Nemotron 3 Ultra 550B-A55B

Made by
NVIDIA
Released
Announced in
NVIDIA Nemotron 3 Ultra Powers Faster, More Efficient Reasoning for Long-Running Agentsdeveloper.nvidia.com
Event
Nemotron 3 Ultra
API record
/api/v1/models/nemotron-3-ultra-550b-a55b

Benchmark results

BenchmarkScoreSource check
Humanity's Last ExamNo tools26.7%
CorroboratedThe same number is printed in 2 other lab documents: 1 2
text-only subset; GPT-5.6 Luna judge · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI29.24%
CorroboratedThe same number is printed in 2 other lab documents: 1 2
Humanity's Last ExamWith tools37.4%
CorroboratedThe same number is printed in 2 other lab documents: 1 2
SWE-bench Verified71.9%
CheckedA person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
in Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning70.7%
CorroboratedThe same number is printed in 3 other lab documents: 1 2 3
OpenCode harness; web search and fetch disabled; average of 3 runs · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI73.93%
CorroboratedThe same number is printed in 1 other lab document: 1
SWE-bench ProOpenCode harness; web search and fetch disabled; average of 3 runs · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI · The source doesn't say whether this is the public or the commercial set49.66%
CorroboratedThe same number is printed in 2 other lab documents: 1 2
GPQA Diamondaverage of 8 runs · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI86.99%
CorroboratedThe same number is printed in 3 other lab documents: 1 2 3
BrowseCompinternal search harness44.4%
CorroboratedThe same number is printed in 2 other lab documents: 1 2
NeMo Gym harness with Exa search; discard-all context management · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI67.85%
CorroboratedThe same number is printed in 2 other lab documents: 1 2
OpenCode harness · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI61.06%
CheckedA person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
Terminal-Bench 2.054%
CorroboratedThe same number is printed in 1 other lab document: 1
Terminal-Bench 2.156.4%
CorroboratedThe same number is printed in 3 other lab documents: 1 2 3
Terminus 2 harness; 3h timeout; average of 8 runs · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI54.78%
CorroboratedThe same number is printed in 1 other lab document: 1
later report; harness not stated · reported later, in nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 model card (revision of 2026-10-15)54.80%
CheckedA person checked this number against its source on 20 Oct 2026. This confirms the number matches what the lab published, not that the score was reproduced.
MMLU-Pro86.8%
CorroboratedThe same number is printed in 1 other lab document: 1
LiveCodeBenchLiveCodeBench v689.0%
CorroboratedThe same number is printed in 1 other lab document: 1
LiveCodeBench v6, problems from Aug 2024 to May 2025; average of 3 samples · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI89.21%
CorroboratedThe same number is printed in 1 other lab document: 1
τ²-benchTelecom · average of 8 trials; Qwen3-235B-A22B-Instruct-2507 user simulator · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI83.00%
CorroboratedThe same number is printed in 1 other lab document: 1
τ²-benchRetail · average of 8 trials; Qwen3-235B-A22B-Instruct-2507 user simulator · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI73.68%
CorroboratedThe same number is printed in 1 other lab document: 1
τ²-benchAirline · average of 8 trials; Qwen3-235B-A22B-Instruct-2507 user simulator · reported later, in Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI79.75%
CorroboratedThe same number is printed in 1 other lab document: 1
Sources

Reported by NVIDIA

Events

Stories

All models