Frontier Log

Claude Sonnet 5.5 vs Gemini 4 Argon

Claude Sonnet 5.5Gemini 4 Argon
Announced
Released
Input / 1M tokens$2$2$4 from 1 Dec 2026
Cached input / 1M tokens$0.10$0.10$0.20 from 1 Dec 2026
Output / 1M tokens$10$10$20 from 1 Dec 2026
Context window1M1M
Output limit128K65K
Benchmarks
Terminal-Bench 4.070.6%157.4%
Terminal-Bench Science 0.159.9%257.6%
AutomationBench 1.0.6 Private set44.7%351.3%4
OSWorld 2.1 Offline set (82 tasks), partial score83.9%574.7%6
FrontierSWE v261.9%755.0%8
DeepSWE v1.171.0%977.9%
Chartography No tools61.6%71.6%10

Download image

Claude Sonnet 5.5Gemini 4 Argon
Other benchmarks
Humanity's Last Exam No tools, full set (2,500 questions)56.9%No data
Humanity's Last Exam With tools, full set (2,500 questions)64.5%No data
SWE-bench Pro Set not stated81.3%11No data
OSWorld 2.1 Full set (108 tasks), partial score80.1%12No data
OSWorld 2.1 Full set (108 tasks), strict pass rate43.5%13No data
OSWorld 2.1 Offline set (82 tasks), strict pass rate48.8%14No data
OSWorld 2.0 (v2026.08.08) Offline set, partial scoreNo data69.2%15
FrontierCode 1.1 Main set52.1%16No data
FrontierCode 1.1 Extended set64.4%17No data
Chartography With tools90.2%18No data
SEC-bench Pro 260505No data83.4%19
Agents' Last Exam v1 Pass rate, set not statedNo data39.5%20
Agents' Last Exam v1.1 Score, all tasks (122)No data69.0%21
GraphWalks BFS, up to 128KNo data99.7%22
GraphWalks BFS, 256K + 1M problemsNo data84.2%

Notes

  1. Terminal-Bench 4.0, Claude Sonnet 5.5: adaptive thinking
  2. Terminal-Bench Science 0.1, Claude Sonnet 5.5: Claude Code in --bare mode · max effort · standard error ±4.7 points
  3. AutomationBench 1.0.6, Private set, Claude Sonnet 5.5: Run by Zapier · max effort · default fallbacks on
  4. AutomationBench 1.0.6, Private set, Gemini 4 Argon: Run by Zapier · set named in the evaluation methodology the post cites
  5. OSWorld 2.1, Offline set (82 tasks), partial score, Claude Sonnet 5.5: max effort · mean of 5 attempts
  6. OSWorld 2.1, Offline set (82 tasks), partial score, Gemini 4 Argon: Gemini CUA harness · highest thinking · max of 3 runs, one attempt per run
  7. FrontierSWE v2, Claude Sonnet 5.5: Run by Proximal · Proximal's agent harness · max reasoning effort · mean of 5 trials per task · 20-hour budget
  8. FrontierSWE v2, Gemini 4 Argon: Run by Proximal · Proximal's public leaderboard
  9. DeepSWE v1.1, Claude Sonnet 5.5: average of 5 trials
  10. Chartography, No tools, Gemini 4 Argon: Run by Surge AI · Surge AI's public leaderboard
  11. SWE-bench Pro, Set not stated, Claude Sonnet 5.5: max effort · adaptive thinking · average of 5 trials
  12. OSWorld 2.1, Full set (108 tasks), partial score, Claude Sonnet 5.5: max effort · mean of 5 runs · September 10, 2026 task files
  13. OSWorld 2.1, Full set (108 tasks), strict pass rate, Claude Sonnet 5.5: max effort · mean of 5 runs · September 10, 2026 task files
  14. OSWorld 2.1, Offline set (82 tasks), strict pass rate, Claude Sonnet 5.5: max effort · mean of 5 attempts
  15. OSWorld 2.0 (v2026.08.08), Offline set, partial score, Gemini 4 Argon: Gemini CUA harness · max of 3 runs, one attempt per run
  16. FrontierCode 1.1, Main set, Claude Sonnet 5.5: Run by Cognition · Claude Code harness · xhigh effort
  17. FrontierCode 1.1, Extended set, Claude Sonnet 5.5: Run by Cognition · Claude Code harness · xhigh effort
  18. Chartography, With tools, Claude Sonnet 5.5: max effort · adaptive thinking · average of 5 runs · container and image-cropping tool · Gemini 3.5 Flash grader
  19. SEC-bench Pro 260505, Gemini 4 Argon: Antigravity harness · pass@1, mean of 3 attempts · 183-task JavaScript engine split (May 2026) · LLM judge for crashes in both builds
  20. Agents' Last Exam v1, Pass rate, set not stated, Gemini 4 Argon: ALE-Claw harness · 5-hour window · safety filters on
  21. Agents' Last Exam v1.1, Score, all tasks (122), Gemini 4 Argon: ALE-Claw harness · 5-hour window · October 17, 2026 task files · safety filters on
  22. GraphWalks, BFS, up to 128K, Gemini 4 Argon: 650 problems up to 128K tokens, per the model evaluation document

Related

All comparisons