Frontier Log

Claude Fable 5.2 vs Gemini 4 Carbon

Claude Fable 5.2Gemini 4 Carbon
Announced
Released
Input / 1M tokens$8$2$4 from 1 Dec 2026
Cached input / 1M tokens$0.20$0.10$0.20 from 1 Dec 2026
Output / 1M tokens$40$10$20 from 1 Dec 2026
Context window1M1M
Output limit128K65K
Benchmarks
Terminal-Bench 4.067.6%165.2%2
Terminal-Bench Science 0.174.6%364.8%4
AutomationBench 1.0.6 Private set51.4%553.9%6
OSWorld 2.1 Offline set (82 tasks), partial score89.3%781.5%8
FrontierSWE v269.9%961.2%10
Chartography No tools73.4%1172.5%12

Download image

Claude Fable 5.2Gemini 4 Carbon
Other benchmarks
Humanity's Last Exam No tools, full set (2,500 questions)73.2%13No data
Humanity's Last Exam With tools, full set (2,500 questions)74.2%14No data
SWE-bench Pro Set not stated93.0%15No data
OSWorld 2.1 Offline set (82 tasks), strict pass rate54.6%16No data
DeepSWE v1.174.3%17No data
ARC-AGI-3 Semi-Private set (55 environments)79.26%18No data
Chartography With tools91.6%19No data
SEC-bench Pro 260505No data88.2%20
Agents' Last Exam v1.1 Score, all tasks (122)No data77.4%21
GraphWalks BFS, 256K + 1M problemsNo data96.2%

Notes

  1. Terminal-Bench 4.0, Claude Fable 5.2: Claude Code in --bare mode · max effort · 10 trials per task · no internet access · safeguards on
  2. Terminal-Bench 4.0, Gemini 4 Carbon: Run by Artificial Analysis · mini-swe-agent harness · high thinking · average of 3 runs
  3. Terminal-Bench Science 0.1, Claude Fable 5.2: Claude Code in --bare mode · highest available effort · averaged over 10 trials per task, per the system card · no internet access, per the system card · safeguards on, flagged requests answered by a fallback model (1.3% of requests), per the system card · standard error ±4.3 points, per the system card
  4. Terminal-Bench Science 0.1, Gemini 4 Carbon: Run by Artificial Analysis · mini-swe-agent harness · high thinking · average of 3 runs
  5. AutomationBench 1.0.6, Private set, Claude Fable 5.2: Run by Zapier · highest available effort · safeguards on · set and release 1.0.6 named in the system card, which prints the same figure
  6. AutomationBench 1.0.6, Private set, Gemini 4 Carbon: Run by Zapier · set named in the evaluation methodology the post cites
  7. OSWorld 2.1, Offline set (82 tasks), partial score, Claude Fable 5.2: highest available effort · safeguards on
  8. OSWorld 2.1, Offline set (82 tasks), partial score, Gemini 4 Carbon: Gemini CUA harness · highest thinking · max of 3 runs, one attempt per run
  9. FrontierSWE v2, Claude Fable 5.2: Run by Proximal · Proximal's agent harness · highest available effort · mean of 5 trials per task, per the system card · 20-hour budget · safeguards on
  10. FrontierSWE v2, Gemini 4 Carbon: Run by Proximal · Proximal's public leaderboard
  11. Chartography, No tools, Claude Fable 5.2: highest available effort · safeguards on
  12. Chartography, No tools, Gemini 4 Carbon: Run by Surge AI · Surge AI's public leaderboard
  13. Humanity's Last Exam, No tools, full set (2,500 questions), Claude Fable 5.2: max effort · thinking set to auto · average of 5 trials · 980k-token task budget per attempt, no context compaction · Claude Opus 4.6 as model grader
  14. Humanity's Last Exam, With tools, full set (2,500 questions), Claude Fable 5.2: highest available effort, max effort, per the system card · thinking set to auto, per the system card · average of 5 trials, per the system card · web search, web fetch, programmatic tool calling and code execution, HLE sources blocklisted, per the system card · 980k-token task budget per attempt, no context compaction, per the system card · Claude Opus 4.6 as model grader, flagged transcripts reviewed by Claude Opus 5, confirmed cheating graded incorrect, per the system card · safeguards on
  15. SWE-bench Pro, Set not stated, Claude Fable 5.2: max effort · adaptive thinking · average of 5 trials
  16. OSWorld 2.1, Offline set (82 tasks), strict pass rate, Claude Fable 5.2: max effort · mean of 5 attempts
  17. DeepSWE v1.1, Claude Fable 5.2: average of 5 trials
  18. ARC-AGI-3, Semi-Private set (55 environments), Claude Fable 5.2: Run by ARC Prize Foundation · Standard harness · high effort
  19. Chartography, With tools, Claude Fable 5.2: max effort · adaptive thinking · average of 5 runs · container and image-cropping tool · Gemini 3.5 Flash grader
  20. SEC-bench Pro 260505, Gemini 4 Carbon: Antigravity harness · pass@1, mean of 3 attempts · 183-task JavaScript engine split (May 2026) · LLM judge for crashes in both builds
  21. Agents' Last Exam v1.1, Score, all tasks (122), Gemini 4 Carbon: ALE-Claw harness · 5-hour window · October 17, 2026 task files · safety filters on

Related

All comparisons