Frontier Log

Gemini 4 Carbon vs GPT-6 Astra

Gemini 4 CarbonGPT-6 Astra
Released
Input / 1M tokens$2$10
Cached input / 1M tokens$0.10$1.00
Output / 1M tokens$10$50
Context windowNo data1,050,000
Output limitNo data128,000

Both labs report

Terminal-Bench 4.0

Gemini 4 Carbon65.2%Run by Artificial Analysis · mini-swe-agent harness; high thinking; average of 3 runs
GPT-6 Astra57.9%high effort

Terminal-Bench Science 0.1

Gemini 4 Carbon64.8%Run by Artificial Analysis · mini-swe-agent harness; high thinking; average of 3 runs
GPT-6 Astra64.6%max effort

AutomationBench 1.0.6Private set

Gemini 4 Carbon53.9%Run by Zapier · set named in the evaluation methodology the post cites
GPT-6 Astra41.4%Run by Zapier · highest score at any effort; set named in the GPT-6.1 Sol post, which prints the same figure

OSWorld 2.1Offline set, partial score

Gemini 4 Carbon81.5%82-task offline subset, no internet access; highest thinking; max of 3 runs, one attempt per run
GPT-6 Astra80.9%max effort

Reported for one model

Humanity's Last ExamWith tools, set not stated

Gemini 4 CarbonNo data
GPT-6 Astra57.2%

GPQA Diamond

Gemini 4 CarbonNo data
GPT-6 Astra96.0%

BrowseComp

Gemini 4 CarbonNo data
GPT-6 Astra91.5%

ARC-AGI-2

Gemini 4 CarbonNo data
GPT-6 Astra95.0%

OSWorld 2.0 (v2026.08.08)Offline set, partial score

Gemini 4 CarbonNo dataReported for another version: OSWorld 2.1, Offline set, partial score
GPT-6 Astra72.6%

FrontierCode 1.1Main set

Gemini 4 CarbonNo data
GPT-6 Astra53.3%

FrontierCode 1.1Extended set

Gemini 4 CarbonNo data
GPT-6 Astra64.5%

DeepSWE v1.1

Gemini 4 CarbonNo data
GPT-6 Astra74.1%

ARC-AGI-1Set not stated

Gemini 4 CarbonNo data
GPT-6 Astra98.5%

ARC-AGI-3Set not stated

Gemini 4 CarbonNo data
GPT-6 Astra99.9%

ChartographyNo tools

Gemini 4 Carbon72.5%Run by Surge AI · Surge AI's public leaderboard
GPT-6 AstraNo data

SEC-bench Pro 260505

Gemini 4 Carbon88.2%183-task JavaScript engine split (May 2026); pass@1, mean of 3 attempts; Antigravity harness; LLM judge for crashes in both builds
GPT-6 AstraNo dataReported for another version: SEC-bench Pro, no release named

SEC-bench Pro, no release named

Gemini 4 CarbonNo dataReported for another version: SEC-bench Pro 260505
GPT-6 Astra85.4%

Agents' Last Exam v1Measure not stated

Gemini 4 CarbonNo dataReported for another version: Agents' Last Exam v1.1, Score
GPT-6 Astra59.3%best of any effort; harness not stated

Agents' Last Exam v1.1Score

Gemini 4 Carbon77.4%122 tasks; ALE-Claw harness; 5-hour window; October 17, 2026 task files; safety filters on
GPT-6 AstraNo dataReported for another version: Agents' Last Exam v1, Measure not stated

MMMU-ProVision setting

Gemini 4 CarbonNo data
GPT-6 Astra86.7%max effort

Related

All comparisons