Frontier Log

Benchmark · Active

ExploitGym

Created by Berkeley RDI, 11 May 2026 · Higher is better · Website

Settings

Task setScoringMitigationsReported by
ExploitGym 1.0 (869 tasks)ScoreMitigations offOpenAIGoogle
ExploitGym 1.0 (869 tasks)ScoreMitigations onOpenAI
ExploitGym 1.0 (869 tasks)ScoreNot specifiedOpenAIMetaDeepSeekMoonshot AI
ExploitGym 1.0 (869 tasks)Successful exploitsMitigations offAnthropicMetaZ.ai
ExploitGym 1.0 (869 tasks)Successful exploitsNot specifiedMeta

Bold: Benchmark default based on Paper and Leaderboard · Download image

Results

Show
020406070Oct 2026Jan 202754.8%Gemini 4 Carbon48.3%Gemini 4 Neon48.45%GPT-6-Cyber42.0%Gemini 4 Argon42.43%GPT-6 Astra020406070Oct 2026Jan 202754.8%Gemini 4 Carbon48.3%Gemini 4 Neon42.0%Gemini 4 Argon42.43%GPT-6 Astra
  • OpenAI
  • Google
  • Reported in a later post or card
ModelHarness and effortScoreSource
Gemini 4 Carbon28 Oct 2026AntigravityEffort not specified54.8%Gemini 4 Carbon: our next frontier modelChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.single attempt per task · no internet access · 10h time cap per task · evaluated without guardrails · mitigations off, per the model evaluation
Gemini 4 Carbon28 Oct 2026AntigravityEffort not specified53.5%Gemini 4 Carbon: our next frontier modelChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.single attempt per task · no internet access · 6h time cap per task · evaluated without guardrails · mitigations off, per the model evaluation · another reportSource image
GPT-6-Cyber5 Nov 2026Harness not specifiedEffort not specified48.45%GPT-6 Cyber System CardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.
Gemini 4 Neon18 Nov 2026AntigravityEffort not specified48.3%Introducing Gemini 4 NeonChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.single attempt per task · no internet access · 10h time cap per task · safety filters off · mitigations off (the default setting), per the model evaluationSource image
Gemini 4 Neon18 Nov 2026AntigravityEffort not specified47.9%Introducing Gemini 4 NeonChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.single attempt per task · no internet access · 6h time cap per task · safety filters off · mitigations off (the default setting), per the model evaluation · another reportSource image
GPT-6 Astra3 Sep 2026Harness not specifiedEffort not specified42.43%GPT-6 Cyber System CardChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.reported later
Gemini 4 Argon30 Sep 2026AntigravityEffort not specified42.0%Gemini 4 Carbon Model evaluation: approach, methodology & resultsCorroborated The same number is printed in 1 other document: 1single attempt per task · no internet access · 10h time cap per task, per the Gemini 4 Carbon post’s chart · reported laterOur capture
Gemini 4 Argon30 Sep 2026AntigravityEffort not specified39.5%Gemini 4 Carbon: our next frontier modelChecked A person checked this number against its source on 20 Nov 2026. This confirms the number matches what the lab published. It doesn't confirm that the score was reproduced.single attempt per task · no internet access · 6h time cap per task · mitigations off, per the model evaluation · reported later · another reportSource image

All benchmarks