{
  "api_version": "1",
  "updated": "2026-10-18",
  "latest_event": "2026-10-17",
  "rights": "© Frontier Log, all rights reserved.",
  "benchmark": {
    "slug": "aime-2025",
    "name": "AIME 2025",
    "family": "aime",
    "unit": "%",
    "higher_is_better": true,
    "status": "superseded",
    "benchmark_url": "https://maa.org/amc/aime",
    "introduced": "2025-02-12",
    "creator": "Mathematical Association of America",
    "introduced_source": "https://maa.org/news/aime-thresholds-are-available/",
    "settings": [
      {
        "setting": "no-tools",
        "name": "No tools",
        "url": "https://frontierlog.ai/benchmark/aime-2025",
        "results": 11
      },
      {
        "setting": "with-tools",
        "name": "With tools",
        "url": "https://frontierlog.ai/benchmark/aime-2025/with-tools",
        "results": 4
      }
    ]
  },
  "total": 15,
  "results": [
    {
      "id": "gemini-2-5-pro:aime-2025:no-tools:1",
      "model": "gemini-2-5-pro",
      "model_name": "Gemini 2.5 Pro",
      "model_date": "2025-03-25",
      "event": "https://frontierlog.ai/event/2025-03-25-gemini-2-5-pro",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 86.7,
      "unit": "%",
      "printed": "86.7%",
      "reported_by": "google",
      "label": "Gemini 2.5 Pro Experimental (03-25)",
      "source": "https://blog.google/innovation-and-ai/models-and-research/google-deepmind/gemini-model-thinking-updates-march-2025/",
      "source_title": "Gemini 2.5: Our most intelligent AI model",
      "source_date": "2025-03-25",
      "where": "benchmark table image (gemini_benchmarks_cropped GIF), row 'Mathematics AIME 2025', sub-row 'single attempt (pass@1)', column 'Gemini 2.5 Pro Experimental (03-25)'",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Methodology under the table: all Gemini 2.5 Pro scores are pass@1 (no majority voting or parallel test-time compute), run via AI Studio API model gemini-2.5-pro-exp-03-25 with default sampling, averaged over multiple trials for smaller benchmarks. No tools printed. The 'multiple attempts' sub-row is a dash for Gemini.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18",
      "source_image": "https://frontierlog.ai/sources/a8f4323681d1a15d.gif",
      "source_image_original": "https://storage.googleapis.com/gweb-uniblog-publish-prod/original_images/gemini_benchmarks_cropped_light2x_1PPmDuP.gif"
    },
    {
      "id": "openai-o3:aime-2025:no-tools:1",
      "model": "openai-o3",
      "model_name": "OpenAI o3",
      "model_date": "2025-04-16",
      "event": "https://frontierlog.ai/event/2025-04-16-o3-and-o4-mini",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 88.9,
      "unit": "%",
      "printed": "88.9",
      "reported_by": "openai",
      "label": "o3 (no tools)",
      "source": "https://openai.com/index/introducing-o3-and-o4-mini",
      "source_title": "Introducing OpenAI o3 and o4-mini",
      "source_date": "2025-04-16",
      "where": "'What's changed' section, bar chart 'AIME 2025 / Competition Math', bar labelled 'o3 (no tools)' (3rd bar)",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Read from the full-page screenshot (shots.jsonl part 2) of the HTML bar chart; value label above the bar. Post note under the chart set: 'All models are evaluated at high reasoning effort settings, similar to variants like o4-mini-high in ChatGPT.' Printed without % (y-axis 'Accuracy (%)'). A cost-vs-score image (o3-o4_AIME2025.png, 'AIME 2025, no tools' vs estimated inference cost, o3 low/medium/high) has no value labels and is not recorded. Read from a full-page screenshot of the post (para-kb image /api/images/79), as openai.com is bot-walled.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "openai-o3:aime-2025:with-tools:1",
      "model": "openai-o3",
      "model_name": "OpenAI o3",
      "model_date": "2025-04-16",
      "event": "https://frontierlog.ai/event/2025-04-16-o3-and-o4-mini",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "with-tools",
      "setting_name": "With tools",
      "score": 98.4,
      "unit": "%",
      "printed": "98.4%",
      "reported_by": "openai",
      "label": "o3",
      "source": "https://openai.com/index/introducing-o3-and-o4-mini",
      "source_title": "Introducing OpenAI o3 and o4-mini",
      "source_date": "2025-04-16",
      "where": "'What's changed' section, paragraph on o4-mini, last sentence (o3 with tool use)",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Sentence in the text; tool is a Python interpreter (the sentence compares to o4-mini 'when given access to a Python interpreter'). pass@1. The post says these with-tools results should not be compared to models without tool access.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "openai-o3:aime-2025:with-tools:2",
      "model": "openai-o3",
      "model_name": "OpenAI o3",
      "model_date": "2025-04-16",
      "event": "https://frontierlog.ai/event/2025-04-16-o3-and-o4-mini",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "with-tools",
      "setting_name": "With tools",
      "score": 100.0,
      "unit": "%",
      "printed": "100%",
      "reported_by": "openai",
      "label": "o3",
      "source": "https://openai.com/index/introducing-o3-and-o4-mini",
      "source_title": "Introducing OpenAI o3 and o4-mini",
      "source_date": "2025-04-16",
      "where": "'What's changed' section, paragraph on o4-mini, last sentence (o3 with tool use), second figure in the parenthesis",
      "own": true,
      "primary": false,
      "parallel": true,
      "notes": "consensus@8 (majority voting over 8 samples), with a Python interpreter.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "claude-opus-4:aime-2025:no-tools:1",
      "model": "claude-opus-4",
      "model_name": "Claude Opus 4",
      "model_date": "2025-05-22",
      "event": "https://frontierlog.ai/event/2025-05-22-claude-4",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 75.5,
      "unit": "%",
      "printed": "75.5%",
      "reported_by": "anthropic",
      "label": "Claude Opus 4",
      "source": "https://www.anthropic.com/news/claude-4",
      "source_title": "Introducing Claude 4",
      "source_date": "2025-05-22",
      "where": "main benchmark table, row 'High school math competition / AIME 2025', column 'Claude Opus 4', first number of the cell",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Extended thinking (up to 64K tokens) per appendix; footnote 4: nucleus sampling, top_p 0.95. No tools stated.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "claude-opus-4:aime-2025:no-tools:2",
      "model": "claude-opus-4",
      "model_name": "Claude Opus 4",
      "model_date": "2025-05-22",
      "event": "https://frontierlog.ai/event/2025-05-22-claude-4",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 90.0,
      "unit": "%",
      "printed": "90.0%",
      "reported_by": "anthropic",
      "label": "Claude Opus 4",
      "source": "https://www.anthropic.com/news/claude-4",
      "source_title": "Introducing Claude 4",
      "source_date": "2025-05-22",
      "where": "main benchmark table, row 'AIME 2025', column 'Claude Opus 4', second number of the cell (after the slash)",
      "own": true,
      "primary": false,
      "parallel": true,
      "notes": "Footnote 5: parallel test-time compute with internal scoring model. Extended thinking up to 64K.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "claude-opus-4:aime-2025:no-tools:3",
      "model": "claude-opus-4",
      "model_name": "Claude Opus 4",
      "model_date": "2025-05-22",
      "event": "https://frontierlog.ai/event/2025-05-22-claude-4",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 33.9,
      "unit": "%",
      "printed": "33.9%",
      "reported_by": "anthropic",
      "label": "Opus 4",
      "source": "https://www.anthropic.com/news/claude-4",
      "source_title": "Introducing Claude 4",
      "source_date": "2025-05-22",
      "where": "appendix 'Performance benchmark reporting', extended-thinking list, AIME line",
      "own": true,
      "primary": false,
      "parallel": false,
      "notes": "Without extended thinking. Printed as 'AIME' (the table row is AIME 2025).",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "claude-opus-4-1:aime-2025:no-tools:1",
      "model": "claude-opus-4-1",
      "model_name": "Claude Opus 4.1",
      "model_date": "2025-08-05",
      "event": "https://frontierlog.ai/event/2025-08-05-claude-opus-4-1",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 78.0,
      "unit": "%",
      "printed": "78.0%",
      "reported_by": "anthropic",
      "label": "Claude Opus 4.1",
      "source": "https://www.anthropic.com/news/claude-opus-4-1",
      "source_title": "Claude Opus 4.1",
      "source_date": "2025-08-05",
      "where": "benchmark table image, row 'High school math competition / AIME 2025', column 'Claude Opus 4.1'",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Extended thinking up to 64K tokens (appendix); footnote 4: nucleus sampling, top_p 0.95.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18",
      "source_image": "https://frontierlog.ai/sources/cebed70e8721d155.png",
      "source_image_original": "https://www-cdn.anthropic.com/images/4zrzovbb/website/bde326699c667506c87f74b09a6355961d29eb26-2600x2084.png"
    },
    {
      "id": "gpt-5:aime-2025:no-tools:1",
      "model": "gpt-5",
      "model_name": "GPT-5",
      "model_date": "2025-08-07",
      "event": "https://frontierlog.ai/event/2025-08-07-gpt-5",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 94.6,
      "unit": "%",
      "printed": "94.6%",
      "reported_by": "openai",
      "label": "GPT‑5",
      "source": "https://openai.com/index/introducing-gpt-5",
      "source_title": "Introducing GPT-5",
      "source_date": "2025-08-07",
      "where": "'Evaluations' section, first paragraph (state of the art across math)",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Sentence in the text. The same value is the 'With thinking' segment of the 'GPT-5 (no tools)' bar in the AIME 2025 chart below it (screenshot part 3, kb:/api/images/97). Chart y-axis 'Accuracy, pass@1'.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "gpt-5:aime-2025:no-tools:2",
      "model": "gpt-5",
      "model_name": "GPT-5",
      "model_date": "2025-08-07",
      "event": "https://frontierlog.ai/event/2025-08-07-gpt-5",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 61.9,
      "unit": "%",
      "printed": "61.9%",
      "reported_by": "openai",
      "label": "GPT-5 (no tools), Without thinking",
      "source": "https://openai.com/index/introducing-gpt-5",
      "source_title": "Introducing GPT-5",
      "source_date": "2025-08-07",
      "where": "'Evaluations' section, bar chart 'AIME 2025 / Competition math', 4th bar 'GPT-5 (no tools)', lower (light) segment",
      "own": true,
      "primary": false,
      "parallel": false,
      "notes": "Read from the full-page screenshot (shots.jsonl part 3, image 97; the bar's x-axis label is at the top of part 4, kb:/api/images/98, sha256 f036431e418305f0b45d54e5179176b075413da4a646a81867a80b28bd140ddb). Chart y-axis 'Accuracy, pass@1'; legend: dark = With thinking, light = Without thinking; stacked bar, value label on each segment. Footnote under the chart: '*AIME results with tools should not be compared directly to the performance of models without tool access; they are an example of how effectively GPT-5 leverages available tools.' This is the without-thinking value; the with-thinking value of the same bar is 94.6% (primary row). Read from a full-page screenshot of the post (para-kb image /api/images/97), as openai.com is bot-walled.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "gpt-5:aime-2025:with-tools:1",
      "model": "gpt-5",
      "model_name": "GPT-5",
      "model_date": "2025-08-07",
      "event": "https://frontierlog.ai/event/2025-08-07-gpt-5",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "with-tools",
      "setting_name": "With tools",
      "score": 99.6,
      "unit": "%",
      "printed": "99.6%",
      "reported_by": "openai",
      "label": "GPT-5 (python), With thinking",
      "source": "https://openai.com/index/introducing-gpt-5",
      "source_title": "Introducing GPT-5",
      "source_date": "2025-08-07",
      "where": "'Evaluations' section, bar chart 'AIME 2025 / Competition math', 3rd bar 'GPT-5 (python)', top (dark) segment",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Read from the full-page screenshot (shots.jsonl part 3, image 97; the bar's x-axis label is at the top of part 4, kb:/api/images/98, sha256 f036431e418305f0b45d54e5179176b075413da4a646a81867a80b28bd140ddb). Chart y-axis 'Accuracy, pass@1'; legend: dark = With thinking, light = Without thinking; stacked bar, value label on each segment. Footnote under the chart: '*AIME results with tools should not be compared directly to the performance of models without tool access; they are an example of how effectively GPT-5 leverages available tools.' Tool: Python. Read from a full-page screenshot of the post (para-kb image /api/images/97), as openai.com is bot-walled.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "gpt-5:aime-2025:with-tools:2",
      "model": "gpt-5",
      "model_name": "GPT-5",
      "model_date": "2025-08-07",
      "event": "https://frontierlog.ai/event/2025-08-07-gpt-5",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "with-tools",
      "setting_name": "With tools",
      "score": 71.0,
      "unit": "%",
      "printed": "71.0%",
      "reported_by": "openai",
      "label": "GPT-5 (python), Without thinking",
      "source": "https://openai.com/index/introducing-gpt-5",
      "source_title": "Introducing GPT-5",
      "source_date": "2025-08-07",
      "where": "'Evaluations' section, bar chart 'AIME 2025 / Competition math', 3rd bar 'GPT-5 (python)', lower (light) segment",
      "own": true,
      "primary": false,
      "parallel": false,
      "notes": "Read from the full-page screenshot (shots.jsonl part 3, image 97; the bar's x-axis label is at the top of part 4, kb:/api/images/98, sha256 f036431e418305f0b45d54e5179176b075413da4a646a81867a80b28bd140ddb). Chart y-axis 'Accuracy, pass@1'; legend: dark = With thinking, light = Without thinking; stacked bar, value label on each segment. Footnote under the chart: '*AIME results with tools should not be compared directly to the performance of models without tool access; they are an example of how effectively GPT-5 leverages available tools.' Tool: Python. Without-thinking value. Read from a full-page screenshot of the post (para-kb image /api/images/97), as openai.com is bot-walled.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    },
    {
      "id": "deepseek-v3-2-exp:aime-2025:no-tools:1",
      "model": "deepseek-v3-2-exp",
      "model_name": "DeepSeek-V3.2-Exp",
      "model_date": "2025-09-29",
      "event": "https://frontierlog.ai/event/2025-09-29-deepseek-v3-2-exp",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 89.3,
      "unit": "%",
      "printed": "89.3",
      "reported_by": "deepseek",
      "label": "DeepSeek-V3.2-Exp",
      "source": "https://api-docs.deepseek.com/news/news250929/",
      "source_title": "Introducing DeepSeek-V3.2-Exp",
      "source_date": "2025-09-29",
      "where": "benchmark table image, section 'Math', row 'AIME 2025', column 'DeepSeek-V3.2-Exp'",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "No tools or metric printed; no-tools by default.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18",
      "source_image": "https://frontierlog.ai/sources/4d9152a1f18ce8ee.webp",
      "source_image_original": "https://api-docs.deepseek.com/img/v3_2_benchmark.webp"
    },
    {
      "id": "deepseek-v3-2:aime-2025:no-tools:1",
      "model": "deepseek-v3-2",
      "model_name": "DeepSeek-V3.2",
      "model_date": "2025-12-01",
      "event": "https://frontierlog.ai/event/2025-12-01-deepseek-v3-2",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 93.1,
      "unit": "%",
      "printed": "93.1",
      "reported_by": "deepseek",
      "label": "DeepSeek-V3.2 Thinking",
      "source": "https://api-docs.deepseek.com/news/news251201/",
      "source_title": "DeepSeek-V3.2 Release",
      "source_date": "2025-12-01",
      "where": "reasoning table image (v3.2_251201_benchmark_table_en), row 'AIME 2025', column 'DeepSeek-V3.2 Thinking'",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Cell prints score with output tokens in parentheses (16k = tokens); score 93.1. No tools printed. The Speciale column (96.0) is a different model. Cell prints \"93.1(16k)\": 16k is the output token count.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18",
      "source_image": "https://frontierlog.ai/sources/a313f1f47e0fe2c0.png",
      "source_image_original": "https://api-docs.deepseek.com/img/v3.2_251201_benchmark_table_en.png"
    },
    {
      "id": "gpt-5-2:aime-2025:no-tools:1",
      "model": "gpt-5-2",
      "model_name": "GPT-5.2",
      "model_date": "2025-12-11",
      "event": "https://frontierlog.ai/event/2025-12-11-gpt-5-2",
      "benchmark": "aime-2025",
      "benchmark_name": "AIME 2025",
      "setting": "no-tools",
      "setting_name": "No tools",
      "score": 100.0,
      "unit": "%",
      "printed": "100.0%",
      "reported_by": "openai",
      "label": "GPT‑5.2 Thinking",
      "source": "https://openai.com/index/introducing-gpt-5-2",
      "source_title": "Introducing GPT-5.2",
      "source_date": "2025-12-11",
      "where": "headline eval table (HTML, first table), row 'AIME 2025 (no tools) / Competition math', column 'GPT-5.2 Thinking'",
      "own": true,
      "primary": true,
      "parallel": false,
      "notes": "Row label prints '(no tools)'. The same value is printed again in the detailed math/science table (row 'AIME 2025 (no tools)', column 'GPT-5.2 Thinking'; GPT-5.2 Pro also 100.0% there). The fetched page text was truncated, so footnotes without a % sign are not visible in the extract.",
      "caveat": "",
      "human_verified": true,
      "human_verified_on": "2026-10-18"
    }
  ]
}