Google: Gemini 2.5 Pro

Gemini 2.5 Pro is Google’s state-of-the-art AI model designed for advanced reasoning, coding, mathematics, and scientific tasks. It employs “thinking” capabilities, enabling it to reason through resp…

google/gemini-2.5-pro

Gemini 2.5 Pro is Google’s state-of-the-art AI model designed for advanced reasoning, coding, mathematics, and scientific tasks. It employs “thinking” capabilities, enabling it to reason through responses with enhanced accuracy and nuanced context handling. Gemini 2.5 Pro achieves top-tier performance on multiple benchmarks, including first-place positioning on the LMArena leaderboard, reflecting superior human-preference alignment and complex problem-solving abilities.

Specifications & capabilities

Input
text, image, file, audio, video
Output
text
Context window · tokens
1048576
Knowledge cutoff
2025-01
Published model ID
gemini-2.5-pro@default
Capabilities
audioInput, batch, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗

Evaluations

21–40 / 1,319

LLM Stats · GitHub archive

0.796

source id
llmStatsArchive
leaderboard name
MMMU · Normalized (reported)
dimension
general
metric
MMMU · Normalized (reported)
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMU
language
en
modality
multimodal
verified
false
max_score
1
categories
  • multimodal
  • reasoning
  • general
created_at
2025-07-19T19:56:12.130105+00:00
paper_link
https://arxiv.org/abs/2311.16502
updated_at
2025-07-19T19:56:12.130105+00:00
description
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
benchmark_id
mmmu
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMU · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/gemini-2.5-pro/benchmarks.json
archive_kind
llmStats
evidence_url
https://deepmind.google/models/gemini/pro/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/gemini-2.5-pro
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gemini-2.5-pro
Original result ↗

LLM Stats · GitHub archive

0.796

source id
llmStatsArchive
leaderboard name
MMMU · Raw
dimension
general
metric
MMMU · Raw
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMU
language
en
modality
multimodal
verified
false
max_score
1
categories
  • multimodal
  • reasoning
  • general
created_at
2025-07-19T19:56:12.130105+00:00
paper_link
https://arxiv.org/abs/2311.16502
updated_at
2025-07-19T19:56:12.130105+00:00
description
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
benchmark_id
mmmu
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMU · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/gemini-2.5-pro/benchmarks.json
archive_kind
llmStats
evidence_url
https://deepmind.google/models/gemini/pro/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/gemini-2.5-pro
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gemini-2.5-pro
Original result ↗

LLM Stats · GitHub archive

0.508

source id
llmStatsArchive
leaderboard name
SimpleQA · Raw
dimension
general
metric
SimpleQA · Raw
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
SimpleQA
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
created_at
2025-09-05T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2411.04368
updated_at
2025-09-05T00:00:00.000000+00:00
description
SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.
benchmark_id
simpleqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
SimpleQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/gemini-2.5-pro/benchmarks.json
archive_kind
llmStats
evidence_url
https://deepmind.google/models/gemini/pro/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/gemini-2.5-pro
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gemini-2.5-pro
Original result ↗

LLM Stats · GitHub archive

0.508

source id
llmStatsArchive
leaderboard name
SimpleQA · Normalized (reported)
dimension
general
metric
SimpleQA · Normalized (reported)
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
SimpleQA
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
created_at
2025-09-05T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2411.04368
updated_at
2025-09-05T00:00:00.000000+00:00
description
SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.
benchmark_id
simpleqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
SimpleQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/gemini-2.5-pro/benchmarks.json
archive_kind
llmStats
evidence_url
https://deepmind.google/models/gemini/pro/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/gemini-2.5-pro
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gemini-2.5-pro
Original result ↗

LMArena

1,303.666

source id
lmarena
leaderboard name
LMArena Vision Captioning
dimension
multimodal
metric
Arena rating
rank
3
sample size
793
evaluated at
2026-11-09T00:00:00.000Z
source model id
Gemini 2.5 Pro
confidence lower
1280.9365498576644
confidence upper
1326.395973837988
Original result ↗

Kaggle Benchmarks

77.142

source id
kaggleBenchmarks
leaderboard name
IndicGenBench · XQuAD-In
dimension
general
metric
IndicGenBench · XQuAD-In
rank
3
aggregation scope
kaggle
true
status
published
license
Apache-2.0
direction
higherIsBetter
conditions
task
id
820
name
XQuAD-In
version
3
sort_order
DESCENDING
display_type
counts
benchmark
id
212
name
IndicGenBench
type
SUITE
owner
deepmind
version
1
categories
  • multilingual and cross-lingual capabilities
version_id
191
maintainer_tier
Kaggle-maintained
maintenance_level
KAGGLE_MAINTAINED
candidate
candidate_class
modelVersion
record_kind
aggregate
attribution
Kaggle · deepmind · IndicGenBench
license_url
https://www.apache.org/licenses/LICENSE-2.0
metric_kind
IndicGenBench · XQuAD-In
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
191/leaderboard.json
source_page
https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
measurements
  • name
    XQuAD-In
    unit
    score
    scale
    1
    metric
    IndicGenBench · XQuAD-In
    result
    resultCase
    numericResult
    booleanResult
    false
    numericResult
    value
    77.14182270156253
    standardDeviation
    0
    confidenceInterval
    0
    hasStandardDeviation
    false
    hasConfidenceInterval
    false
    hasUnevenConfidenceInterval
    false
    taskVersionId
    0
    evaluationDate
    2025-10-29T00:38:04.288378Z
    hasBooleanResult
    false
    hasNumericResult
    true
    hasTaskVersionId
    false
    hasEvaluationDate
    true
    numericResultNullable
    value
    77.14182270156253
    standardDeviation
    0
    confidenceInterval
    0
    hasStandardDeviation
    false
    hasConfidenceInterval
    false
    hasUnevenConfidenceInterval
    false
    hasNumericResultPublic
    false
    customAdditionalResults
      hasNumericResultPrivate
      false
      task_id
      820
      task_url
      /benchmarks/tasks/aminmohamedmohami/xquad-in-task
      sort_order
      DESCENDING
      source_name
      XQuAD-In
      display_type
      counts
      task_version
      3
      task_display_type
      RAW_VALUES
      normalization_status
      normalized
    published_at
    2025-12-04T15:09:28.942941700Z
    candidate_class
    modelVersion
    source_revision
    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
    publisher_anchor
    google/gemini-2.5-pro
    metric_population
    14
    benchmark_evaluation
    true
    task_metadata_source
    benchmarkVersion
    candidate_metadata_source
    benchmarkVersionMappings
    evaluated at
    2025-10-29T00:38:04.288Z
    source model id
    Gemini 2.5 Pro
    Original result ↗

    LLM Stats · GitHub archive

    0.049

    source id
    llmStatsArchive
    leaderboard name
    ARC-AGI v2 · Normalized (reported)
    dimension
    general
    metric
    ARC-AGI v2 · Normalized (reported)
    rank
    3
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    ARC-AGI v2
    language
    en
    modality
    multimodal
    verified
    false
    max_score
    1
    categories
    • reasoning
    • vision
    • spatial_reasoning
    created_at
    2025-07-19T19:56:13.916360+00:00
    paper_link
    https://arxiv.org/abs/2505.11831
    updated_at
    2025-07-19T19:56:13.916360+00:00
    description
    ARC-AGI-2 is an upgraded benchmark for measuring abstract reasoning and problem-solving abilities in AI systems through visual grid transformation tasks. It evaluates fluid intelligence via input-output grid pairs (1x1 to 30x30) using colored cells (0-9), requiring models to identify underlying transformation rules from demonstration examples and apply them to test cases. Designed to be easy for humans but challenging for AI, focusing on core cognitive abilities like spatial reasoning, pattern recognition, and compositional generalization.
    benchmark_id
    arc-agi-v2
    multilingual
    false
    analysis_method
    accuracy
    is_self_reported
    false
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    ARC-AGI v2 · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-pro/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://x.com/xai/status/1943158495588815072
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-pro
    metric_population
    3
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-pro
    Original result ↗

    LLM Stats · GitHub archive

    0.049

    source id
    llmStatsArchive
    leaderboard name
    ARC-AGI v2 · Raw
    dimension
    general
    metric
    ARC-AGI v2 · Raw
    rank
    3
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    ARC-AGI v2
    language
    en
    modality
    multimodal
    verified
    false
    max_score
    1
    categories
    • reasoning
    • vision
    • spatial_reasoning
    created_at
    2025-07-19T19:56:13.916360+00:00
    paper_link
    https://arxiv.org/abs/2505.11831
    updated_at
    2025-07-19T19:56:13.916360+00:00
    description
    ARC-AGI-2 is an upgraded benchmark for measuring abstract reasoning and problem-solving abilities in AI systems through visual grid transformation tasks. It evaluates fluid intelligence via input-output grid pairs (1x1 to 30x30) using colored cells (0-9), requiring models to identify underlying transformation rules from demonstration examples and apply them to test cases. Designed to be easy for humans but challenging for AI, focusing on core cognitive abilities like spatial reasoning, pattern recognition, and compositional generalization.
    benchmark_id
    arc-agi-v2
    multilingual
    false
    analysis_method
    accuracy
    is_self_reported
    false
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    ARC-AGI v2 · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-pro/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://x.com/xai/status/1943158495588815072
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-pro
    metric_population
    3
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-pro
    Original result ↗

    LMArena

    1,250.603

    source id
    lmarena
    leaderboard name
    LMArena Vision Style Control Captioning
    dimension
    multimodal
    metric
    Arena rating
    rank
    4
    sample size
    793
    evaluated at
    2026-11-09T00:00:00.000Z
    source model id
    Gemini 2.5 Pro
    confidence lower
    1227.0539923189956
    confidence upper
    1274.1522396983437
    Original result ↗

    Kaggle Benchmarks

    0.948

    source id
    kaggleBenchmarks
    leaderboard name
    Global MMLU Lite Arabic · Arabic
    dimension
    reasoning
    metric
    Global MMLU Lite Arabic · Arabic
    rank
    5
    aggregation scope
    kaggle
    true
    status
    published
    license
    Apache-2.0
    direction
    higherIsBetter
    conditions
    task
    id
    41737
    name
    Arabic
    version
    2
    sort_order
    DESCENDING
    display_type
    rawValues
    benchmark
    id
    148
    name
    Global MMLU Lite Arabic
    type
    INDIVIDUAL
    owner
    cohere-labs
    version
    1
    categories
    • multilingual and cross-lingual capabilities
    • general knowledge and reasoning
    version_id
    130
    maintainer_tier
    Kaggle-maintained
    maintenance_level
    KAGGLE_MAINTAINED
    candidate
    candidate_class
    modelVersion
    record_kind
    aggregate
    attribution
    Kaggle · cohere-labs · Global MMLU Lite Arabic
    license_url
    https://www.apache.org/licenses/LICENSE-2.0
    metric_kind
    Global MMLU Lite Arabic · Arabic
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    130/leaderboard.json
    source_page
    https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-arabic/versions/1
    measurements
    • name
      Arabic
      unit
      score
      scale
      1
      metric
      Global MMLU Lite Arabic · Arabic
      result
      resultCase
      numericResult
      booleanResult
      false
      numericResult
      value
      0.9475
      standardDeviation
      0
      confidenceInterval
      0.0218568391591684
      hasStandardDeviation
      false
      hasConfidenceInterval
      true
      confidenceIntervalNullable
      0.0218568391591684
      hasUnevenConfidenceInterval
      false
      taskVersionId
      0
      evaluationDate
      2025-10-30T00:00:00Z
      hasBooleanResult
      false
      hasNumericResult
      true
      hasTaskVersionId
      false
      hasEvaluationDate
      true
      numericResultNullable
      value
      0.9475
      standardDeviation
      0
      confidenceInterval
      0.0218568391591684
      hasStandardDeviation
      false
      hasConfidenceInterval
      true
      confidenceIntervalNullable
      0.0218568391591684
      hasUnevenConfidenceInterval
      false
      hasNumericResultPublic
      false
      customAdditionalResults
        hasNumericResultPrivate
        false
        task_id
        41737
        task_url
        /benchmarks/tasks/benchmarkler/global-mmlu-lite-arabic
        sort_order
        DESCENDING
        source_name
        Arabic
        display_type
        rawValues
        task_version
        2
        task_display_type
        RAW_VALUES
        normalization_status
        normalized
      published_at
      2025-10-17T15:30:54.963186300Z
      candidate_class
      modelVersion
      source_revision
      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
      publisher_anchor
      google/gemini-2.5-pro
      metric_population
      41
      benchmark_evaluation
      true
      task_metadata_source
      benchmarkVersion
      candidate_metadata_source
      benchmarkVersionMappings
      evaluated at
      2025-10-30T00:00:00.000Z
      source model id
      Gemini 2.5 Pro
      confidence lower
      0.9256431608408316
      confidence upper
      0.9693568391591684
      Original result ↗

      LMArena

      1,277.362

      source id
      lmarena
      leaderboard name
      LMArena Vision Creative Writing
      dimension
      multimodal
      metric
      Arena rating
      rank
      7
      sample size
      2418
      evaluated at
      2026-01-09T00:00:00.000Z
      source model id
      gemini-2.5-pro
      confidence lower
      1263.5289926823252
      confidence upper
      1291.1946040628686
      Original result ↗

      LMArena

      1,262.387

      source id
      lmarena
      leaderboard name
      LMArena Vision Style Control Creative Writing
      dimension
      multimodal
      metric
      Arena rating
      rank
      7
      sample size
      2418
      evaluated at
      2026-01-09T00:00:00.000Z
      source model id
      gemini-2.5-pro
      confidence lower
      1248.49317268865
      confidence upper
      1276.281490777886
      Original result ↗

      LMArena

      1,250.867

      source id
      lmarena
      leaderboard name
      LMArena Vision Style Control Entity Recognition
      dimension
      multimodal
      metric
      Arena rating
      rank
      7
      sample size
      915
      evaluated at
      2026-11-09T00:00:00.000Z
      source model id
      Gemini 2.5 Pro
      confidence lower
      1228.3983917673872
      confidence upper
      1273.335383797393
      Original result ↗

      Kaggle Benchmarks

      0.925

      source id
      kaggleBenchmarks
      leaderboard name
      Global MMLU Lite Yoruba · Yoruba
      dimension
      reasoning
      metric
      Global MMLU Lite Yoruba · Yoruba
      rank
      7
      aggregation scope
      kaggle
      true
      status
      published
      license
      Apache-2.0
      direction
      higherIsBetter
      conditions
      task
      id
      41743
      name
      Yoruba
      version
      2
      sort_order
      DESCENDING
      display_type
      rawValues
      benchmark
      id
      165
      name
      Global MMLU Lite Yoruba
      type
      INDIVIDUAL
      owner
      cohere-labs
      version
      1
      categories
      • multilingual and cross-lingual capabilities
      • general knowledge and reasoning
      version_id
      147
      maintainer_tier
      Kaggle-maintained
      maintenance_level
      KAGGLE_MAINTAINED
      candidate
      candidate_class
      modelVersion
      record_kind
      aggregate
      attribution
      Kaggle · cohere-labs · Global MMLU Lite Yoruba
      license_url
      https://www.apache.org/licenses/LICENSE-2.0
      metric_kind
      Global MMLU Lite Yoruba · Yoruba
      rank_method
      competition rank among reported evaluations in identical conditions, including unlinked observations
      source_file
      147/leaderboard.json
      source_page
      https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-yoruba/versions/1
      measurements
      • name
        Yoruba
        unit
        score
        scale
        1
        metric
        Global MMLU Lite Yoruba · Yoruba
        result
        resultCase
        numericResult
        booleanResult
        false
        numericResult
        value
        0.925
        standardDeviation
        0
        confidenceInterval
        0.0258118773864695
        hasStandardDeviation
        false
        hasConfidenceInterval
        true
        confidenceIntervalNullable
        0.0258118773864695
        hasUnevenConfidenceInterval
        false
        taskVersionId
        0
        evaluationDate
        2025-10-30T00:00:00Z
        hasBooleanResult
        false
        hasNumericResult
        true
        hasTaskVersionId
        false
        hasEvaluationDate
        true
        numericResultNullable
        value
        0.925
        standardDeviation
        0
        confidenceInterval
        0.0258118773864695
        hasStandardDeviation
        false
        hasConfidenceInterval
        true
        confidenceIntervalNullable
        0.0258118773864695
        hasUnevenConfidenceInterval
        false
        hasNumericResultPublic
        false
        customAdditionalResults
          hasNumericResultPrivate
          false
          task_id
          41743
          task_url
          /benchmarks/tasks/benchmarkler/global-mmlu-lite-yoruba
          sort_order
          DESCENDING
          source_name
          Yoruba
          display_type
          rawValues
          task_version
          2
          task_display_type
          RAW_VALUES
          normalization_status
          normalized
        published_at
        2025-10-17T15:30:24.645552400Z
        candidate_class
        modelVersion
        source_revision
        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
        publisher_anchor
        google/gemini-2.5-pro
        metric_population
        41
        benchmark_evaluation
        true
        task_metadata_source
        benchmarkVersion
        candidate_metadata_source
        benchmarkVersionMappings
        evaluated at
        2025-10-30T00:00:00.000Z
        source model id
        Gemini 2.5 Pro
        confidence lower
        0.8991881226135305
        confidence upper
        0.9508118773864696
        Original result ↗

        Kaggle Benchmarks

        78.947%

        source id
        kaggleBenchmarks
        leaderboard name
        VideoQA · Accuracy
        dimension
        reasoning
        metric
        VideoQA · Accuracy
        rank
        7
        aggregation scope
        kaggle
        true
        status
        published
        license
        Apache-2.0
        direction
        higherIsBetter
        conditions
        task
        id
        33688
        name
        Accuracy
        version
        21
        sort_order
        DESCENDING
        display_type
        percentages
        benchmark
        id
        3424
        name
        VideoQA
        type
        INDIVIDUAL
        owner
        deepmind
        version
        1
        categories
        • general knowledge and reasoning
        • multimodal capabilities
        version_id
        3447
        maintainer_tier
        Kaggle-maintained
        maintenance_level
        KAGGLE_MAINTAINED
        candidate
        candidate_class
        modelVersion
        record_kind
        aggregate
        attribution
        Kaggle · deepmind · VideoQA
        license_url
        https://www.apache.org/licenses/LICENSE-2.0
        metric_kind
        VideoQA · Accuracy
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        source_file
        3447/leaderboard.json
        source_page
        https://www.kaggle.com/benchmarks/deepmind/video-qa/versions/1
        measurements
        • name
          Accuracy
          unit
          percent
          scale
          1
          metric
          VideoQA · Accuracy
          result
          resultCase
          numericResult
          booleanResult
          false
          numericResult
          value
          0.7894736842105263
          standardDeviation
          0
          confidenceInterval
          0
          hasStandardDeviation
          false
          hasConfidenceInterval
          false
          hasUnevenConfidenceInterval
          false
          taskVersionId
          0
          evaluationDate
          2026-06-08T15:57:54.573686Z
          hasBooleanResult
          false
          hasNumericResult
          true
          hasTaskVersionId
          false
          hasEvaluationDate
          true
          numericResultNullable
          value
          0.7894736842105263
          standardDeviation
          0
          confidenceInterval
          0
          hasStandardDeviation
          false
          hasConfidenceInterval
          false
          hasUnevenConfidenceInterval
          false
          hasNumericResultPublic
          false
          customAdditionalResults
            hasNumericResultPrivate
            false
            task_id
            33688
            task_url
            /benchmarks/tasks/aminmohamedmohami/video-qa-benchmark
            sort_order
            DESCENDING
            source_name
            Accuracy
            display_type
            percentages
            task_version
            21
            task_display_type
            PERCENTAGES
            normalization_status
            normalized
          published_at
          2026-06-10T16:01:43.077213600Z
          candidate_class
          modelVersion
          source_revision
          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
          publisher_anchor
          google/gemini-2.5-pro
          metric_population
          20
          benchmark_evaluation
          true
          task_metadata_source
          benchmarkVersion
          candidate_metadata_source
          benchmarkVersionMappings
          evaluated at
          2026-06-08T15:57:54.573Z
          source model id
          Gemini 2.5 Pro
          Original result ↗

          Kaggle Benchmarks

          66.979%

          source id
          kaggleBenchmarks
          leaderboard name
          AssetOpsBench · Score
          dimension
          reasoning
          metric
          AssetOpsBench · Score
          rank
          7
          aggregation scope
          kaggle
          true
          status
          published
          license
          Apache-2.0
          direction
          higherIsBetter
          conditions
          task
          id
          487619
          name
          Score
          version
          5
          sort_order
          DESCENDING
          display_type
          percentages
          benchmark
          id
          243
          name
          AssetOpsBench
          type
          INDIVIDUAL
          owner
          ibm-research
          version
          1
          categories
          • evaluation
          • general knowledge and reasoning
          version_id
          222
          maintainer_tier
          Kaggle-maintained
          maintenance_level
          KAGGLE_MAINTAINED
          candidate
          candidate_class
          modelVersion
          record_kind
          aggregate
          attribution
          Kaggle · ibm-research · AssetOpsBench
          license_url
          https://www.apache.org/licenses/LICENSE-2.0
          metric_kind
          AssetOpsBench · Score
          rank_method
          competition rank among reported evaluations in identical conditions, including unlinked observations
          source_file
          222/leaderboard.json
          source_page
          https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
          measurements
          • name
            Score
            unit
            percent
            scale
            1
            metric
            AssetOpsBench · Score
            result
            resultCase
            numericResult
            booleanResult
            false
            numericResult
            value
            0.6697912760904887
            standardDeviation
            0
            confidenceInterval
            0
            hasStandardDeviation
            false
            hasConfidenceInterval
            false
            hasUnevenConfidenceInterval
            false
            taskVersionId
            0
            evaluationDate
            2025-11-22T05:00:13.915959Z
            hasBooleanResult
            false
            hasNumericResult
            true
            hasTaskVersionId
            false
            hasEvaluationDate
            true
            numericResultNullable
            value
            0.6697912760904887
            standardDeviation
            0
            confidenceInterval
            0
            hasStandardDeviation
            false
            hasConfidenceInterval
            false
            hasUnevenConfidenceInterval
            false
            hasNumericResultPublic
            false
            customAdditionalResults
              hasNumericResultPrivate
              false
              task_id
              487619
              task_url
              /benchmarks/tasks/benchmarkler/failuresensoriq-main-benchmark-task
              sort_order
              DESCENDING
              source_name
              Score
              display_type
              percentages
              task_version
              5
              task_display_type
              PERCENTAGES
              normalization_status
              normalized
            published_at
            2025-12-01T22:15:26.261663300Z
            candidate_class
            modelVersion
            source_revision
            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
            publisher_anchor
            google/gemini-2.5-pro
            metric_population
            25
            benchmark_evaluation
            true
            task_metadata_source
            benchmarkVersion
            candidate_metadata_source
            benchmarkVersionMappings
            evaluated at
            2025-11-22T05:00:13.915Z
            source model id
            Gemini 2.5 Pro
            Original result ↗

            Kaggle Benchmarks

            53.333%

            source id
            kaggleBenchmarks
            leaderboard name
            ITBench · FinOps Score
            dimension
            reasoning
            metric
            ITBench · FinOps Score
            rank
            7
            aggregation scope
            kaggle
            true
            status
            published
            license
            Apache-2.0
            direction
            higherIsBetter
            conditions
            task
            id
            1167
            name
            FinOps Score
            version
            8
            sort_order
            DESCENDING
            display_type
            rawPercentages
            benchmark
            id
            246
            name
            ITBench
            type
            INDIVIDUAL
            owner
            ibm-research
            version
            1
            categories
            • evaluation
            • general knowledge and reasoning
            version_id
            225
            maintainer_tier
            Kaggle-maintained
            maintenance_level
            KAGGLE_MAINTAINED
            candidate
            candidate_class
            modelVersion
            record_kind
            aggregate
            attribution
            Kaggle · ibm-research · ITBench
            license_url
            https://www.apache.org/licenses/LICENSE-2.0
            metric_kind
            ITBench · FinOps Score
            rank_method
            competition rank among reported evaluations in identical conditions, including unlinked observations
            source_file
            225/leaderboard.json
            source_page
            https://www.kaggle.com/benchmarks/ibm-research/itbench/versions/1
            measurements
            • name
              FinOps Score
              unit
              percent
              scale
              100
              metric
              ITBench · FinOps Score
              result
              resultCase
              numericResult
              booleanResult
              false
              numericResult
              value
              53.333333333333336
              standardDeviation
              0
              confidenceInterval
              0
              hasStandardDeviation
              false
              hasConfidenceInterval
              false
              hasUnevenConfidenceInterval
              false
              taskVersionId
              0
              evaluationDate
              2025-12-01T19:44:23.555836Z
              hasBooleanResult
              false
              hasNumericResult
              true
              hasTaskVersionId
              false
              hasEvaluationDate
              true
              numericResultNullable
              value
              53.333333333333336
              standardDeviation
              0
              confidenceInterval
              0
              hasStandardDeviation
              false
              hasConfidenceInterval
              false
              hasUnevenConfidenceInterval
              false
              hasNumericResultPublic
              false
              customAdditionalResults
                hasNumericResultPrivate
                false
                task_id
                1167
                task_url
                /benchmarks/tasks/benchmarkler/itbench-finops-main-1
                sort_order
                DESCENDING
                source_name
                FinOps Score
                display_type
                rawPercentages
                task_version
                8
                task_display_type
                RAW_VALUES
                normalization_status
                normalized
              published_at
              2025-12-02T12:36:17.407125500Z
              candidate_class
              modelVersion
              source_revision
              b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
              publisher_anchor
              google/gemini-2.5-pro
              metric_population
              13
              benchmark_evaluation
              true
              task_metadata_source
              benchmarkVersion
              candidate_metadata_source
              benchmarkVersionMappings
              evaluated at
              2025-12-01T19:44:23.555Z
              source model id
              Gemini 2.5 Pro
              Original result ↗

              LMArena

              1,283.599

              source id
              lmarena
              leaderboard name
              LMArena Vision Entity Recognition
              dimension
              multimodal
              metric
              Arena rating
              rank
              8
              sample size
              915
              evaluated at
              2026-11-09T00:00:00.000Z
              source model id
              Gemini 2.5 Pro
              confidence lower
              1261.2027166627993
              confidence upper
              1305.9960930299312
              Original result ↗

              Kaggle Benchmarks

              0.938

              source id
              kaggleBenchmarks
              leaderboard name
              Global MMLU Lite Swahili · Swahili
              dimension
              reasoning
              metric
              Global MMLU Lite Swahili · Swahili
              rank
              8
              aggregation scope
              kaggle
              true
              status
              published
              license
              Apache-2.0
              direction
              higherIsBetter
              conditions
              task
              id
              41750
              name
              Swahili
              version
              2
              sort_order
              DESCENDING
              display_type
              rawValues
              benchmark
              id
              164
              name
              Global MMLU Lite Swahili
              type
              INDIVIDUAL
              owner
              cohere-labs
              version
              1
              categories
              • multilingual and cross-lingual capabilities
              • general knowledge and reasoning
              version_id
              146
              maintainer_tier
              Kaggle-maintained
              maintenance_level
              KAGGLE_MAINTAINED
              candidate
              candidate_class
              modelVersion
              record_kind
              aggregate
              attribution
              Kaggle · cohere-labs · Global MMLU Lite Swahili
              license_url
              https://www.apache.org/licenses/LICENSE-2.0
              metric_kind
              Global MMLU Lite Swahili · Swahili
              rank_method
              competition rank among reported evaluations in identical conditions, including unlinked observations
              source_file
              146/leaderboard.json
              source_page
              https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-swahili/versions/1
              measurements
              • name
                Swahili
                unit
                score
                scale
                1
                metric
                Global MMLU Lite Swahili · Swahili
                result
                resultCase
                numericResult
                booleanResult
                false
                numericResult
                value
                0.9375
                standardDeviation
                0
                confidenceInterval
                0.0237215870977811
                hasStandardDeviation
                false
                hasConfidenceInterval
                true
                confidenceIntervalNullable
                0.0237215870977811
                hasUnevenConfidenceInterval
                false
                taskVersionId
                0
                evaluationDate
                2025-10-30T00:00:00Z
                hasBooleanResult
                false
                hasNumericResult
                true
                hasTaskVersionId
                false
                hasEvaluationDate
                true
                numericResultNullable
                value
                0.9375
                standardDeviation
                0
                confidenceInterval
                0.0237215870977811
                hasStandardDeviation
                false
                hasConfidenceInterval
                true
                confidenceIntervalNullable
                0.0237215870977811
                hasUnevenConfidenceInterval
                false
                hasNumericResultPublic
                false
                customAdditionalResults
                  hasNumericResultPrivate
                  false
                  task_id
                  41750
                  task_url
                  /benchmarks/tasks/benchmarkler/global-mmlu-lite-swahili
                  sort_order
                  DESCENDING
                  source_name
                  Swahili
                  display_type
                  rawValues
                  task_version
                  2
                  task_display_type
                  RAW_VALUES
                  normalization_status
                  normalized
                published_at
                2025-10-17T15:30:27.253350900Z
                candidate_class
                modelVersion
                source_revision
                b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                publisher_anchor
                google/gemini-2.5-pro
                metric_population
                41
                benchmark_evaluation
                true
                task_metadata_source
                benchmarkVersion
                candidate_metadata_source
                benchmarkVersionMappings
                evaluated at
                2025-10-30T00:00:00.000Z
                source model id
                Gemini 2.5 Pro
                confidence lower
                0.913778412902219
                confidence upper
                0.961221587097781
                Original result ↗

                OpenRouter Benchmarks

                82.239%

                source id
                openRouterBenchmarks
                leaderboard name
                GPQA Diamond · Accuracy · Google AI Studio
                dimension
                knowledge
                metric
                GPQA Diamond · Accuracy
                rank
                8
                sample size
                396
                aggregation scope
                status
                published
                harness
                url
                https://github.com/OpenRouterTeam/benchmark-harness
                name
                OpenRouter benchmark harness
                direction
                higherIsBetter
                run_count
                2
                conditions
                window
                newest 90 days; successful visible nonzero runs
                dataset
                benchmark-leaderboard
                benchmark
                id
                gpqa_diamond
                name
                GPQA Diamond
                record_kind
                aggregate
                provider_name
                Google AI Studio
                attribution
                Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
                last_run_at
                2026-10-01T19:12:31.045Z
                license_url
                https://creativecommons.org/licenses/by/4.0/
                methodology
                Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
                metric_kind
                GPQA Diamond · Accuracy
                rank_method
                competition rank among reported evaluations in identical conditions, including unlinked observations
                sample_size
                396
                source_file
                benchmark-leaderboard.json
                source_page
                https://openrouter.ai/benchmarks/gpqa-diamond
                measurements
                accuracy
                0.8223905000000001
                accuracy_std_dev
                0.0244105
                cost_per_task_usd
                0.2415952904040404
                output_tokens_per_task
                24066.49494949495
                latency_seconds_per_task
                181.95301010101008
                published_at
                2026-11-16T01:46:02.653Z
                snapshot_date
                2026-11-16
                source_revision
                7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
                transformations
                TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
                metric_population
                12
                benchmark_evaluation
                true
                openrouter_permaslug
                google/gemini-2.5-pro
                source model id
                google/gemini-2.5-pro
                Original result ↗

                Published prices

                1–20 / 60

                google/gemini-2.5-pro ↗

                $0.13

                Inference provider
                cloudflareWorkersAI
                Platform
                Google Vertex AI
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                Billing label
                Cached input tokens (per 1M)
                Pricing via
                Cloudflare AI Gateway

                google/gemini-2.5-pro ↗

                $1.25

                Inference provider
                cloudflareWorkersAI
                Platform
                Google Vertex AI
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions
                Billing label
                Input tokens (per 1M)
                Pricing via
                Cloudflare AI Gateway

                google/gemini-2.5-pro ↗

                $10.00

                Inference provider
                cloudflareWorkersAI
                Platform
                Google Vertex AI
                Unit
                output_tokens_per_million
                Currency
                USD
                Billing conditions
                Billing label
                Output tokens (per 1M)
                Pricing via
                Cloudflare AI Gateway

                google/gemini-2.5-pro ↗

                $1.25

                Inference provider
                deepInfra
                Platform
                DeepInfra
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-pro ↗

                $10.00

                Inference provider
                deepInfra
                Platform
                DeepInfra
                Unit
                output_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-pro ↗

                $0.13

                Inference provider
                openRouter
                Platform
                OpenRouter
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-pro ↗

                $0.38

                Inference provider
                openRouter
                Platform
                OpenRouter
                Unit
                cache_write_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-pro ↗

                $1.25

                Inference provider
                openRouter
                Platform
                OpenRouter
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-pro ↗

                $10.00

                Inference provider
                openRouter
                Platform
                OpenRouter
                Unit
                output_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-pro ↗

                $0.25

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                context_tokens
                min_inclusive
                200001
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-pro ↗

                $0.13

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                context_tokens
                max_exclusive
                200001
                min_inclusive
                0
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-pro ↗

                $0.22

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                service_tier
                priority
                billing_scope
                Vercel public catalog; endpoint prices may differ
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-pro ↗

                $0.25

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                region
                eu
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                context_tokens
                min_inclusive
                200001
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-pro ↗

                $0.13

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                region
                eu
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                context_tokens
                max_exclusive
                200001
                min_inclusive
                0
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-pro ↗

                $0.25

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                region
                us
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                context_tokens
                min_inclusive
                200001
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-pro ↗

                $0.13

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                region
                us
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                context_tokens
                max_exclusive
                200001
                min_inclusive
                0
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-pro ↗

                $2.50

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                context_tokens
                min_inclusive
                200001
                Billing label
                Input tokens / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-pro ↗

                $1.25

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                context_tokens
                max_exclusive
                200001
                min_inclusive
                0
                Billing label
                Input tokens / 1M
                Pricing via
                Vercel AI Gateway

                Provider specifications & access

                1–4 / 4

                gemini-2.5-pro ↗

                source model id
                gemini-2.5-pro
                hosting
                gateway
                platforms
                • Cheaper Inference
                context length
                1048576
                max output tokens
                65535
                architecture
                input_modalities
                • text
                • image
                output_modalities
                • text
                capabilities
                • reasoning
                • structuredOutputs
                • toolCalling
                • visionInput
                • webSearch

                Gemini 2.5 Pro ↗

                source model id
                google/gemini-2.5-pro
                hosting
                third-party
                platforms
                • Google Vertex AI
                context length
                1000000
                max output tokens
                65536
                architecture
                input_modalities
                • text
                output_modalities
                • text
                capabilities
                • reasoning

                Gemini 2.5 Pro ↗

                source model id
                google/gemini-2.5-pro
                hosting
                vercel-gateway
                platforms
                • Vercel AI Gateway
                context length
                1048576
                max output tokens
                65535
                architecture
                input_modalities
                • text
                • image
                • pdf
                output_modalities
                • text
                capabilities
                • promptCaching
                • reasoning
                • structuredOutputs
                • toolCalling
                • visionInput
                • webSearch
                regions
                • eu
                • us

                Gemini 2.5 Pro ↗

                source model id
                publishers/google/models/gemini-2.5-pro@default
                hosting
                vertex-managed
                platforms
                • Google Cloud Vertex AI
                context length
                1048576
                max output tokens
                65536
                architecture
                input_modalities
                • audio
                • image
                • text
                • video
                output_modalities
                • text
                capabilities
                • batch
                • multimodal
                • promptCaching
                • reasoning
                • structuredOutputs
                • toolCalling
                • visionInput

                Safety assessments

                1–1 / 1

                Foundation Model Transparency Index 2025 ↗

                41 / 100

                source id
                stanfordFMTI
                canonical model id
                google/gemini-2.5-pro
                taxonomy
                transparency
                hazard category
                transparency
                hazard name
                Organization transparency
                policy area
                Google
                assessment type
                organization-level disclosure
                metric
                disclosure_score
                result label
                41 / 100
                sample size
                100
                updated at
                2026-11-09T18:08:05.227Z

                Source specification observations

                1–10 / 10

                cloudflareWorkersAI ↗

                field
                context_length
                value
                1000000
                normalized value
                1000000
                scope
                source-model
                source id
                cloudflareWorkersAI
                confidence
                1

                openRouter ↗

                field
                context_length
                value
                1048576
                normalized value
                1048576
                scope
                source-model
                source id
                openRouter
                confidence
                1

                vercelAIGatewayCatalog ↗

                field
                context_length
                value
                1048576
                normalized value
                1048576
                scope
                source-model
                source id
                vercelAIGatewayCatalog
                confidence
                1

                vertexAI ↗

                field
                context_length
                value
                1048576
                normalized value
                1048576
                scope
                source-model
                source id
                vertexAI
                confidence
                1

                openRouter ↗

                field
                knowledge_cutoff
                reported field
                knowledge_cutoff
                value
                2025-01-31
                normalized value
                2025-01-31
                precision
                day
                scope
                source-model
                source id
                openRouter
                confidence
                1

                vercelAIGatewayCatalog ↗

                field
                knowledge_cutoff
                reported field
                knowledge_cutoff
                value
                2025-01
                normalized value
                2025-01
                precision
                month
                scope
                source-model
                source id
                vercelAIGatewayCatalog
                confidence
                1

                cloudflareWorkersAI ↗

                field
                max_output_tokens
                value
                65536
                normalized value
                65536
                scope
                source-model
                source id
                cloudflareWorkersAI
                confidence
                1

                openRouter ↗

                field
                max_output_tokens
                value
                65536
                normalized value
                65536
                scope
                source-model
                source id
                openRouter
                confidence
                1

                vercelAIGatewayCatalog ↗

                field
                max_output_tokens
                value
                65535
                normalized value
                65535
                scope
                source-model
                source id
                vercelAIGatewayCatalog
                confidence
                1

                vertexAI ↗

                field
                max_output_tokens
                value
                65536
                normalized value
                65536
                scope
                source-model
                source id
                vertexAI
                confidence
                1

                Original sources

                1–20 / 20

                Google · Model reference ↗

                DeepInfra · Model page ↗

                Google Cloud · Model page ↗

                OpenRouter · Model page ↗

                Vercel AI Gateway · Model page ↗

                Google Cloud · Documentation ↗

                GitHub · Source snapshot ↗

                api.cheaperinference.com · Source snapshot ↗

                Cloudflare AI · Catalog ↗

                DeepInfra · Catalog ↗

                Epoch AI · Catalog ↗

                GitHub · Catalog ↗

                GitHub · Catalog ↗

                Google Cloud · Catalog ↗

                Kaggle · Catalog ↗

                LMArena · Catalog ↗

                OpenRouter · Catalog ↗

                OpenRouter · Catalog ↗

                Vercel AI Gateway · Catalog ↗

                www.swebench.com · Catalog ↗

                Source links

                1–14 / 14

                Explore more models

                Models

                Loading
                EVIDENCE / COMPARISON

                Benchmarks

                Explore every benchmark

                Loading

                These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                Install TensorTrace