Google: Gemini 2.5 Flash

Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, e…

google/gemini-2.5-flash

Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, enabling it to provide responses with greater accuracy and nuanced context handling.

Additionally, Gemini 2.5 Flash is configurable through the "max tokens for reasoning" parameter, as described in the documentation (https://openrouter.ai/docs/use-cases/reasoning-tokens#max-tokens-for-reasoning).

Specifications & capabilities

Input
file, image, text, audio, video
Output
text
Context window · tokens
1048576
Knowledge cutoff
2025-01
Published model ID
gemini-2.5-flash@default
Capabilities
audioInput, batch, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗

Evaluations

1–20 / 2,449

LLM Stats · GitHub archive

0.853

source id
llmStatsArchive
leaderboard name
FACTS Grounding · Raw
dimension
general
metric
FACTS Grounding · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
FACTS Grounding
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
created_at
2025-07-19T19:56:13.260285+00:00
paper_link
https://arxiv.org/abs/2501.03200
updated_at
2025-07-19T19:56:13.260285+00:00
description
A benchmark evaluating language models' ability to generate factually accurate and well-grounded responses based on long-form input context, comprising 1,719 examples with documents up to 32k tokens requiring detailed responses that are fully grounded in provided documents
benchmark_id
facts-grounding
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
FACTS Grounding · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/gemini-2.5-flash/benchmarks.json
archive_kind
llmStats
evidence_url
https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/gemini-2.5-flash
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gemini-2.5-flash
Original result ↗

LLM Stats · GitHub archive

0.853

source id
llmStatsArchive
leaderboard name
FACTS Grounding · Normalized (reported)
dimension
general
metric
FACTS Grounding · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
FACTS Grounding
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
created_at
2025-07-19T19:56:13.260285+00:00
paper_link
https://arxiv.org/abs/2501.03200
updated_at
2025-07-19T19:56:13.260285+00:00
description
A benchmark evaluating language models' ability to generate factually accurate and well-grounded responses based on long-form input context, comprising 1,719 examples with documents up to 32k tokens requiring detailed responses that are fully grounded in provided documents
benchmark_id
facts-grounding
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
FACTS Grounding · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/gemini-2.5-flash/benchmarks.json
archive_kind
llmStats
evidence_url
https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/gemini-2.5-flash
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gemini-2.5-flash
Original result ↗

LLM Stats · GitHub archive

0.797

source id
llmStatsArchive
leaderboard name
MMMU · Raw
dimension
general
metric
MMMU · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMU
language
en
modality
multimodal
verified
false
max_score
1
categories
  • multimodal
  • reasoning
  • general
created_at
2025-07-19T19:56:12.130105+00:00
paper_link
https://arxiv.org/abs/2311.16502
updated_at
2025-07-19T19:56:12.130105+00:00
description
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
benchmark_id
mmmu
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMU · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/gemini-2.5-flash/benchmarks.json
archive_kind
llmStats
evidence_url
https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/gemini-2.5-flash
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gemini-2.5-flash
Original result ↗

LLM Stats · GitHub archive

0.797

source id
llmStatsArchive
leaderboard name
MMMU · Normalized (reported)
dimension
general
metric
MMMU · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMU
language
en
modality
multimodal
verified
false
max_score
1
categories
  • multimodal
  • reasoning
  • general
created_at
2025-07-19T19:56:12.130105+00:00
paper_link
https://arxiv.org/abs/2311.16502
updated_at
2025-07-19T19:56:12.130105+00:00
description
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
benchmark_id
mmmu
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMU · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/gemini-2.5-flash/benchmarks.json
archive_kind
llmStats
evidence_url
https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/gemini-2.5-flash
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gemini-2.5-flash
Original result ↗

Kaggle Benchmarks

50%

source id
kaggleBenchmarks
leaderboard name
ITBench · SRE Score
dimension
reasoning
metric
ITBench · SRE Score
rank
1
aggregation scope
kaggle
true
status
published
license
Apache-2.0
direction
higherIsBetter
conditions
task
id
1172
name
SRE Score
version
8
sort_order
DESCENDING
display_type
rawPercentages
benchmark
id
246
name
ITBench
type
INDIVIDUAL
owner
ibm-research
version
1
categories
  • evaluation
  • general knowledge and reasoning
version_id
225
maintainer_tier
Kaggle-maintained
maintenance_level
KAGGLE_MAINTAINED
candidate
candidate_class
modelVersion
record_kind
aggregate
attribution
Kaggle · ibm-research · ITBench
license_url
https://www.apache.org/licenses/LICENSE-2.0
metric_kind
ITBench · SRE Score
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
225/leaderboard.json
source_page
https://www.kaggle.com/benchmarks/ibm-research/itbench/versions/1
measurements
  • name
    SRE Score
    unit
    percent
    scale
    100
    metric
    ITBench · SRE Score
    result
    resultCase
    numericResult
    booleanResult
    false
    numericResult
    value
    50
    standardDeviation
    0
    confidenceInterval
    0
    hasStandardDeviation
    false
    hasConfidenceInterval
    false
    hasUnevenConfidenceInterval
    false
    taskVersionId
    0
    evaluationDate
    2025-12-01T22:28:02.522152Z
    hasBooleanResult
    false
    hasNumericResult
    true
    hasTaskVersionId
    false
    hasEvaluationDate
    true
    numericResultNullable
    value
    50
    standardDeviation
    0
    confidenceInterval
    0
    hasStandardDeviation
    false
    hasConfidenceInterval
    false
    hasUnevenConfidenceInterval
    false
    hasNumericResultPublic
    false
    customAdditionalResults
      hasNumericResultPrivate
      false
      task_id
      1172
      task_url
      /benchmarks/tasks/benchmarkler/itbench-sre-main
      sort_order
      DESCENDING
      source_name
      SRE Score
      display_type
      rawPercentages
      task_version
      8
      task_display_type
      RAW_VALUES
      normalization_status
      normalized
    published_at
    2025-12-02T12:36:17.407125500Z
    candidate_class
    modelVersion
    source_revision
    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    13
    benchmark_evaluation
    true
    task_metadata_source
    benchmarkVersion
    candidate_metadata_source
    benchmarkVersionMappings
    evaluated at
    2025-12-01T22:28:02.522Z
    source model id
    Gemini 2.5 Flash
    Original result ↗

    LLM Stats · GitHub archive

    0.884

    source id
    llmStatsArchive
    leaderboard name
    Global-MMLU-Lite · Normalized (reported)
    dimension
    general
    metric
    Global-MMLU-Lite · Normalized (reported)
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    Global-MMLU-Lite
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • general
    • language
    • reasoning
    created_at
    2025-07-19T19:56:13.534515+00:00
    paper_link
    https://arxiv.org/abs/2412.03304
    updated_at
    2025-07-19T19:56:13.534515+00:00
    description
    A lightweight version of Global MMLU benchmark that evaluates language models across multiple languages while addressing cultural and linguistic biases in multilingual evaluation.
    benchmark_id
    global-mmlu-lite
    multilingual
    true
    analysis_method
    Accuracy
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    Global-MMLU-Lite · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    2
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.884

    source id
    llmStatsArchive
    leaderboard name
    Global-MMLU-Lite · Raw
    dimension
    general
    metric
    Global-MMLU-Lite · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    Global-MMLU-Lite
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • general
    • language
    • reasoning
    created_at
    2025-07-19T19:56:13.534515+00:00
    paper_link
    https://arxiv.org/abs/2412.03304
    updated_at
    2025-07-19T19:56:13.534515+00:00
    description
    A lightweight version of Global MMLU benchmark that evaluates language models across multiple languages while addressing cultural and linguistic biases in multilingual evaluation.
    benchmark_id
    global-mmlu-lite
    multilingual
    true
    analysis_method
    Accuracy
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    Global-MMLU-Lite · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    2
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.88

    source id
    llmStatsArchive
    leaderboard name
    AIME 2024 · Normalized (reported)
    dimension
    general
    metric
    AIME 2024 · Normalized (reported)
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    AIME 2024
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • math
    • reasoning
    created_at
    2025-07-19T19:56:11.941652+00:00
    paper_link
    https://arxiv.org/html/2503.21380v2
    updated_at
    2025-09-30T00:00:00.000000+00:00
    description
    American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
    benchmark_id
    aime-2024
    multilingual
    false
    analysis_method
    Pass@1
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    AIME 2024 · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    5
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.88

    source id
    llmStatsArchive
    leaderboard name
    AIME 2024 · Raw
    dimension
    general
    metric
    AIME 2024 · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    AIME 2024
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • math
    • reasoning
    created_at
    2025-07-19T19:56:11.941652+00:00
    paper_link
    https://arxiv.org/html/2503.21380v2
    updated_at
    2025-09-30T00:00:00.000000+00:00
    description
    American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
    benchmark_id
    aime-2024
    multilingual
    false
    analysis_method
    Pass@1
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    AIME 2024 · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    5
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.828

    source id
    llmStatsArchive
    leaderboard name
    GPQA · Raw
    dimension
    general
    metric
    GPQA · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    GPQA
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • reasoning
    • general
    created_at
    2025-07-19T19:56:11.588605+00:00
    paper_link
    https://arxiv.org/abs/2311.12022
    updated_at
    2025-07-19T19:56:11.588605+00:00
    description
    A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
    benchmark_id
    gpqa
    multilingual
    false
    analysis_method
    Pass@1
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    GPQA · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    5
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.828

    source id
    llmStatsArchive
    leaderboard name
    GPQA · Normalized (reported)
    dimension
    general
    metric
    GPQA · Normalized (reported)
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    GPQA
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • reasoning
    • general
    created_at
    2025-07-19T19:56:11.588605+00:00
    paper_link
    https://arxiv.org/abs/2311.12022
    updated_at
    2025-07-19T19:56:11.588605+00:00
    description
    A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
    benchmark_id
    gpqa
    multilingual
    false
    analysis_method
    Pass@1
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    GPQA · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    5
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.72

    source id
    llmStatsArchive
    leaderboard name
    AIME 2025 · Raw
    dimension
    general
    metric
    AIME 2025 · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    AIME 2025
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • math
    • reasoning
    created_at
    2025-09-05T00:00:00.000000+00:00
    paper_link
    https://arxiv.org/abs/2503.21380
    updated_at
    2025-09-05T00:00:00.000000+00:00
    description
    All 30 problems from the 2025 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999. Used as an AI benchmark to evaluate large language models' ability to solve complex mathematical problems requiring multi-step logical deductions and structured symbolic reasoning.
    benchmark_id
    aime-2025
    multilingual
    false
    analysis_method
    Pass@1
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    AIME 2025 · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    2
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.72

    source id
    llmStatsArchive
    leaderboard name
    AIME 2025 · Normalized (reported)
    dimension
    general
    metric
    AIME 2025 · Normalized (reported)
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    AIME 2025
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • math
    • reasoning
    created_at
    2025-09-05T00:00:00.000000+00:00
    paper_link
    https://arxiv.org/abs/2503.21380
    updated_at
    2025-09-05T00:00:00.000000+00:00
    description
    All 30 problems from the 2025 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999. Used as an AI benchmark to evaluate large language models' ability to solve complex mathematical problems requiring multi-step logical deductions and structured symbolic reasoning.
    benchmark_id
    aime-2025
    multilingual
    false
    analysis_method
    Pass@1
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    AIME 2025 · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    2
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.654

    source id
    llmStatsArchive
    leaderboard name
    Vibe-Eval · Raw
    dimension
    general
    metric
    Vibe-Eval · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    Vibe-Eval
    language
    en
    modality
    multimodal
    verified
    false
    max_score
    1
    categories
    • multimodal
    • vision
    • general
    created_at
    2025-07-19T19:56:13.871369+00:00
    paper_link
    https://arxiv.org/abs/2405.02287
    updated_at
    2025-07-19T19:56:13.871369+00:00
    description
    VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
    benchmark_id
    vibe-eval
    multilingual
    false
    analysis_method
    Accuracy
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    Vibe-Eval · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    4
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.654

    source id
    llmStatsArchive
    leaderboard name
    Vibe-Eval · Normalized (reported)
    dimension
    general
    metric
    Vibe-Eval · Normalized (reported)
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    Vibe-Eval
    language
    en
    modality
    multimodal
    verified
    false
    max_score
    1
    categories
    • multimodal
    • vision
    • general
    created_at
    2025-07-19T19:56:13.871369+00:00
    paper_link
    https://arxiv.org/abs/2405.02287
    updated_at
    2025-07-19T19:56:13.871369+00:00
    description
    VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
    benchmark_id
    vibe-eval
    multilingual
    false
    analysis_method
    Accuracy
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    Vibe-Eval · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    4
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.639

    source id
    llmStatsArchive
    leaderboard name
    LiveCodeBench v5 · Normalized (reported)
    dimension
    general
    metric
    LiveCodeBench v5 · Normalized (reported)
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    LiveCodeBench v5
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • reasoning
    • general
    created_at
    2025-07-19T19:56:13.759330+00:00
    paper_link
    https://arxiv.org/abs/2403.07974
    updated_at
    2025-07-19T19:56:13.759330+00:00
    description
    LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
    benchmark_id
    livecodebench-v5
    multilingual
    false
    analysis_method
    Pass@1
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    LiveCodeBench v5 · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    3
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.639

    source id
    llmStatsArchive
    leaderboard name
    LiveCodeBench v5 · Raw
    dimension
    general
    metric
    LiveCodeBench v5 · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    LiveCodeBench v5
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • reasoning
    • general
    created_at
    2025-07-19T19:56:13.759330+00:00
    paper_link
    https://arxiv.org/abs/2403.07974
    updated_at
    2025-07-19T19:56:13.759330+00:00
    description
    LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
    benchmark_id
    livecodebench-v5
    multilingual
    false
    analysis_method
    Pass@1
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    LiveCodeBench v5 · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    3
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.604

    source id
    llmStatsArchive
    leaderboard name
    SWE-Bench Verified · Raw
    dimension
    general
    metric
    SWE-Bench Verified · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    SWE-Bench Verified
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • reasoning
    • frontend_development
    • code
    created_at
    2025-07-19T19:56:13.812805+00:00
    paper_link
    https://arxiv.org/abs/2310.06770
    updated_at
    2025-07-19T19:56:13.812805+00:00
    description
    A verified subset of 500 software engineering problems from real GitHub issues, validated by human annotators for evaluating language models' ability to resolve real-world coding issues by generating patches for Python codebases.
    benchmark_id
    swe-bench-verified
    multilingual
    false
    analysis_method
    Accuracy
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    SWE-Bench Verified · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    3
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.604

    source id
    llmStatsArchive
    leaderboard name
    SWE-Bench Verified · Normalized (reported)
    dimension
    general
    metric
    SWE-Bench Verified · Normalized (reported)
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    SWE-Bench Verified
    language
    en
    modality
    text
    verified
    false
    max_score
    1
    categories
    • reasoning
    • frontend_development
    • code
    created_at
    2025-07-19T19:56:13.812805+00:00
    paper_link
    https://arxiv.org/abs/2310.06770
    updated_at
    2025-07-19T19:56:13.812805+00:00
    description
    A verified subset of 500 software engineering problems from real GitHub issues, validated by human annotators for evaluating language models' ability to resolve real-world coding issues by generating patches for Python codebases.
    benchmark_id
    swe-bench-verified
    multilingual
    false
    analysis_method
    Accuracy
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    SWE-Bench Verified · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    3
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    LLM Stats · GitHub archive

    0.11

    source id
    llmStatsArchive
    leaderboard name
    Humanity's Last Exam · Raw
    dimension
    general
    metric
    Humanity's Last Exam · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    Humanity's Last Exam
    language
    en
    modality
    multimodal
    verified
    false
    max_score
    1
    categories
    • general
    created_at
    2025-07-19T19:56:12.507693+00:00
    paper_link
    https://arxiv.org/abs/2501.14249
    updated_at
    2025-07-19T19:56:12.507693+00:00
    description
    A multi-modal benchmark at the frontier of human knowledge with 2,500 questions across dozens of subjects including mathematics, humanities, and natural sciences, created by nearly 1000 subject expert contributors from over 500 institutions
    benchmark_id
    humanity's-last-exam
    multilingual
    false
    analysis_method
    Accuracy
    is_self_reported
    true
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    Humanity's Last Exam · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/google/models/gemini-2.5-flash/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    2
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    gemini-2.5-flash
    Original result ↗

    Published prices

    1–20 / 45

    gemini-2.5-flash ↗

    $0.02

    Inference provider
    cheaperInference
    Platform
    Cheaper Inference
    Unit
    cache_read_tokens_per_million
    Currency
    USD
    Billing conditions
    zdr
    false
    protocol
    chat_completions
    service_tier
    standard
    Billing label
    Cached input tokens (per 1M)

    gemini-2.5-flash ↗

    $0.20

    Inference provider
    cheaperInference
    Platform
    Cheaper Inference
    Unit
    input_tokens_per_million
    Currency
    USD
    Billing conditions
    zdr
    false
    protocol
    chat_completions
    service_tier
    standard
    Billing label
    Input tokens (per 1M)

    gemini-2.5-flash ↗

    $1.63

    Inference provider
    cheaperInference
    Platform
    Cheaper Inference
    Unit
    output_tokens_per_million
    Currency
    USD
    Billing conditions
    zdr
    false
    protocol
    chat_completions
    service_tier
    standard
    Billing label
    Output tokens (per 1M)

    google/gemini-2.5-flash ↗

    $0.03

    Inference provider
    cloudflareWorkersAI
    Platform
    Google Vertex AI
    Unit
    cache_read_tokens_per_million
    Currency
    USD
    Billing conditions
    Billing label
    Cached input tokens (per 1M)
    Pricing via
    Cloudflare AI Gateway

    google/gemini-2.5-flash ↗

    $0.30

    Inference provider
    cloudflareWorkersAI
    Platform
    Google Vertex AI
    Unit
    input_tokens_per_million
    Currency
    USD
    Billing conditions
    Billing label
    Input tokens (per 1M)
    Pricing via
    Cloudflare AI Gateway

    google/gemini-2.5-flash ↗

    $2.50

    Inference provider
    cloudflareWorkersAI
    Platform
    Google Vertex AI
    Unit
    output_tokens_per_million
    Currency
    USD
    Billing conditions
    Billing label
    Output tokens (per 1M)
    Pricing via
    Cloudflare AI Gateway

    google/gemini-2.5-flash ↗

    $0.03

    Inference provider
    openRouter
    Platform
    OpenRouter
    Unit
    cache_read_tokens_per_million
    Currency
    USD
    Billing conditions

    google/gemini-2.5-flash ↗

    $0.08

    Inference provider
    openRouter
    Platform
    OpenRouter
    Unit
    cache_write_tokens_per_million
    Currency
    USD
    Billing conditions

    google/gemini-2.5-flash ↗

    $0.30

    Inference provider
    openRouter
    Platform
    OpenRouter
    Unit
    input_tokens_per_million
    Currency
    USD
    Billing conditions

    google/gemini-2.5-flash ↗

    $2.50

    Inference provider
    openRouter
    Platform
    OpenRouter
    Unit
    output_tokens_per_million
    Currency
    USD
    Billing conditions

    google/gemini-2.5-flash ↗

    $0.03

    Inference provider
    vercelAIGatewayCatalog
    Platform
    Vercel AI Gateway
    Unit
    cache_read_tokens_per_million
    Currency
    USD
    Billing conditions
    model_type
    language
    quote_scope
    gateway_catalog
    billing_scope
    Vercel public catalog; endpoint prices may differ
    Billing label
    Cached input / 1M
    Pricing via
    Vercel AI Gateway

    google/gemini-2.5-flash ↗

    $0.05

    Inference provider
    vercelAIGatewayCatalog
    Platform
    Vercel AI Gateway
    Unit
    cache_read_tokens_per_million
    Currency
    USD
    Billing conditions
    model_type
    language
    quote_scope
    gateway_catalog
    service_tier
    priority
    billing_scope
    Vercel public catalog; endpoint prices may differ
    Billing label
    Cached input / 1M
    Pricing via
    Vercel AI Gateway

    google/gemini-2.5-flash ↗

    $0.03

    Inference provider
    vercelAIGatewayCatalog
    Platform
    Vercel AI Gateway
    Unit
    cache_read_tokens_per_million
    Currency
    USD
    Billing conditions
    region
    eu
    model_type
    language
    quote_scope
    gateway_catalog
    billing_scope
    Vercel public catalog; endpoint prices may differ
    Billing label
    Cached input / 1M
    Pricing via
    Vercel AI Gateway

    google/gemini-2.5-flash ↗

    $0.03

    Inference provider
    vercelAIGatewayCatalog
    Platform
    Vercel AI Gateway
    Unit
    cache_read_tokens_per_million
    Currency
    USD
    Billing conditions
    region
    us
    model_type
    language
    quote_scope
    gateway_catalog
    billing_scope
    Vercel public catalog; endpoint prices may differ
    Billing label
    Cached input / 1M
    Pricing via
    Vercel AI Gateway

    google/gemini-2.5-flash ↗

    $0.30

    Inference provider
    vercelAIGatewayCatalog
    Platform
    Vercel AI Gateway
    Unit
    input_tokens_per_million
    Currency
    USD
    Billing conditions
    model_type
    language
    quote_scope
    gateway_catalog
    billing_scope
    Vercel public catalog; endpoint prices may differ
    Billing label
    Input tokens / 1M
    Pricing via
    Vercel AI Gateway

    google/gemini-2.5-flash ↗

    $0.15

    Inference provider
    vercelAIGatewayCatalog
    Platform
    Vercel AI Gateway
    Unit
    input_tokens_per_million
    Currency
    USD
    Billing conditions
    model_type
    language
    quote_scope
    gateway_catalog
    service_tier
    flex
    billing_scope
    Vercel public catalog; endpoint prices may differ
    Billing label
    Input tokens / 1M
    Pricing via
    Vercel AI Gateway

    Provider specifications & access

    1–4 / 4

    gemini-2.5-flash ↗

    source model id
    gemini-2.5-flash
    hosting
    gateway
    platforms
    • Cheaper Inference
    context length
    1048576
    max output tokens
    65535
    architecture
    input_modalities
    • text
    • image
    output_modalities
    • text
    capabilities
    • structuredOutputs
    • toolCalling
    • visionInput
    • webSearch

    Gemini 2.5 Flash ↗

    source model id
    google/gemini-2.5-flash
    hosting
    third-party
    platforms
    • Google Vertex AI
    context length
    1000000
    max output tokens
    8192
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities
    • reasoning

    Gemini 2.5 Flash ↗

    source model id
    google/gemini-2.5-flash
    hosting
    vercel-gateway
    platforms
    • Vercel AI Gateway
    context length
    1000000
    max output tokens
    65535
    architecture
    input_modalities
    • text
    • image
    • pdf
    output_modalities
    • text
    capabilities
    • promptCaching
    • reasoning
    • structuredOutputs
    • toolCalling
    • visionInput
    • webSearch
    regions
    • eu
    • us

    Gemini 2.5 Flash ↗

    source model id
    publishers/google/models/gemini-2.5-flash@default
    hosting
    vertex-managed
    platforms
    • Google Cloud Vertex AI
    context length
    1048576
    max output tokens
    65536
    architecture
    input_modalities
    • audio
    • image
    • text
    • video
    output_modalities
    • text
    capabilities
    • batch
    • multimodal
    • promptCaching
    • reasoning
    • structuredOutputs
    • toolCalling
    • visionInput

    Safety assessments

    1–1 / 1

    Foundation Model Transparency Index 2025 ↗

    41 / 100

    source id
    stanfordFMTI
    canonical model id
    google/gemini-2.5-flash
    taxonomy
    transparency
    hazard category
    transparency
    hazard name
    Organization transparency
    policy area
    Google
    assessment type
    organization-level disclosure
    metric
    disclosure_score
    result label
    41 / 100
    sample size
    100
    updated at
    2026-11-09T18:08:05.227Z

    Source specification observations

    1–10 / 10

    cloudflareWorkersAI ↗

    field
    context_length
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    cloudflareWorkersAI
    confidence
    1

    openRouter ↗

    field
    context_length
    value
    1048576
    normalized value
    1048576
    scope
    source-model
    source id
    openRouter
    confidence
    1

    vercelAIGatewayCatalog ↗

    field
    context_length
    value
    1000000
    normalized value
    1000000
    scope
    source-model
    source id
    vercelAIGatewayCatalog
    confidence
    1

    vertexAI ↗

    field
    context_length
    value
    1048576
    normalized value
    1048576
    scope
    source-model
    source id
    vertexAI
    confidence
    1

    openRouter ↗

    field
    knowledge_cutoff
    reported field
    knowledge_cutoff
    value
    2025-01-31
    normalized value
    2025-01-31
    precision
    day
    scope
    source-model
    source id
    openRouter
    confidence
    1

    vercelAIGatewayCatalog ↗

    field
    knowledge_cutoff
    reported field
    knowledge_cutoff
    value
    2025-01
    normalized value
    2025-01
    precision
    month
    scope
    source-model
    source id
    vercelAIGatewayCatalog
    confidence
    1

    cloudflareWorkersAI ↗

    field
    max_output_tokens
    value
    8192
    normalized value
    8192
    scope
    source-model
    source id
    cloudflareWorkersAI
    confidence
    1

    openRouter ↗

    field
    max_output_tokens
    value
    65535
    normalized value
    65535
    scope
    source-model
    source id
    openRouter
    confidence
    1

    vercelAIGatewayCatalog ↗

    field
    max_output_tokens
    value
    65535
    normalized value
    65535
    scope
    source-model
    source id
    vercelAIGatewayCatalog
    confidence
    1

    vertexAI ↗

    field
    max_output_tokens
    value
    65536
    normalized value
    65536
    scope
    source-model
    source id
    vertexAI
    confidence
    1

    Original sources

    21–22 / 22

    gorilla.cs.berkeley.edu · Catalog ↗

    www.swebench.com · Catalog ↗

    Source links

    1–15 / 15

    Explore more models

    Models

    Loading
    EVIDENCE / COMPARISON

    Benchmarks

    Explore every benchmark

    Loading

    These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

    Install TensorTrace