Google: Gemini 2.5 Flash
Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, e…
google/gemini-2.5-flash
Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, enabling it to provide responses with greater accuracy and nuanced context handling.
Additionally, Gemini 2.5 Flash is configurable through the "max tokens for reasoning" parameter, as described in the documentation (https://openrouter.ai/docs/use-cases/reasoning-tokens#max-tokens-for-reasoning).
Specifications & capabilities
- Input
- file, image, text, audio, video
- Output
- text
- Context window · tokens
- 1048576
- Knowledge cutoff
- 2025-01
- Published model ID
- gemini-2.5-flash@default
- Capabilities
- audioInput, batch, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗Evaluations
1–20 / 2,449
0.853
- source id
- llmStatsArchive
- leaderboard name
- FACTS Grounding · Raw
- dimension
- general
- metric
- FACTS Grounding · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- FACTS Grounding
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.260285+00:00
- paper_link
- https://arxiv.org/abs/2501.03200
- updated_at
- 2025-07-19T19:56:13.260285+00:00
- description
- A benchmark evaluating language models' ability to generate factually accurate and well-grounded responses based on long-form input context, comprising 1,719 examples with documents up to 32k tokens requiring detailed responses that are fully grounded in provided documents
- benchmark_id
- facts-grounding
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- FACTS Grounding · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.853
- source id
- llmStatsArchive
- leaderboard name
- FACTS Grounding · Normalized (reported)
- dimension
- general
- metric
- FACTS Grounding · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- FACTS Grounding
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.260285+00:00
- paper_link
- https://arxiv.org/abs/2501.03200
- updated_at
- 2025-07-19T19:56:13.260285+00:00
- description
- A benchmark evaluating language models' ability to generate factually accurate and well-grounded responses based on long-form input context, comprising 1,719 examples with documents up to 32k tokens requiring detailed responses that are fully grounded in provided documents
- benchmark_id
- facts-grounding
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- FACTS Grounding · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.797
- source id
- llmStatsArchive
- leaderboard name
- MMMU · Raw
- dimension
- general
- metric
- MMMU · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMMU
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- categories
- multimodal
- reasoning
- general
- created_at
- 2025-07-19T19:56:12.130105+00:00
- paper_link
- https://arxiv.org/abs/2311.16502
- updated_at
- 2025-07-19T19:56:12.130105+00:00
- description
- MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
- benchmark_id
- mmmu
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMMU · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.797
- source id
- llmStatsArchive
- leaderboard name
- MMMU · Normalized (reported)
- dimension
- general
- metric
- MMMU · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMMU
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- categories
- multimodal
- reasoning
- general
- created_at
- 2025-07-19T19:56:12.130105+00:00
- paper_link
- https://arxiv.org/abs/2311.16502
- updated_at
- 2025-07-19T19:56:12.130105+00:00
- description
- MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
- benchmark_id
- mmmu
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMMU · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗50%
- source id
- kaggleBenchmarks
- leaderboard name
- ITBench · SRE Score
- dimension
- reasoning
- metric
- ITBench · SRE Score
- rank
- 1
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1172
- name
- SRE Score
- version
- 8
- sort_order
- DESCENDING
- display_type
- rawPercentages
- benchmark
- id
- 246
- name
- ITBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 225
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · ITBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- ITBench · SRE Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 225/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/itbench/versions/1
- measurements
- name
- SRE Score
- unit
- percent
- scale
- 100
- metric
- ITBench · SRE Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 50
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-12-01T22:28:02.522152Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 50
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1172
- task_url
- /benchmarks/tasks/benchmarkler/itbench-sre-main
- sort_order
- DESCENDING
- source_name
- SRE Score
- display_type
- rawPercentages
- task_version
- 8
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-02T12:36:17.407125500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 13
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-12-01T22:28:02.522Z
- source model id
- Gemini 2.5 Flash
Original result ↗0.884
- source id
- llmStatsArchive
- leaderboard name
- Global-MMLU-Lite · Normalized (reported)
- dimension
- general
- metric
- Global-MMLU-Lite · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Global-MMLU-Lite
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.534515+00:00
- paper_link
- https://arxiv.org/abs/2412.03304
- updated_at
- 2025-07-19T19:56:13.534515+00:00
- description
- A lightweight version of Global MMLU benchmark that evaluates language models across multiple languages while addressing cultural and linguistic biases in multilingual evaluation.
- benchmark_id
- global-mmlu-lite
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Global-MMLU-Lite · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.884
- source id
- llmStatsArchive
- leaderboard name
- Global-MMLU-Lite · Raw
- dimension
- general
- metric
- Global-MMLU-Lite · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Global-MMLU-Lite
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.534515+00:00
- paper_link
- https://arxiv.org/abs/2412.03304
- updated_at
- 2025-07-19T19:56:13.534515+00:00
- description
- A lightweight version of Global MMLU benchmark that evaluates language models across multiple languages while addressing cultural and linguistic biases in multilingual evaluation.
- benchmark_id
- global-mmlu-lite
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Global-MMLU-Lite · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.88
- source id
- llmStatsArchive
- leaderboard name
- AIME 2024 · Normalized (reported)
- dimension
- general
- metric
- AIME 2024 · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AIME 2024
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.941652+00:00
- paper_link
- https://arxiv.org/html/2503.21380v2
- updated_at
- 2025-09-30T00:00:00.000000+00:00
- description
- American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
- benchmark_id
- aime-2024
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AIME 2024 · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.88
- source id
- llmStatsArchive
- leaderboard name
- AIME 2024 · Raw
- dimension
- general
- metric
- AIME 2024 · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AIME 2024
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.941652+00:00
- paper_link
- https://arxiv.org/html/2503.21380v2
- updated_at
- 2025-09-30T00:00:00.000000+00:00
- description
- American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
- benchmark_id
- aime-2024
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AIME 2024 · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.828
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Raw
- dimension
- general
- metric
- GPQA · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.828
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Normalized (reported)
- dimension
- general
- metric
- GPQA · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.72
- source id
- llmStatsArchive
- leaderboard name
- AIME 2025 · Raw
- dimension
- general
- metric
- AIME 2025 · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AIME 2025
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2503.21380
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- All 30 problems from the 2025 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999. Used as an AI benchmark to evaluate large language models' ability to solve complex mathematical problems requiring multi-step logical deductions and structured symbolic reasoning.
- benchmark_id
- aime-2025
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AIME 2025 · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.72
- source id
- llmStatsArchive
- leaderboard name
- AIME 2025 · Normalized (reported)
- dimension
- general
- metric
- AIME 2025 · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AIME 2025
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2503.21380
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- All 30 problems from the 2025 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999. Used as an AI benchmark to evaluate large language models' ability to solve complex mathematical problems requiring multi-step logical deductions and structured symbolic reasoning.
- benchmark_id
- aime-2025
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AIME 2025 · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.654
- source id
- llmStatsArchive
- leaderboard name
- Vibe-Eval · Raw
- dimension
- general
- metric
- Vibe-Eval · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Vibe-Eval
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.871369+00:00
- paper_link
- https://arxiv.org/abs/2405.02287
- updated_at
- 2025-07-19T19:56:13.871369+00:00
- description
- VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
- benchmark_id
- vibe-eval
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Vibe-Eval · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.654
- source id
- llmStatsArchive
- leaderboard name
- Vibe-Eval · Normalized (reported)
- dimension
- general
- metric
- Vibe-Eval · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Vibe-Eval
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.871369+00:00
- paper_link
- https://arxiv.org/abs/2405.02287
- updated_at
- 2025-07-19T19:56:13.871369+00:00
- description
- VIBE-Eval is a hard evaluation suite for measuring progress of multimodal language models, consisting of 269 visual understanding prompts with gold-standard responses authored by experts. The benchmark has dual objectives: vibe checking multimodal chat models for day-to-day tasks and rigorously testing frontier models, with the hard set containing >50% questions that all frontier models answer incorrectly.
- benchmark_id
- vibe-eval
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Vibe-Eval · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.639
- source id
- llmStatsArchive
- leaderboard name
- LiveCodeBench v5 · Normalized (reported)
- dimension
- general
- metric
- LiveCodeBench v5 · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- LiveCodeBench v5
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.759330+00:00
- paper_link
- https://arxiv.org/abs/2403.07974
- updated_at
- 2025-07-19T19:56:13.759330+00:00
- description
- LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
- benchmark_id
- livecodebench-v5
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- LiveCodeBench v5 · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.639
- source id
- llmStatsArchive
- leaderboard name
- LiveCodeBench v5 · Raw
- dimension
- general
- metric
- LiveCodeBench v5 · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- LiveCodeBench v5
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.759330+00:00
- paper_link
- https://arxiv.org/abs/2403.07974
- updated_at
- 2025-07-19T19:56:13.759330+00:00
- description
- LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
- benchmark_id
- livecodebench-v5
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- LiveCodeBench v5 · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.604
- source id
- llmStatsArchive
- leaderboard name
- SWE-Bench Verified · Raw
- dimension
- general
- metric
- SWE-Bench Verified · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- SWE-Bench Verified
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- reasoning
- frontend_development
- code
- created_at
- 2025-07-19T19:56:13.812805+00:00
- paper_link
- https://arxiv.org/abs/2310.06770
- updated_at
- 2025-07-19T19:56:13.812805+00:00
- description
- A verified subset of 500 software engineering problems from real GitHub issues, validated by human annotators for evaluating language models' ability to resolve real-world coding issues by generating patches for Python codebases.
- benchmark_id
- swe-bench-verified
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- SWE-Bench Verified · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.604
- source id
- llmStatsArchive
- leaderboard name
- SWE-Bench Verified · Normalized (reported)
- dimension
- general
- metric
- SWE-Bench Verified · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- SWE-Bench Verified
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- reasoning
- frontend_development
- code
- created_at
- 2025-07-19T19:56:13.812805+00:00
- paper_link
- https://arxiv.org/abs/2310.06770
- updated_at
- 2025-07-19T19:56:13.812805+00:00
- description
- A verified subset of 500 software engineering problems from real GitHub issues, validated by human annotators for evaluating language models' ability to resolve real-world coding issues by generating patches for Python codebases.
- benchmark_id
- swe-bench-verified
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- SWE-Bench Verified · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗0.11
- source id
- llmStatsArchive
- leaderboard name
- Humanity's Last Exam · Raw
- dimension
- general
- metric
- Humanity's Last Exam · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Humanity's Last Exam
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.507693+00:00
- paper_link
- https://arxiv.org/abs/2501.14249
- updated_at
- 2025-07-19T19:56:12.507693+00:00
- description
- A multi-modal benchmark at the frontier of human knowledge with 2,500 questions across dozens of subjects including mathematics, humanities, and natural sciences, created by nearly 1000 subject expert contributors from over 500 institutions
- benchmark_id
- humanity's-last-exam
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Humanity's Last Exam · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/google/models/gemini-2.5-flash/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://developers.googleblog.com/en/start-building-with-gemini-25-flash/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- gemini-2.5-flash
Original result ↗Published prices
1–20 / 45
$0.02
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cached input tokens (per 1M)
$0.20
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Input tokens (per 1M)
$1.63
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Output tokens (per 1M)
$0.03
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.30
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$2.50
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.30
- Inference provider
- deepInfra
- Platform
- DeepInfra
- Unit
- input_tokens_per_million
- Currency
- USD
$2.50
- Inference provider
- deepInfra
- Platform
- DeepInfra
- Unit
- output_tokens_per_million
- Currency
- USD
$0.03
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.08
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_write_tokens_per_million
- Currency
- USD
$0.00000030
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- image
- Currency
- USD
$0.30
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$2.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.05
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- service_tier
- priority
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- eu
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- us
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.30
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.15
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- service_tier
- flex
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–4 / 4
- source model id
- gemini-2.5-flash
- hosting
- gateway
- context length
- 1048576
- max output tokens
- 65535
- capabilities
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- google/gemini-2.5-flash
- hosting
- third-party
- context length
- 1000000
- max output tokens
- 8192
- source model id
- google/gemini-2.5-flash
- hosting
- vercel-gateway
- context length
- 1000000
- max output tokens
- 65535
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- publishers/google/models/gemini-2.5-flash@default
- hosting
- vertex-managed
- context length
- 1048576
- max output tokens
- 65536
- capabilities
- batch
- multimodal
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- visionInput
Safety assessments
1–1 / 1
41 / 100
- source id
- stanfordFMTI
- canonical model id
- google/gemini-2.5-flash
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Google
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 41 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–10 / 10
- field
- context_length
- value
- 1000000
- normalized value
- 1000000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1000000
- normalized value
- 1000000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- vertexAI
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-01-31
- normalized value
- 2025-01-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-01
- normalized value
- 2025-01
- precision
- month
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 8192
- normalized value
- 8192
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 65535
- normalized value
- 65535
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 65535
- normalized value
- 65535
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 65536
- normalized value
- 65536
- scope
- source-model
- source id
- vertexAI
- confidence
- 1
Explore more models