Google: Gemini 2.5 Flash
Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, e…
google/gemini-2.5-flash
Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, enabling it to provide responses with greater accuracy and nuanced context handling.
Additionally, Gemini 2.5 Flash is configurable through the "max tokens for reasoning" parameter, as described in the documentation (https://openrouter.ai/docs/use-cases/reasoning-tokens#max-tokens-for-reasoning).
Specifications & capabilities
- Input
- file, image, text, audio, video
- Output
- text
- Context window · tokens
- 1048576
- Knowledge cutoff
- 2025-01
- Published model ID
- gemini-2.5-flash@default
- Capabilities
- audioInput, batch, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗Evaluations
61–80 / 2,449
0.91
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite French · French
- dimension
- reasoning
- metric
- Global MMLU Lite French · French
- rank
- 16
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41752
- name
- French
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 156
- name
- Global MMLU Lite French
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 138
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite French
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite French · French
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 138/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-french/versions/1
- measurements
- name
- French
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite French · French
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.91
- standardDeviation
- 0
- confidenceInterval
- 0.0280452971732717
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0280452971732717
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.91
- standardDeviation
- 0
- confidenceInterval
- 0.0280452971732717
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0280452971732717
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41752
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-french
- sort_order
- DESCENDING
- source_name
- French
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:43.548947500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8819547028267283
- confidence upper
- 0.9380452971732718
Original result ↗0.908
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Indonesian · Indonesian
- dimension
- reasoning
- metric
- Global MMLU Lite Indonesian · Indonesian
- rank
- 16
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41740
- name
- Indonesian
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 158
- name
- Global MMLU Lite Indonesian
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 140
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Indonesian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Indonesian · Indonesian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 140/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-indonesian/versions/1
- measurements
- name
- Indonesian
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Indonesian · Indonesian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9075
- standardDeviation
- 0
- confidenceInterval
- 0.0283930651251164
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0283930651251164
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9075
- standardDeviation
- 0
- confidenceInterval
- 0.0283930651251164
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0283930651251164
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41740
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-indonesian
- sort_order
- DESCENDING
- source_name
- Indonesian
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:39.240452500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8791069348748836
- confidence upper
- 0.9358930651251164
Original result ↗0.915
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Chinese · Chinese
- dimension
- reasoning
- metric
- Global MMLU Lite Chinese · Chinese
- rank
- 17
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41735
- name
- Chinese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 166
- name
- Global MMLU Lite Chinese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 148
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Chinese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Chinese · Chinese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 148/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-chinese/versions/1
- measurements
- name
- Chinese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Chinese · Chinese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.915
- standardDeviation
- 0
- confidenceInterval
- 0.0273299039414468
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0273299039414468
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.915
- standardDeviation
- 0
- confidenceInterval
- 0.0273299039414468
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0273299039414468
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41735
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-chinese
- sort_order
- DESCENDING
- source_name
- Chinese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:22.953186100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8876700960585532
- confidence upper
- 0.9423299039414469
Original result ↗41.29%
- source id
- bfcl
- leaderboard name
- BFCL Memory
- dimension
- agentic
- metric
- Memory accuracy
- rank
- 17
Original result ↗1,251.841
- source id
- lmarena
- leaderboard name
- LMArena Vision Entity Recognition
- dimension
- multimodal
- metric
- Arena rating
- rank
- 18
- sample size
- 628
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 1227.8505859692873
- confidence upper
- 1275.8322586786646
Original result ↗0.913
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Portuguese · Portuguese
- dimension
- reasoning
- metric
- Global MMLU Lite Portuguese · Portuguese
- rank
- 18
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41746
- name
- Portuguese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 162
- name
- Global MMLU Lite Portuguese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 144
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Portuguese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Portuguese · Portuguese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 144/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-portuguese/versions/1
- measurements
- name
- Portuguese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Portuguese · Portuguese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9125
- standardDeviation
- 0
- confidenceInterval
- 0.0276909948229923
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0276909948229923
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9125
- standardDeviation
- 0
- confidenceInterval
- 0.0276909948229923
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0276909948229923
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41746
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-portuguese
- sort_order
- DESCENDING
- source_name
- Portuguese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:30.662890500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8848090051770077
- confidence upper
- 0.9401909948229923
Original result ↗38.71%
- source id
- bfcl
- leaderboard name
- BFCL Memory
- dimension
- agentic
- metric
- Memory accuracy
- rank
- 18
Original result ↗0.903
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite German · German
- dimension
- reasoning
- metric
- Global MMLU Lite German · German
- rank
- 20
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41738
- name
- German
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 155
- name
- Global MMLU Lite German
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 137
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite German
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite German · German
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 137/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-german/versions/1
- measurements
- name
- German
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite German · German
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9025
- standardDeviation
- 0
- confidenceInterval
- 0.0290699315059157
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0290699315059157
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9025
- standardDeviation
- 0
- confidenceInterval
- 0.0290699315059157
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0290699315059157
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41738
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-german
- sort_order
- DESCENDING
- source_name
- German
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:45.767915600Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8734300684940842
- confidence upper
- 0.9315699315059157
Original result ↗62%
- source id
- bfcl
- leaderboard name
- BFCL Web Search
- dimension
- agentic
- metric
- Web-search accuracy
- rank
- 20
Original result ↗1,221.155
- source id
- lmarena
- leaderboard name
- LMArena Vision Style Control Entity Recognition
- dimension
- multimodal
- metric
- Arena rating
- rank
- 21
- sample size
- 628
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 1196.8978619359902
- confidence upper
- 1245.411326152644
Original result ↗88.08%
- source id
- bfcl
- leaderboard name
- BFCL Non-Live AST
- dimension
- toolUse
- metric
- Non-live AST accuracy
- rank
- 21
Original result ↗59%
- source id
- bfcl
- leaderboard name
- BFCL Web Search
- dimension
- agentic
- metric
- Web-search accuracy
- rank
- 21
Original result ↗50.9%
- source id
- bfcl
- leaderboard name
- BFCL Overall
- dimension
- toolUse
- metric
- Overall accuracy
- rank
- 26
Original result ↗78.16%
- source id
- bfcl
- leaderboard name
- BFCL Live
- dimension
- toolUse
- metric
- Live accuracy
- rank
- 27
Original result ↗33.655%
- source id
- kaggleBenchmarks
- leaderboard name
- SimpleQA · simpleqa_task
- dimension
- knowledge
- metric
- SimpleQA · simpleqa_task
- rank
- 27
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40103
- name
- simpleqa_task
- version
- 7
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 40
- name
- SimpleQA
- type
- INDIVIDUAL
- owner
- openai
- version
- 1
- version_id
- 26
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · openai · SimpleQA
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- SimpleQA · simpleqa_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 26/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/openai/simpleqa/versions/1
- measurements
- name
- simpleqa_task
- unit
- percent
- scale
- 1
- metric
- SimpleQA · simpleqa_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.3365462780164905
- standardDeviation
- 0
- confidenceInterval
- 0.0143611653772581
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0143611653772581
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2026-03-10T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.3365462780164905
- standardDeviation
- 0
- confidenceInterval
- 0.0143611653772581
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0143611653772581
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40103
- task_url
- /benchmarks/tasks/benchmarkler/simpleqa-task
- sort_order
- DESCENDING
- source_name
- simpleqa_task
- display_type
- percentages
- task_version
- 7
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 49
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2026-03-10T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.3221851126392324
- confidence upper
- 0.35090744339374863
Original result ↗49.23%
- source id
- kaggleBenchmarks
- leaderboard name
- ParseBench: Doc OCR for AI agents · Score
- dimension
- knowledge
- metric
- ParseBench: Doc OCR for AI agents · Score
- rank
- 28
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 392887
- name
- Score
- version
- 4
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 4974
- name
- ParseBench: Doc OCR for AI agents
- type
- INDIVIDUAL
- owner
- llamaindex-org
- version
- 1
- version_id
- 4997
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · llamaindex-org · ParseBench: Doc OCR for AI agents
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- ParseBench: Doc OCR for AI agents · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 4997/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/llamaindex-org/parsebench/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- ParseBench: Doc OCR for AI agents · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.49229519491630386
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2026-04-16T16:42:55.059072Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.49229519491630386
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 392887
- task_url
- /benchmarks/tasks/benchmarkler/parsebench
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 4
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2026-04-17T23:39:17.199068200Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 34
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2026-04-16T16:42:55.059Z
- source model id
- Gemini 2.5 Flash
Original result ↗73.85%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Google
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 30
- sample size
- 1188
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Google
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-01T19:15:31.968Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1188
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.7384959999999999
- accuracy_std_dev
- 0.008170584087648423
- cost_per_task_usd
- 0.06071549915824916
- output_tokens_per_task
- 24199.07239057239
- latency_seconds_per_task
- 125.90270959595959
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 40
- benchmark_evaluation
- true
- openrouter_permaslug
- google/gemini-2.5-flash
- source model id
- google/gemini-2.5-flash
Original result ↗1,411.346
- source id
- lmarena
- leaderboard name
- LMArena Text Factuality German
- dimension
- factuality
- metric
- Arena rating
- rank
- 31
- sample size
- 570
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 1389.1402422977237
- confidence upper
- 1433.5524680951612
Original result ↗59.111%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Google
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 31
- sample size
- 300
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Google
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-01T19:05:46.198Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 300
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.591111
- accuracy_std_dev
- 0.006285295405415193
- cost_per_task_usd
- 0.029060479999999996
- output_tokens_per_task
- 7763.98
- latency_seconds_per_task
- 77.75587
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 36
- benchmark_evaluation
- true
- openrouter_permaslug
- google/gemini-2.5-flash
- source model id
- google/gemini-2.5-flash
Original result ↗36.25%
- source id
- bfcl
- leaderboard name
- BFCL Multi Turn
- dimension
- agentic
- metric
- Multi-turn accuracy
- rank
- 32
Original result ↗Published prices
1–20 / 45
$0.02
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cached input tokens (per 1M)
$0.20
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Input tokens (per 1M)
$1.63
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Output tokens (per 1M)
$0.03
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.30
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$2.50
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.30
- Inference provider
- deepInfra
- Platform
- DeepInfra
- Unit
- input_tokens_per_million
- Currency
- USD
$2.50
- Inference provider
- deepInfra
- Platform
- DeepInfra
- Unit
- output_tokens_per_million
- Currency
- USD
$0.03
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.08
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_write_tokens_per_million
- Currency
- USD
$0.00000030
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- image
- Currency
- USD
$0.30
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$2.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.05
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- service_tier
- priority
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- eu
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- us
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.30
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.15
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- service_tier
- flex
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–4 / 4
- source model id
- gemini-2.5-flash
- hosting
- gateway
- context length
- 1048576
- max output tokens
- 65535
- capabilities
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- google/gemini-2.5-flash
- hosting
- third-party
- context length
- 1000000
- max output tokens
- 8192
- source model id
- google/gemini-2.5-flash
- hosting
- vercel-gateway
- context length
- 1000000
- max output tokens
- 65535
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- publishers/google/models/gemini-2.5-flash@default
- hosting
- vertex-managed
- context length
- 1048576
- max output tokens
- 65536
- capabilities
- batch
- multimodal
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- visionInput
Safety assessments
1–1 / 1
41 / 100
- source id
- stanfordFMTI
- canonical model id
- google/gemini-2.5-flash
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Google
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 41 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–10 / 10
- field
- context_length
- value
- 1000000
- normalized value
- 1000000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1000000
- normalized value
- 1000000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- vertexAI
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-01-31
- normalized value
- 2025-01-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-01
- normalized value
- 2025-01
- precision
- month
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 8192
- normalized value
- 8192
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 65535
- normalized value
- 65535
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 65535
- normalized value
- 65535
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 65536
- normalized value
- 65536
- scope
- source-model
- source id
- vertexAI
- confidence
- 1
Explore more models