Google: Gemini 2.5 Flash
Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, e…
google/gemini-2.5-flash
Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, enabling it to provide responses with greater accuracy and nuanced context handling.
Additionally, Gemini 2.5 Flash is configurable through the "max tokens for reasoning" parameter, as described in the documentation (https://openrouter.ai/docs/use-cases/reasoning-tokens#max-tokens-for-reasoning).
Specifications & capabilities
- Input
- file, image, text, audio, video
- Output
- text
- Context window · tokens
- 1048576
- Knowledge cutoff
- 2025-01
- Published model ID
- gemini-2.5-flash@default
- Capabilities
- audioInput, batch, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗Evaluations
41–60 / 2,449
0.908
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Yoruba · Yoruba
- dimension
- reasoning
- metric
- Global MMLU Lite Yoruba · Yoruba
- rank
- 12
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41743
- name
- Yoruba
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 165
- name
- Global MMLU Lite Yoruba
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 147
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Yoruba
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Yoruba · Yoruba
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 147/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-yoruba/versions/1
- measurements
- name
- Yoruba
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Yoruba · Yoruba
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9075
- standardDeviation
- 0
- confidenceInterval
- 0.0283930651251164
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0283930651251164
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9075
- standardDeviation
- 0
- confidenceInterval
- 0.0283930651251164
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0283930651251164
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41743
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-yoruba
- sort_order
- DESCENDING
- source_name
- Yoruba
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:24.645552400Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8791069348748836
- confidence upper
- 0.9358930651251164
Original result ↗60.78%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · Selection Accuracy
- dimension
- reasoning
- metric
- AssetOpsBench · Selection Accuracy
- rank
- 12
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1042
- name
- Selection Accuracy
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · Selection Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- Selection Accuracy
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · Selection Accuracy
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.6078006500541712
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-22T14:59:40.578777Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.6078006500541712
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1042
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-selectionaccuracy-benchmark-task
- sort_order
- DESCENDING
- source_name
- Selection Accuracy
- display_type
- percentages
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-22T14:59:40.578Z
- source model id
- Gemini 2.5 Flash
Original result ↗1,230.888
- source id
- lmarena
- leaderboard name
- LMArena Vision Style Control Creative Writing
- dimension
- multimodal
- metric
- Arena rating
- rank
- 13
- sample size
- 1745
- evaluated at
- 2026-01-09T00:00:00.000Z
- source model id
- gemini-2.5-flash
- confidence lower
- 1215.862144676827
- confidence upper
- 1245.9129322178349
Original result ↗1,209.802
- source id
- lmarena
- leaderboard name
- LMArena Vision Style Control Captioning
- dimension
- multimodal
- metric
- Arena rating
- rank
- 13
- sample size
- 578
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 1183.7522709510938
- confidence upper
- 1235.8527227184636
Original result ↗0.929
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Culturally Agnostic · CA Score
- dimension
- reasoning
- metric
- Global MMLU Lite Culturally Agnostic · CA Score
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41744
- name
- CA Score
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 209
- name
- Global MMLU Lite Culturally Agnostic
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 188
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Culturally Agnostic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Culturally Agnostic · CA Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 188/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-ca/versions/1
- measurements
- name
- CA Score
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Culturally Agnostic · CA Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9290625
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-03T22:31:06.2733333Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9290625
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41744
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-ca-score
- sort_order
- DESCENDING
- source_name
- CA Score
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-11-10T16:52:12.590864700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-03T22:31:06.273Z
- source model id
- Gemini 2.5 Flash
Original result ↗0.925
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Hindi · Hindi
- dimension
- reasoning
- metric
- Global MMLU Lite Hindi · Hindi
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41742
- name
- Hindi
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 157
- name
- Global MMLU Lite Hindi
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 139
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Hindi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Hindi · Hindi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 139/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-hindi/versions/1
- measurements
- name
- Hindi
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Hindi · Hindi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.925
- standardDeviation
- 0
- confidenceInterval
- 0.0258118773864695
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0258118773864695
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.925
- standardDeviation
- 0
- confidenceInterval
- 0.0258118773864695
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0258118773864695
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41742
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-hindi
- sort_order
- DESCENDING
- source_name
- Hindi
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:41.662768400Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8991881226135305
- confidence upper
- 0.9508118773864696
Original result ↗0.915
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Burmese · Burmese
- dimension
- reasoning
- metric
- Global MMLU Lite Burmese · Burmese
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41736
- name
- Burmese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 175
- name
- Global MMLU Lite Burmese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 153
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Burmese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Burmese · Burmese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 153/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-burmese/versions/1
- measurements
- name
- Burmese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Burmese · Burmese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.915
- standardDeviation
- 0
- confidenceInterval
- 0.0273299039414468
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0273299039414468
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.915
- standardDeviation
- 0
- confidenceInterval
- 0.0273299039414468
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0273299039414468
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41736
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-burmese
- sort_order
- DESCENDING
- source_name
- Burmese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:19.694332700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8876700960585532
- confidence upper
- 0.9423299039414469
Original result ↗0.913
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Arabic · Arabic
- dimension
- reasoning
- metric
- Global MMLU Lite Arabic · Arabic
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41737
- name
- Arabic
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 148
- name
- Global MMLU Lite Arabic
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 130
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Arabic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Arabic · Arabic
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 130/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-arabic/versions/1
- measurements
- name
- Arabic
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Arabic · Arabic
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9125
- standardDeviation
- 0
- confidenceInterval
- 0.0276909948229923
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0276909948229923
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9125
- standardDeviation
- 0
- confidenceInterval
- 0.0276909948229923
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0276909948229923
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41737
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-arabic
- sort_order
- DESCENDING
- source_name
- Arabic
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:54.963186300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8848090051770077
- confidence upper
- 0.9401909948229923
Original result ↗60.817%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · Pert. Complex Accuracy
- dimension
- reasoning
- metric
- AssetOpsBench · Pert. Complex Accuracy
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1044
- name
- Pert. Complex Accuracy
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · Pert. Complex Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- Pert. Complex Accuracy
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · Pert. Complex Accuracy
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.6081739782527185
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-24T03:34:11.002741Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.6081739782527185
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1044
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-perturbationcomplex-benchmark-task
- sort_order
- DESCENDING
- source_name
- Pert. Complex Accuracy
- display_type
- percentages
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-24T03:34:11.002Z
- source model id
- Gemini 2.5 Flash
Original result ↗56.13%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · Consistency
- dimension
- reasoning
- metric
- AssetOpsBench · Consistency
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1045
- name
- Consistency
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · Consistency
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- Consistency
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · Consistency
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.5613048368953881
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-24T03:35:08.264072Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.5613048368953881
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1045
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-consistency-benchmark-task
- sort_order
- DESCENDING
- source_name
- Consistency
- display_type
- percentages
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-24T03:35:08.264Z
- source model id
- Gemini 2.5 Flash
Original result ↗0.933
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite English · English
- dimension
- reasoning
- metric
- Global MMLU Lite English · English
- rank
- 14
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41749
- name
- English
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 153
- name
- Global MMLU Lite English
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 135
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite English
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite English · English
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 135/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-english/versions/1
- measurements
- name
- English
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite English · English
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9325
- standardDeviation
- 0
- confidenceInterval
- 0.0245863693763976
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0245863693763976
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9325
- standardDeviation
- 0
- confidenceInterval
- 0.0245863693763976
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0245863693763976
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41749
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-english
- sort_order
- DESCENDING
- source_name
- English
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:49.982103800Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.9079136306236024
- confidence upper
- 0.9570863693763976
Original result ↗0.915
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Swahili · Swahili
- dimension
- reasoning
- metric
- Global MMLU Lite Swahili · Swahili
- rank
- 14
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41750
- name
- Swahili
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 164
- name
- Global MMLU Lite Swahili
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 146
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Swahili
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Swahili · Swahili
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 146/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-swahili/versions/1
- measurements
- name
- Swahili
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Swahili · Swahili
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.915
- standardDeviation
- 0
- confidenceInterval
- 0.0273299039414468
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0273299039414468
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.915
- standardDeviation
- 0
- confidenceInterval
- 0.0273299039414468
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0273299039414468
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41750
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-swahili
- sort_order
- DESCENDING
- source_name
- Swahili
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:27.253350900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8876700960585532
- confidence upper
- 0.9423299039414469
Original result ↗0.91
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Bengali · Bengali
- dimension
- reasoning
- metric
- Global MMLU Lite Bengali · Bengali
- rank
- 14
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41751
- name
- Bengali
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 154
- name
- Global MMLU Lite Bengali
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 136
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Bengali
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Bengali · Bengali
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 136/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-bengali/versions/1
- measurements
- name
- Bengali
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Bengali · Bengali
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.91
- standardDeviation
- 0
- confidenceInterval
- 0.0280452971732717
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0280452971732717
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.91
- standardDeviation
- 0
- confidenceInterval
- 0.0280452971732717
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0280452971732717
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41751
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-bengali
- sort_order
- DESCENDING
- source_name
- Bengali
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:47.931282Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8819547028267283
- confidence upper
- 0.9380452971732718
Original result ↗62.242%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · Pert. Simple Accuracy
- dimension
- reasoning
- metric
- AssetOpsBench · Pert. Simple Accuracy
- rank
- 14
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1043
- name
- Pert. Simple Accuracy
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · Pert. Simple Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- Pert. Simple Accuracy
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · Pert. Simple Accuracy
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.6224221972253469
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-23T14:36:08.084592Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.6224221972253469
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1043
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-perturbationsimple-benchmark-task
- sort_order
- DESCENDING
- source_name
- Pert. Simple Accuracy
- display_type
- percentages
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-23T14:36:08.084Z
- source model id
- Gemini 2.5 Flash
Original result ↗0.915
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Korean · Korean
- dimension
- reasoning
- metric
- Global MMLU Lite Korean · Korean
- rank
- 15
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41748
- name
- Korean
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 161
- name
- Global MMLU Lite Korean
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 143
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Korean
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Korean · Korean
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 143/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-korean/versions/1
- measurements
- name
- Korean
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Korean · Korean
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.915
- standardDeviation
- 0
- confidenceInterval
- 0.0273299039414468
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0273299039414468
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.915
- standardDeviation
- 0
- confidenceInterval
- 0.0273299039414468
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0273299039414468
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41748
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-korean
- sort_order
- DESCENDING
- source_name
- Korean
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:33.197926100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8876700960585532
- confidence upper
- 0.9423299039414469
Original result ↗0.913
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Japanese · Japanese
- dimension
- reasoning
- metric
- Global MMLU Lite Japanese · Japanese
- rank
- 15
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41745
- name
- Japanese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 160
- name
- Global MMLU Lite Japanese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 142
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Japanese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Japanese · Japanese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 142/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-japanese/versions/1
- measurements
- name
- Japanese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Japanese · Japanese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9125
- standardDeviation
- 0
- confidenceInterval
- 0.0276909948229923
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0276909948229923
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9125
- standardDeviation
- 0
- confidenceInterval
- 0.0276909948229923
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0276909948229923
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41745
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-japanese
- sort_order
- DESCENDING
- source_name
- Japanese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:34.896663Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8848090051770077
- confidence upper
- 0.9401909948229923
Original result ↗0.9
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Culturally Sensitive · CS Score
- dimension
- reasoning
- metric
- Global MMLU Lite Culturally Sensitive · CS Score
- rank
- 15
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41747
- name
- CS Score
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 208
- name
- Global MMLU Lite Culturally Sensitive
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 187
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Culturally Sensitive
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Culturally Sensitive · CS Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 187/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-cs/versions/1
- measurements
- name
- CS Score
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Culturally Sensitive · CS Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-03T22:31:06.2733333Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41747
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-cs-score
- sort_order
- DESCENDING
- source_name
- CS Score
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-11-10T16:52:15.372518Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-03T22:31:06.273Z
- source model id
- Gemini 2.5 Flash
Original result ↗56.24%
- source id
- bfcl
- leaderboard name
- BFCL Overall
- dimension
- toolUse
- metric
- Overall accuracy
- rank
- 15
Original result ↗0.923
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Italian · Italian
- dimension
- reasoning
- metric
- Global MMLU Lite Italian · Italian
- rank
- 16
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41739
- name
- Italian
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 159
- name
- Global MMLU Lite Italian
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 141
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Italian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Italian · Italian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 141/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-italian/versions/1
- measurements
- name
- Italian
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Italian · Italian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9225
- standardDeviation
- 0
- confidenceInterval
- 0.0262030674045044
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0262030674045044
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9225
- standardDeviation
- 0
- confidenceInterval
- 0.0262030674045044
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0262030674045044
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41739
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-italian
- sort_order
- DESCENDING
- source_name
- Italian
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:36.963753600Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8962969325954956
- confidence upper
- 0.9487030674045044
Original result ↗0.918
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Spanish · Spanish
- dimension
- reasoning
- metric
- Global MMLU Lite Spanish · Spanish
- rank
- 16
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41741
- name
- Spanish
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 163
- name
- Global MMLU Lite Spanish
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 145
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Spanish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Spanish · Spanish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 145/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-spanish/versions/1
- measurements
- name
- Spanish
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Spanish · Spanish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.9175
- standardDeviation
- 0
- confidenceInterval
- 0.0269617517795541
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0269617517795541
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.9175
- standardDeviation
- 0
- confidenceInterval
- 0.0269617517795541
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0269617517795541
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41741
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-spanish
- sort_order
- DESCENDING
- source_name
- Spanish
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:28.873808200Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- google/gemini-2.5-flash
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- Gemini 2.5 Flash
- confidence lower
- 0.8905382482204459
- confidence upper
- 0.944461751779554
Original result ↗Published prices
1–20 / 45
$0.02
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Cached input tokens (per 1M)
$0.20
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Input tokens (per 1M)
$1.63
- Inference provider
- cheaperInference
- Platform
- Cheaper Inference
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- zdr
- false
- protocol
- chat_completions
- service_tier
- standard
- Billing label
- Output tokens (per 1M)
$0.03
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.30
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$2.50
- Inference provider
- cloudflareWorkersAI
- Platform
- Google Vertex AI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.30
- Inference provider
- deepInfra
- Platform
- DeepInfra
- Unit
- input_tokens_per_million
- Currency
- USD
$2.50
- Inference provider
- deepInfra
- Platform
- DeepInfra
- Unit
- output_tokens_per_million
- Currency
- USD
$0.03
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.08
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_write_tokens_per_million
- Currency
- USD
$0.00000030
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- image
- Currency
- USD
$0.30
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$2.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.05
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- service_tier
- priority
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- eu
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- us
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.30
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.15
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- service_tier
- flex
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–4 / 4
- source model id
- gemini-2.5-flash
- hosting
- gateway
- context length
- 1048576
- max output tokens
- 65535
- capabilities
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- google/gemini-2.5-flash
- hosting
- third-party
- context length
- 1000000
- max output tokens
- 8192
- source model id
- google/gemini-2.5-flash
- hosting
- vercel-gateway
- context length
- 1000000
- max output tokens
- 65535
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- visionInput
- webSearch
- source model id
- publishers/google/models/gemini-2.5-flash@default
- hosting
- vertex-managed
- context length
- 1048576
- max output tokens
- 65536
- capabilities
- batch
- multimodal
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- visionInput
Safety assessments
1–1 / 1
41 / 100
- source id
- stanfordFMTI
- canonical model id
- google/gemini-2.5-flash
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Google
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 41 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–10 / 10
- field
- context_length
- value
- 1000000
- normalized value
- 1000000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1000000
- normalized value
- 1000000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- vertexAI
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-01-31
- normalized value
- 2025-01-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-01
- normalized value
- 2025-01
- precision
- month
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 8192
- normalized value
- 8192
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 65535
- normalized value
- 65535
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 65535
- normalized value
- 65535
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 65536
- normalized value
- 65536
- scope
- source-model
- source id
- vertexAI
- confidence
- 1
Explore more models