OpenAI: o3 Mini (batch)
OpenAI o3-mini is a cost-efficient language model optimized for STEM reasoning tasks, particularly excelling in science, mathematics, and coding. This model supports the `reasoning_effort` parameter,…
openai/o3-mini-2025-01-31
OpenAI o3-mini is a cost-efficient language model optimized for STEM reasoning tasks, particularly excelling in science, mathematics, and coding. This model supports the `reasoning_effort` parameter, which can be set to...
Specifications & capabilities
- Input
- text, file
- Output
- text
- Context window · tokens
- 200000
- Published model ID
- o3-mini-2025-01-31
- Capabilities
- fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling
Evaluations
21–40 / 130
30.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Spanish · EM Spanish
- dimension
- reasoning
- metric
- MultiLoKo Spanish · EM Spanish
- rank
- 28
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 52
- name
- EM Spanish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 75
- name
- MultiLoKo Spanish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 61
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Spanish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Spanish · EM Spanish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 61/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-spanish/versions/1
- measurements
- name
- EM Spanish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Spanish · EM Spanish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.304
- standardDeviation
- 0
- confidenceInterval
- 0.0570189871224575
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0570189871224575
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.304
- standardDeviation
- 0
- confidenceInterval
- 0.0570189871224575
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0570189871224575
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 52
- sort_order
- DESCENDING
- source_name
- EM Spanish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.24698101287754248
- confidence upper
- 0.3610189871224575
Original result ↗18.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Turkish · EM Turkish
- dimension
- reasoning
- metric
- MultiLoKo Turkish · EM Turkish
- rank
- 29
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 57
- name
- EM Turkish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 80
- name
- MultiLoKo Turkish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 66
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Turkish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Turkish · EM Turkish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 66/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-turkish/versions/1
- measurements
- name
- EM Turkish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Turkish · EM Turkish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.188
- standardDeviation
- 0
- confidenceInterval
- 0.048432292439317
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.048432292439317
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.188
- standardDeviation
- 0
- confidenceInterval
- 0.048432292439317
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.048432292439317
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 57
- sort_order
- DESCENDING
- source_name
- EM Turkish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.13956770756068299
- confidence upper
- 0.23643229243931702
Original result ↗79.85%
- source id
- liveBench
- leaderboard name
- LiveBench Instruction Following
- dimension
- general
- metric
- Average score
- rank
- 30
- sample size
- 50
- evaluated at
- 2025-02-01T00:41:38.803Z
- source model id
- o3-mini-2025-01-31-medium
Original result ↗0.773
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Yoruba · Yoruba
- dimension
- reasoning
- metric
- Global MMLU Lite Yoruba · Yoruba
- rank
- 30
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41743
- name
- Yoruba
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 165
- name
- Global MMLU Lite Yoruba
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 147
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Yoruba
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Yoruba · Yoruba
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 147/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-yoruba/versions/1
- measurements
- name
- Yoruba
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Yoruba · Yoruba
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7725
- standardDeviation
- 0
- confidenceInterval
- 0.0410826112430601
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0410826112430601
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7725
- standardDeviation
- 0
- confidenceInterval
- 0.0410826112430601
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0410826112430601
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41743
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-yoruba
- sort_order
- DESCENDING
- source_name
- Yoruba
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:24.645552400Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7314173887569398
- confidence upper
- 0.8135826112430601
Original result ↗22.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Italian · EM Italian
- dimension
- reasoning
- metric
- MultiLoKo Italian · EM Italian
- rank
- 30
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41
- name
- EM Italian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 64
- name
- MultiLoKo Italian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 50
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Italian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Italian · EM Italian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 50/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-italian/versions/1
- measurements
- name
- EM Italian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Italian · EM Italian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.228
- standardDeviation
- 0
- confidenceInterval
- 0.0520060848664191
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0520060848664191
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.228
- standardDeviation
- 0
- confidenceInterval
- 0.0520060848664191
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0520060848664191
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41
- sort_order
- DESCENDING
- source_name
- EM Italian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.17599391513358092
- confidence upper
- 0.2800060848664191
Original result ↗17.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Czech · EM Czech
- dimension
- reasoning
- metric
- MultiLoKo Czech · EM Czech
- rank
- 30
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 33
- name
- EM Czech
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 56
- name
- MultiLoKo Czech
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 42
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Czech
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Czech · EM Czech
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 42/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-czech/versions/1
- measurements
- name
- EM Czech
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Czech · EM Czech
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.176
- standardDeviation
- 0
- confidenceInterval
- 0.0472060896076912
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0472060896076912
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.176
- standardDeviation
- 0
- confidenceInterval
- 0.0472060896076912
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0472060896076912
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 33
- sort_order
- DESCENDING
- source_name
- EM Czech
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.12879391039230878
- confidence upper
- 0.2232060896076912
Original result ↗12.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Japanese · EM Japanese
- dimension
- reasoning
- metric
- MultiLoKo Japanese · EM Japanese
- rank
- 30
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 42
- name
- EM Japanese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 65
- name
- MultiLoKo Japanese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 51
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Japanese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Japanese · EM Japanese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 51/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-japanese/versions/1
- measurements
- name
- EM Japanese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Japanese · EM Japanese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.128
- standardDeviation
- 0
- confidenceInterval
- 0.0414134407037423
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0414134407037423
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.128
- standardDeviation
- 0
- confidenceInterval
- 0.0414134407037423
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0414134407037423
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 42
- sort_order
- DESCENDING
- source_name
- EM Japanese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.08658655929625769
- confidence upper
- 0.1694134407037423
Original result ↗0.808
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Burmese · Burmese
- dimension
- reasoning
- metric
- Global MMLU Lite Burmese · Burmese
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41736
- name
- Burmese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 175
- name
- Global MMLU Lite Burmese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 153
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Burmese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Burmese · Burmese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 153/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-burmese/versions/1
- measurements
- name
- Burmese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Burmese · Burmese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8075
- standardDeviation
- 0
- confidenceInterval
- 0.0386371183112584
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0386371183112584
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8075
- standardDeviation
- 0
- confidenceInterval
- 0.0386371183112584
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0386371183112584
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41736
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-burmese
- sort_order
- DESCENDING
- source_name
- Burmese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:19.694332700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7688628816887416
- confidence upper
- 0.8461371183112584
Original result ↗0.803
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite English · English
- dimension
- reasoning
- metric
- Global MMLU Lite English · English
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41749
- name
- English
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 153
- name
- Global MMLU Lite English
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 135
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite English
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite English · English
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 135/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-english/versions/1
- measurements
- name
- English
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite English · English
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8025
- standardDeviation
- 0
- confidenceInterval
- 0.0390143311477458
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0390143311477458
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8025
- standardDeviation
- 0
- confidenceInterval
- 0.0390143311477458
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0390143311477458
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41749
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-english
- sort_order
- DESCENDING
- source_name
- English
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:49.982103800Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7634856688522542
- confidence upper
- 0.8415143311477458
Original result ↗0.798
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Portuguese · Portuguese
- dimension
- reasoning
- metric
- Global MMLU Lite Portuguese · Portuguese
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41746
- name
- Portuguese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 162
- name
- Global MMLU Lite Portuguese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 144
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Portuguese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Portuguese · Portuguese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 144/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-portuguese/versions/1
- measurements
- name
- Portuguese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Portuguese · Portuguese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7975
- standardDeviation
- 0
- confidenceInterval
- 0.0393818356106108
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0393818356106108
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7975
- standardDeviation
- 0
- confidenceInterval
- 0.0393818356106108
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0393818356106108
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41746
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-portuguese
- sort_order
- DESCENDING
- source_name
- Portuguese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:30.662890500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7581181643893892
- confidence upper
- 0.8368818356106108
Original result ↗27.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Portuguese · EM Portuguese
- dimension
- reasoning
- metric
- MultiLoKo Portuguese · EM Portuguese
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 48
- name
- EM Portuguese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 71
- name
- MultiLoKo Portuguese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 57
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Portuguese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Portuguese · EM Portuguese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 57/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-portuguese/versions/1
- measurements
- name
- EM Portuguese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Portuguese · EM Portuguese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.276
- standardDeviation
- 0
- confidenceInterval
- 0.0554117556980965
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0554117556980965
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.276
- standardDeviation
- 0
- confidenceInterval
- 0.0554117556980965
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0554117556980965
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 48
- sort_order
- DESCENDING
- source_name
- EM Portuguese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.22058824430190352
- confidence upper
- 0.3314117556980965
Original result ↗20.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Romanian · EM Romanian
- dimension
- reasoning
- metric
- MultiLoKo Romanian · EM Romanian
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 49
- name
- EM Romanian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 72
- name
- MultiLoKo Romanian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 58
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Romanian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Romanian · EM Romanian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 58/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-romanian/versions/1
- measurements
- name
- EM Romanian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Romanian · EM Romanian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.204
- standardDeviation
- 0
- confidenceInterval
- 0.0499516345734389
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0499516345734389
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.204
- standardDeviation
- 0
- confidenceInterval
- 0.0499516345734389
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0499516345734389
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 49
- sort_order
- DESCENDING
- source_name
- EM Romanian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.15404836542656108
- confidence upper
- 0.2539516345734389
Original result ↗13.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Thai · EM Thai
- dimension
- reasoning
- metric
- MultiLoKo Thai · EM Thai
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 55
- name
- EM Thai
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 78
- name
- MultiLoKo Thai
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 64
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Thai
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Thai · EM Thai
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 64/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-thai/versions/1
- measurements
- name
- EM Thai
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Thai · EM Thai
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.136
- standardDeviation
- 0
- confidenceInterval
- 0.0424917298902665
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0424917298902665
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.136
- standardDeviation
- 0
- confidenceInterval
- 0.0424917298902665
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0424917298902665
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 55
- sort_order
- DESCENDING
- source_name
- EM Thai
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.0935082701097335
- confidence upper
- 0.17849172989026652
Original result ↗12.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Korean · EM Korean
- dimension
- reasoning
- metric
- MultiLoKo Korean · EM Korean
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 44
- name
- EM Korean
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 67
- name
- MultiLoKo Korean
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 53
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Korean
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Korean · EM Korean
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 53/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-korean/versions/1
- measurements
- name
- EM Korean
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Korean · EM Korean
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.128
- standardDeviation
- 0
- confidenceInterval
- 0.0414134407037423
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0414134407037423
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.128
- standardDeviation
- 0
- confidenceInterval
- 0.0414134407037423
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0414134407037423
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 44
- sort_order
- DESCENDING
- source_name
- EM Korean
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.08658655929625769
- confidence upper
- 0.1694134407037423
Original result ↗11.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Cantonese · EM Cantonese
- dimension
- reasoning
- metric
- MultiLoKo Cantonese · EM Cantonese
- rank
- 31
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 32
- name
- EM Cantonese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 55
- name
- MultiLoKo Cantonese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 41
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Cantonese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Cantonese · EM Cantonese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 41/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-cantonese/versions/1
- measurements
- name
- EM Cantonese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Cantonese · EM Cantonese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.116
- standardDeviation
- 0
- confidenceInterval
- 0.0396947630454828
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0396947630454828
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.116
- standardDeviation
- 0
- confidenceInterval
- 0.0396947630454828
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0396947630454828
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 32
- sort_order
- DESCENDING
- source_name
- EM Cantonese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.07630523695451721
- confidence upper
- 0.1556947630454828
Original result ↗0.813
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Chinese · Chinese
- dimension
- reasoning
- metric
- Global MMLU Lite Chinese · Chinese
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41735
- name
- Chinese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 166
- name
- Global MMLU Lite Chinese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 148
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Chinese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Chinese · Chinese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 148/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-chinese/versions/1
- measurements
- name
- Chinese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Chinese · Chinese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8125
- standardDeviation
- 0
- confidenceInterval
- 0.0382499098762049
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0382499098762049
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8125
- standardDeviation
- 0
- confidenceInterval
- 0.0382499098762049
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0382499098762049
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41735
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-chinese
- sort_order
- DESCENDING
- source_name
- Chinese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:22.953186100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7742500901237951
- confidence upper
- 0.8507499098762049
Original result ↗0.81
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Japanese · Japanese
- dimension
- reasoning
- metric
- Global MMLU Lite Japanese · Japanese
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41745
- name
- Japanese
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 160
- name
- Global MMLU Lite Japanese
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 142
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Japanese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Japanese · Japanese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 142/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-japanese/versions/1
- measurements
- name
- Japanese
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Japanese · Japanese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.81
- standardDeviation
- 0
- confidenceInterval
- 0.0384447822371523
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0384447822371523
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.81
- standardDeviation
- 0
- confidenceInterval
- 0.0384447822371523
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0384447822371523
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41745
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-japanese
- sort_order
- DESCENDING
- source_name
- Japanese
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:34.896663Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7715552177628477
- confidence upper
- 0.8484447822371524
Original result ↗0.808
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Korean · Korean
- dimension
- reasoning
- metric
- Global MMLU Lite Korean · Korean
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41748
- name
- Korean
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 161
- name
- Global MMLU Lite Korean
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 143
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Korean
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Korean · Korean
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 143/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-korean/versions/1
- measurements
- name
- Korean
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Korean · Korean
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8075
- standardDeviation
- 0
- confidenceInterval
- 0.0386371183112584
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0386371183112584
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8075
- standardDeviation
- 0
- confidenceInterval
- 0.0386371183112584
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0386371183112584
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41748
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-korean
- sort_order
- DESCENDING
- source_name
- Korean
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:33.197926100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7688628816887416
- confidence upper
- 0.8461371183112584
Original result ↗0.795
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Culturally Agnostic · CA Score
- dimension
- reasoning
- metric
- Global MMLU Lite Culturally Agnostic · CA Score
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41744
- name
- CA Score
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 209
- name
- Global MMLU Lite Culturally Agnostic
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 188
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Culturally Agnostic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Culturally Agnostic · CA Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 188/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-ca/versions/1
- measurements
- name
- CA Score
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Culturally Agnostic · CA Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.795
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-03T22:31:06.2966667Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.795
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41744
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-ca-score
- sort_order
- DESCENDING
- source_name
- CA Score
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-11-10T16:52:12.590864700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-03T22:31:06.296Z
- source model id
- o3 mini
Original result ↗0.77
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Bengali · Bengali
- dimension
- reasoning
- metric
- Global MMLU Lite Bengali · Bengali
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41751
- name
- Bengali
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 154
- name
- Global MMLU Lite Bengali
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 136
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Bengali
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Bengali · Bengali
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 136/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-bengali/versions/1
- measurements
- name
- Bengali
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Bengali · Bengali
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.77
- standardDeviation
- 0
- confidenceInterval
- 0.0412408279846843
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0412408279846843
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.77
- standardDeviation
- 0
- confidenceInterval
- 0.0412408279846843
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0412408279846843
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41751
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-bengali
- sort_order
- DESCENDING
- source_name
- Bengali
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:47.931282Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7287591720153157
- confidence upper
- 0.8112408279846843
Original result ↗Published prices
1–15 / 15
$0.55
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$1.10
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$4.40
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.28
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.55
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$2.20
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
- Billing conditions
- mode
- batch
$0.55
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$1.10
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$4.40
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
$0.55
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$1.10
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$4.40
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–3 / 3
- source model id
- azure-openai/o3-mini@2025-01-31
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 200000
- max output tokens
- 100000
- source model id
- openai/o3-mini
- hosting
- third-party
- context length
- 200000
- max output tokens
- 100000
- source model id
- openai/o3-mini
- hosting
- vercel-gateway
- context length
- 200000
- max output tokens
- 100000
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/o3-mini-2025-01-31
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–11 / 11
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2023-10-31
- normalized value
- 2023-10-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2023-10-01
- normalized value
- 2023-10-01
- precision
- day
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- September 2023
- normalized value
- 2023-09
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models