OpenAI: o3 Mini (batch)
OpenAI o3-mini is a cost-efficient language model optimized for STEM reasoning tasks, particularly excelling in science, mathematics, and coding. This model supports the `reasoning_effort` parameter,…
openai/o3-mini-2025-01-31
OpenAI o3-mini is a cost-efficient language model optimized for STEM reasoning tasks, particularly excelling in science, mathematics, and coding. This model supports the `reasoning_effort` parameter, which can be set to...
Specifications & capabilities
- Input
- text, file
- Output
- text
- Context window · tokens
- 200000
- Published model ID
- o3-mini-2025-01-31
- Capabilities
- fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling
Evaluations
41–60 / 130
0.765
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Culturally Sensitive · CS Score
- dimension
- reasoning
- metric
- Global MMLU Lite Culturally Sensitive · CS Score
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41747
- name
- CS Score
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 208
- name
- Global MMLU Lite Culturally Sensitive
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 187
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Culturally Sensitive
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Culturally Sensitive · CS Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 187/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-cs/versions/1
- measurements
- name
- CS Score
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Culturally Sensitive · CS Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7650000000000001
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-03T22:31:06.2966667Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7650000000000001
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41747
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-cs-score
- sort_order
- DESCENDING
- source_name
- CS Score
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-11-10T16:52:15.372518Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-03T22:31:06.296Z
- source model id
- o3 mini
Original result ↗17.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Marathi · EM Marathi
- dimension
- reasoning
- metric
- MultiLoKo Marathi · EM Marathi
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 46
- name
- EM Marathi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 69
- name
- MultiLoKo Marathi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 55
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Marathi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Marathi · EM Marathi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 55/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-marathi/versions/1
- measurements
- name
- EM Marathi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Marathi · EM Marathi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.176
- standardDeviation
- 0
- confidenceInterval
- 0.0472060896076912
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0472060896076912
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.176
- standardDeviation
- 0
- confidenceInterval
- 0.0472060896076912
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0472060896076912
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 46
- sort_order
- DESCENDING
- source_name
- EM Marathi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.12879391039230878
- confidence upper
- 0.2232060896076912
Original result ↗17.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Swedish · EM Swedish
- dimension
- reasoning
- metric
- MultiLoKo Swedish · EM Swedish
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 53
- name
- EM Swedish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 76
- name
- MultiLoKo Swedish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 62
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Swedish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Swedish · EM Swedish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 62/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-swedish/versions/1
- measurements
- name
- EM Swedish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Swedish · EM Swedish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.176
- standardDeviation
- 0
- confidenceInterval
- 0.0472060896076912
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0472060896076912
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.176
- standardDeviation
- 0
- confidenceInterval
- 0.0472060896076912
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0472060896076912
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 53
- sort_order
- DESCENDING
- source_name
- EM Swedish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.12879391039230878
- confidence upper
- 0.2232060896076912
Original result ↗12.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Arabic · EM Arabic
- dimension
- reasoning
- metric
- MultiLoKo Arabic · EM Arabic
- rank
- 32
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 30
- name
- EM Arabic
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 53
- name
- MultiLoKo Arabic
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 39
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Arabic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Arabic · EM Arabic
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 39/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-arabic/versions/1
- measurements
- name
- EM Arabic
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Arabic · EM Arabic
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.124
- standardDeviation
- 0
- confidenceInterval
- 0.0408546018430765
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0408546018430765
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.124
- standardDeviation
- 0
- confidenceInterval
- 0.0408546018430765
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0408546018430765
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 30
- sort_order
- DESCENDING
- source_name
- EM Arabic
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.0831453981569235
- confidence upper
- 0.1648546018430765
Original result ↗0.8
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Italian · Italian
- dimension
- reasoning
- metric
- Global MMLU Lite Italian · Italian
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41739
- name
- Italian
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 159
- name
- Global MMLU Lite Italian
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 141
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Italian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Italian · Italian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 141/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-italian/versions/1
- measurements
- name
- Italian
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Italian · Italian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8
- standardDeviation
- 0
- confidenceInterval
- 0.039199279690801
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.039199279690801
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8
- standardDeviation
- 0
- confidenceInterval
- 0.039199279690801
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.039199279690801
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41739
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-italian
- sort_order
- DESCENDING
- source_name
- Italian
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:36.963753600Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7608007203091991
- confidence upper
- 0.839199279690801
Original result ↗77.5%
- source id
- kaggleBenchmarks
- leaderboard name
- AIME 2025 · aime_task
- dimension
- math
- metric
- AIME 2025 · aime_task
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40087
- name
- aime_task
- version
- 6
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 125
- name
- AIME 2025
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- math
- version_id
- 111
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · AIME 2025
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AIME 2025 · aime_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 111/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/aime-2025/versions/1
- measurements
- name
- aime_task
- unit
- percent
- scale
- 1
- metric
- AIME 2025 · aime_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.775
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-07-07T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.775
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40087
- task_url
- /benchmarks/tasks/benchmarkler/aime-task
- sort_order
- DESCENDING
- source_name
- aime_task
- display_type
- percentages
- task_version
- 6
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 68
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-07-07T00:00:00.000Z
- source model id
- o3 mini
Original result ↗0.775
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Spanish · Spanish
- dimension
- reasoning
- metric
- Global MMLU Lite Spanish · Spanish
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41741
- name
- Spanish
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 163
- name
- Global MMLU Lite Spanish
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 145
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Spanish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Spanish · Spanish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 145/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-spanish/versions/1
- measurements
- name
- Spanish
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Spanish · Spanish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.775
- standardDeviation
- 0
- confidenceInterval
- 0.0409223160958216
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0409223160958216
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.775
- standardDeviation
- 0
- confidenceInterval
- 0.0409223160958216
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0409223160958216
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41741
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-spanish
- sort_order
- DESCENDING
- source_name
- Spanish
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:28.873808200Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7340776839041784
- confidence upper
- 0.8159223160958217
Original result ↗0.773
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Arabic · Arabic
- dimension
- reasoning
- metric
- Global MMLU Lite Arabic · Arabic
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41737
- name
- Arabic
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 148
- name
- Global MMLU Lite Arabic
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 130
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Arabic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Arabic · Arabic
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 130/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-arabic/versions/1
- measurements
- name
- Arabic
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Arabic · Arabic
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7725
- standardDeviation
- 0
- confidenceInterval
- 0.0410826112430601
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0410826112430601
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7725
- standardDeviation
- 0
- confidenceInterval
- 0.0410826112430601
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0410826112430601
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41737
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-arabic
- sort_order
- DESCENDING
- source_name
- Arabic
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:54.963186300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7314173887569398
- confidence upper
- 0.8135826112430601
Original result ↗0.765
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Swahili · Swahili
- dimension
- reasoning
- metric
- Global MMLU Lite Swahili · Swahili
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41750
- name
- Swahili
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 164
- name
- Global MMLU Lite Swahili
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 146
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Swahili
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Swahili · Swahili
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 146/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-swahili/versions/1
- measurements
- name
- Swahili
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Swahili · Swahili
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.765
- standardDeviation
- 0
- confidenceInterval
- 0.0415511209081742
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0415511209081742
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.765
- standardDeviation
- 0
- confidenceInterval
- 0.0415511209081742
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0415511209081742
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41750
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-swahili
- sort_order
- DESCENDING
- source_name
- Swahili
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:27.253350900Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7234488790918259
- confidence upper
- 0.8065511209081742
Original result ↗0.753
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Hindi · Hindi
- dimension
- reasoning
- metric
- Global MMLU Lite Hindi · Hindi
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41742
- name
- Hindi
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 157
- name
- Global MMLU Lite Hindi
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 139
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Hindi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Hindi · Hindi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 139/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-hindi/versions/1
- measurements
- name
- Hindi
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Hindi · Hindi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7525
- standardDeviation
- 0
- confidenceInterval
- 0.0422920706585954
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0422920706585954
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7525
- standardDeviation
- 0
- confidenceInterval
- 0.0422920706585954
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0422920706585954
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41742
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-hindi
- sort_order
- DESCENDING
- source_name
- Hindi
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:41.662768400Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7102079293414045
- confidence upper
- 0.7947920706585954
Original result ↗23.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Hindi · EM Hindi
- dimension
- reasoning
- metric
- MultiLoKo Hindi · EM Hindi
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 39
- name
- EM Hindi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 62
- name
- MultiLoKo Hindi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 48
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Hindi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Hindi · EM Hindi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 48/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-hindi/versions/1
- measurements
- name
- EM Hindi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Hindi · EM Hindi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.232
- standardDeviation
- 0
- confidenceInterval
- 0.0523242111010188
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0523242111010188
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.232
- standardDeviation
- 0
- confidenceInterval
- 0.0523242111010188
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0523242111010188
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 39
- sort_order
- DESCENDING
- source_name
- EM Hindi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.17967578889898123
- confidence upper
- 0.2843242111010188
Original result ↗20.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Dutch · EM Dutch
- dimension
- reasoning
- metric
- MultiLoKo Dutch · EM Dutch
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 34
- name
- EM Dutch
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 57
- name
- MultiLoKo Dutch
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 43
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Dutch
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Dutch · EM Dutch
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 43/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-dutch/versions/1
- measurements
- name
- EM Dutch
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Dutch · EM Dutch
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.204
- standardDeviation
- 0
- confidenceInterval
- 0.0499516345734389
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0499516345734389
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.204
- standardDeviation
- 0
- confidenceInterval
- 0.0499516345734389
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0499516345734389
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 34
- sort_order
- DESCENDING
- source_name
- EM Dutch
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.15404836542656108
- confidence upper
- 0.2539516345734389
Original result ↗20.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo German · EM German
- dimension
- reasoning
- metric
- MultiLoKo German · EM German
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 37
- name
- EM German
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 60
- name
- MultiLoKo German
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 46
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo German
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo German · EM German
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 46/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-german/versions/1
- measurements
- name
- EM German
- unit
- percent
- scale
- 1
- metric
- MultiLoKo German · EM German
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.204
- standardDeviation
- 0
- confidenceInterval
- 0.0499516345734389
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0499516345734389
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.204
- standardDeviation
- 0
- confidenceInterval
- 0.0499516345734389
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0499516345734389
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 37
- sort_order
- DESCENDING
- source_name
- EM German
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.15404836542656108
- confidence upper
- 0.2539516345734389
Original result ↗19.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- dimension
- reasoning
- metric
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 56
- name
- EM Traditional Mandarin
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 79
- name
- MultiLoKo Traditional Mandarin
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 65
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Traditional Mandarin
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 65/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-traditional-mandarin/versions/1
- measurements
- name
- EM Traditional Mandarin
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Traditional Mandarin · EM Traditional Mandarin
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.192
- standardDeviation
- 0
- confidenceInterval
- 0.048824115173025
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.048824115173025
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.192
- standardDeviation
- 0
- confidenceInterval
- 0.048824115173025
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.048824115173025
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 56
- sort_order
- DESCENDING
- source_name
- EM Traditional Mandarin
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.14317588482697502
- confidence upper
- 0.240824115173025
Original result ↗17.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Simplified Mandarin · EM Simplified Mandarin
- dimension
- reasoning
- metric
- MultiLoKo Simplified Mandarin · EM Simplified Mandarin
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 51
- name
- EM Simplified Mandarin
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 74
- name
- MultiLoKo Simplified Mandarin
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 60
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Simplified Mandarin
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Simplified Mandarin · EM Simplified Mandarin
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 60/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-simplified-mandarin/versions/1
- measurements
- name
- EM Simplified Mandarin
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Simplified Mandarin · EM Simplified Mandarin
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.176
- standardDeviation
- 0
- confidenceInterval
- 0.0472060896076912
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0472060896076912
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.176
- standardDeviation
- 0
- confidenceInterval
- 0.0472060896076912
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0472060896076912
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 51
- sort_order
- DESCENDING
- source_name
- EM Simplified Mandarin
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.12879391039230878
- confidence upper
- 0.2232060896076912
Original result ↗13.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Vietnamese · EM Vietnamese
- dimension
- reasoning
- metric
- MultiLoKo Vietnamese · EM Vietnamese
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 59
- name
- EM Vietnamese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 82
- name
- MultiLoKo Vietnamese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 68
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Vietnamese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Vietnamese · EM Vietnamese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 68/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-vietnamese/versions/1
- measurements
- name
- EM Vietnamese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Vietnamese · EM Vietnamese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.136
- standardDeviation
- 0
- confidenceInterval
- 0.0424917298902665
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0424917298902665
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.136
- standardDeviation
- 0
- confidenceInterval
- 0.0424917298902665
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0424917298902665
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 59
- sort_order
- DESCENDING
- source_name
- EM Vietnamese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.0935082701097335
- confidence upper
- 0.17849172989026652
Original result ↗5.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Khmer · EM Khmer
- dimension
- reasoning
- metric
- MultiLoKo Khmer · EM Khmer
- rank
- 33
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 43
- name
- EM Khmer
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 66
- name
- MultiLoKo Khmer
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 52
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Khmer
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Khmer · EM Khmer
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 52/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-khmer/versions/1
- measurements
- name
- EM Khmer
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Khmer · EM Khmer
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.052
- standardDeviation
- 0
- confidenceInterval
- 0.0275222494738302
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0275222494738302
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.052
- standardDeviation
- 0
- confidenceInterval
- 0.0275222494738302
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0275222494738302
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 43
- sort_order
- DESCENDING
- source_name
- EM Khmer
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.024477750526169798
- confidence upper
- 0.0795222494738302
Original result ↗78.583%
- source id
- liveBench
- leaderboard name
- LiveBench Instruction Following
- dimension
- general
- metric
- Average score
- rank
- 34
- sample size
- 50
- evaluated at
- 2025-01-31T20:32:20.804Z
- source model id
- o3-mini-2025-01-31-low
Original result ↗22.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo French · EM French
- dimension
- reasoning
- metric
- MultiLoKo French · EM French
- rank
- 34
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 36
- name
- EM French
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 59
- name
- MultiLoKo French
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 45
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo French
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo French · EM French
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 45/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-french/versions/1
- measurements
- name
- EM French
- unit
- percent
- scale
- 1
- metric
- MultiLoKo French · EM French
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.228
- standardDeviation
- 0
- confidenceInterval
- 0.0520060848664191
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0520060848664191
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.228
- standardDeviation
- 0
- confidenceInterval
- 0.0520060848664191
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0520060848664191
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 36
- sort_order
- DESCENDING
- source_name
- EM French
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.17599391513358092
- confidence upper
- 0.2800060848664191
Original result ↗0.753
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite German · German
- dimension
- reasoning
- metric
- Global MMLU Lite German · German
- rank
- 35
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41738
- name
- German
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 155
- name
- Global MMLU Lite German
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 137
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite German
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite German · German
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 137/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-german/versions/1
- measurements
- name
- German
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite German · German
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7525
- standardDeviation
- 0
- confidenceInterval
- 0.0422920706585954
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0422920706585954
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7525
- standardDeviation
- 0
- confidenceInterval
- 0.0422920706585954
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0422920706585954
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41738
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-german
- sort_order
- DESCENDING
- source_name
- German
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:45.767915600Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7102079293414045
- confidence upper
- 0.7947920706585954
Original result ↗Published prices
1–15 / 15
$0.55
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$1.10
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$4.40
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.28
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.55
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$2.20
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
- Billing conditions
- mode
- batch
$0.55
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$1.10
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$4.40
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
$0.55
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$1.10
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$4.40
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–3 / 3
- source model id
- azure-openai/o3-mini@2025-01-31
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 200000
- max output tokens
- 100000
- source model id
- openai/o3-mini
- hosting
- third-party
- context length
- 200000
- max output tokens
- 100000
- source model id
- openai/o3-mini
- hosting
- vercel-gateway
- context length
- 200000
- max output tokens
- 100000
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/o3-mini-2025-01-31
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–11 / 11
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2023-10-31
- normalized value
- 2023-10-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2023-10-01
- normalized value
- 2023-10-01
- precision
- day
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- September 2023
- normalized value
- 2023-09
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models