OpenAI: o3 Mini (batch)
OpenAI o3-mini is a cost-efficient language model optimized for STEM reasoning tasks, particularly excelling in science, mathematics, and coding. This model supports the `reasoning_effort` parameter,…
openai/o3-mini-2025-01-31
OpenAI o3-mini is a cost-efficient language model optimized for STEM reasoning tasks, particularly excelling in science, mathematics, and coding. This model supports the `reasoning_effort` parameter, which can be set to...
Specifications & capabilities
- Input
- text, file
- Output
- text
- Context window · tokens
- 200000
- Published model ID
- o3-mini-2025-01-31
- Capabilities
- fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling
Evaluations
61–80 / 130
0.74
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite French · French
- dimension
- reasoning
- metric
- Global MMLU Lite French · French
- rank
- 35
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41752
- name
- French
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 156
- name
- Global MMLU Lite French
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 138
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite French
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite French · French
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 138/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-french/versions/1
- measurements
- name
- French
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite French · French
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.74
- standardDeviation
- 0
- confidenceInterval
- 0.0429853660302419
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0429853660302419
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.74
- standardDeviation
- 0
- confidenceInterval
- 0.0429853660302419
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0429853660302419
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41752
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-french
- sort_order
- DESCENDING
- source_name
- French
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:43.548947500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.697014633969758
- confidence upper
- 0.7829853660302419
Original result ↗20.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Russian · EM Russian
- dimension
- reasoning
- metric
- MultiLoKo Russian · EM Russian
- rank
- 35
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 50
- name
- EM Russian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 73
- name
- MultiLoKo Russian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 59
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Russian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Russian · EM Russian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 59/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-russian/versions/1
- measurements
- name
- EM Russian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Russian · EM Russian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.204
- standardDeviation
- 0
- confidenceInterval
- 0.0499516345734389
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0499516345734389
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.204
- standardDeviation
- 0
- confidenceInterval
- 0.0499516345734389
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0499516345734389
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 50
- sort_order
- DESCENDING
- source_name
- EM Russian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.15404836542656108
- confidence upper
- 0.2539516345734389
Original result ↗14%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Indonesian · EM Indonesian
- dimension
- reasoning
- metric
- MultiLoKo Indonesian · EM Indonesian
- rank
- 35
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40
- name
- EM Indonesian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 63
- name
- MultiLoKo Indonesian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 49
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Indonesian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Indonesian · EM Indonesian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 49/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-indonesian/versions/1
- measurements
- name
- EM Indonesian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Indonesian · EM Indonesian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.14
- standardDeviation
- 0
- confidenceInterval
- 0.0430121676743487
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0430121676743487
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.14
- standardDeviation
- 0
- confidenceInterval
- 0.0430121676743487
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0430121676743487
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40
- sort_order
- DESCENDING
- source_name
- EM Indonesian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.09698783232565131
- confidence upper
- 0.18301216767434872
Original result ↗0.743
- source id
- kaggleBenchmarks
- leaderboard name
- Global MMLU Lite Indonesian · Indonesian
- dimension
- reasoning
- metric
- Global MMLU Lite Indonesian · Indonesian
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41740
- name
- Indonesian
- version
- 2
- sort_order
- DESCENDING
- display_type
- rawValues
- benchmark
- id
- 158
- name
- Global MMLU Lite Indonesian
- type
- INDIVIDUAL
- owner
- cohere-labs
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 140
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · cohere-labs · Global MMLU Lite Indonesian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Global MMLU Lite Indonesian · Indonesian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 140/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-indonesian/versions/1
- measurements
- name
- Indonesian
- unit
- score
- scale
- 1
- metric
- Global MMLU Lite Indonesian · Indonesian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7425
- standardDeviation
- 0
- confidenceInterval
- 0.042850405989882
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.042850405989882
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7425
- standardDeviation
- 0
- confidenceInterval
- 0.042850405989882
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.042850405989882
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41740
- task_url
- /benchmarks/tasks/benchmarkler/global-mmlu-lite-indonesian
- sort_order
- DESCENDING
- source_name
- Indonesian
- display_type
- rawValues
- task_version
- 2
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-10-17T15:30:39.240452500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 41
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-30T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.6996495940101181
- confidence upper
- 0.785350405989882
Original result ↗12.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Bengali · EM Bengali
- dimension
- reasoning
- metric
- MultiLoKo Bengali · EM Bengali
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 31
- name
- EM Bengali
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 54
- name
- MultiLoKo Bengali
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 40
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Bengali
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Bengali · EM Bengali
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 40/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-bengali/versions/1
- measurements
- name
- EM Bengali
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Bengali · EM Bengali
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.128
- standardDeviation
- 0
- confidenceInterval
- 0.0414134407037423
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0414134407037423
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.128
- standardDeviation
- 0
- confidenceInterval
- 0.0414134407037423
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0414134407037423
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 31
- sort_order
- DESCENDING
- source_name
- EM Bengali
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.08658655929625769
- confidence upper
- 0.1694134407037423
Original result ↗8.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Malay · EM Malay
- dimension
- reasoning
- metric
- MultiLoKo Malay · EM Malay
- rank
- 36
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 45
- name
- EM Malay
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 68
- name
- MultiLoKo Malay
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 54
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Malay
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Malay · EM Malay
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 54/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-malay/versions/1
- measurements
- name
- EM Malay
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Malay · EM Malay
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.088
- standardDeviation
- 0
- confidenceInterval
- 0.0351169542593675
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0351169542593675
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.088
- standardDeviation
- 0
- confidenceInterval
- 0.0351169542593675
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0351169542593675
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 45
- sort_order
- DESCENDING
- source_name
- EM Malay
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.052883045740632496
- confidence upper
- 0.1231169542593675
Original result ↗51.906%
- source id
- liveBench
- leaderboard name
- LiveBench Overall
- dimension
- general
- metric
- Average score
- rank
- 37
- sample size
- 418
- evaluated at
- 2025-02-06T20:21:54.259Z
- source model id
- o3-mini-2025-01-31-low
Original result ↗10.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Polish · EM Polish
- dimension
- reasoning
- metric
- MultiLoKo Polish · EM Polish
- rank
- 38
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 47
- name
- EM Polish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 70
- name
- MultiLoKo Polish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 56
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Polish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Polish · EM Polish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 56/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-polish/versions/1
- measurements
- name
- EM Polish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Polish · EM Polish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.108
- standardDeviation
- 0
- confidenceInterval
- 0.0384744474658383
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0384744474658383
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.108
- standardDeviation
- 0
- confidenceInterval
- 0.0384744474658383
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0384744474658383
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 47
- sort_order
- DESCENDING
- source_name
- EM Polish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.0695255525341617
- confidence upper
- 0.1464744474658383
Original result ↗77.718%
- source id
- kaggleBenchmarks
- leaderboard name
- MMLU-Pro · mmlu_pro_task
- dimension
- reasoning
- metric
- MMLU-Pro · mmlu_pro_task
- rank
- 39
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40138
- name
- mmlu_pro_task
- version
- 12
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 44
- name
- MMLU-Pro
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- version_id
- 30
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MMLU-Pro
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MMLU-Pro · mmlu_pro_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 30/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mmlu-pro/versions/1
- measurements
- name
- mmlu_pro_task
- unit
- percent
- scale
- 1
- metric
- MMLU-Pro · mmlu_pro_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.7771775265957447
- standardDeviation
- 0
- confidenceInterval
- 0.0074356474751302
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0074356474751302
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-13T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.7771775265957447
- standardDeviation
- 0
- confidenceInterval
- 0.0074356474751302
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0074356474751302
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40138
- task_url
- /benchmarks/tasks/benchmarkler/mmlu-pro-task
- sort_order
- DESCENDING
- source_name
- mmlu_pro_task
- display_type
- percentages
- task_version
- 12
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 64
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-13T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.7697418791206145
- confidence upper
- 0.7846131740708748
Original result ↗8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Hebrew · EM Hebrew
- dimension
- reasoning
- metric
- MultiLoKo Hebrew · EM Hebrew
- rank
- 40
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 38
- name
- EM Hebrew
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 61
- name
- MultiLoKo Hebrew
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 47
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Hebrew
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Hebrew · EM Hebrew
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 47/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-hebrew/versions/1
- measurements
- name
- EM Hebrew
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Hebrew · EM Hebrew
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.08
- standardDeviation
- 0
- confidenceInterval
- 0.0336292354479305
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0336292354479305
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.08
- standardDeviation
- 0
- confidenceInterval
- 0.0336292354479305
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0336292354479305
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 38
- sort_order
- DESCENDING
- source_name
- EM Hebrew
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.0463707645520695
- confidence upper
- 0.1136292354479305
Original result ↗85.878%
- source id
- kaggleBenchmarks
- leaderboard name
- MMLU · mmlu_task
- dimension
- reasoning
- metric
- MMLU · mmlu_task
- rank
- 44
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40139
- name
- mmlu_task
- version
- 7
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 42
- name
- MMLU
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- general knowledge and reasoning
- version_id
- 28
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MMLU
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MMLU · mmlu_task
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 28/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mmlu/versions/1
- measurements
- name
- mmlu_task
- unit
- percent
- scale
- 1
- metric
- MMLU · mmlu_task
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.8587808004557755
- standardDeviation
- 0
- confidenceInterval
- 0.0057599888489916
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0057599888489916
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-02T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.8587808004557755
- standardDeviation
- 0
- confidenceInterval
- 0.0057599888489916
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0057599888489916
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40139
- task_url
- /benchmarks/tasks/benchmarkler/mmlu-task
- sort_order
- DESCENDING
- source_name
- mmlu_task
- display_type
- percentages
- task_version
- 7
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 61
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-02T00:00:00.000Z
- source model id
- o3 mini
- confidence lower
- 0.8530208116067839
- confidence upper
- 0.8645407893047672
Original result ↗44.98%
- source id
- liveBench
- leaderboard name
- LiveBench Language
- dimension
- intelligence
- metric
- Average score
- rank
- 44
- sample size
- 240
- evaluated at
- 2025-02-06T20:21:58.009Z
- source model id
- o3-mini-2025-01-31-medium
Original result ↗39.448%
- source id
- liveBench
- leaderboard name
- LiveBench Language
- dimension
- intelligence
- metric
- Average score
- rank
- 65
- sample size
- 240
- evaluated at
- 2025-02-06T20:21:54.259Z
- source model id
- o3-mini-2025-01-31-low
Original result ↗12.5
- source id
- artificialAnalysis
- dimension
- intelligence
- metric
- Intelligence Index
Original result ↗93.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM Chinese · Score
- dimension
- math
- metric
- MGSM Chinese · Score
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 131
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 124
- name
- MGSM Chinese
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 110
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM Chinese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM Chinese · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 110/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm-chinese/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- MGSM Chinese · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.936
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.936
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 131
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- o3 mini
Original result ↗93.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM: Multilingual Grade School Math Benchmark · MGSM Chinese
- dimension
- math
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM Chinese
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 131
- name
- MGSM Chinese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 113
- name
- MGSM: Multilingual Grade School Math Benchmark
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 99
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM: Multilingual Grade School Math Benchmark
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM: Multilingual Grade School Math Benchmark · MGSM Chinese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 99/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm/versions/1
- measurements
- name
- MGSM Chinese
- unit
- percent
- scale
- 1
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM Chinese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.936
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.936
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 131
- task_url
- /benchmarks/open-benchmarks/mgsm-chinese/versions/1
- sort_order
- DESCENDING
- source_name
- MGSM Chinese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- o3 mini
Original result ↗98.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM: Multilingual Grade School Math Benchmark · MGSM English
- dimension
- math
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM English
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 123
- name
- MGSM English
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 113
- name
- MGSM: Multilingual Grade School Math Benchmark
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 99
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM: Multilingual Grade School Math Benchmark
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM: Multilingual Grade School Math Benchmark · MGSM English
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 99/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm/versions/1
- measurements
- name
- MGSM English
- unit
- percent
- scale
- 1
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM English
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.984
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.984
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 123
- task_url
- /benchmarks/open-benchmarks/mgsm-english/versions/1
- sort_order
- DESCENDING
- source_name
- MGSM English
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- o3 mini
Original result ↗98.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM English · Score
- dimension
- math
- metric
- MGSM English · Score
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 123
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 116
- name
- MGSM English
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 102
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM English
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM English · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 102/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm-english/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- MGSM English · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.984
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.984
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 123
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- o3 mini
Original result ↗94.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM: Multilingual Grade School Math Benchmark · MGSM Thai
- dimension
- math
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM Thai
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 130
- name
- MGSM Thai
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 113
- name
- MGSM: Multilingual Grade School Math Benchmark
- type
- SUITE
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 99
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM: Multilingual Grade School Math Benchmark
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM: Multilingual Grade School Math Benchmark · MGSM Thai
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 99/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm/versions/1
- measurements
- name
- MGSM Thai
- unit
- percent
- scale
- 1
- metric
- MGSM: Multilingual Grade School Math Benchmark · MGSM Thai
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.948
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.948
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 130
- task_url
- /benchmarks/open-benchmarks/mgsm-thai/versions/1
- sort_order
- DESCENDING
- source_name
- MGSM Thai
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- o3 mini
Original result ↗94.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MGSM Thai · Score
- dimension
- math
- metric
- MGSM Thai · Score
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 130
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 123
- name
- MGSM Thai
- type
- INDIVIDUAL
- owner
- open-benchmarks
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- math
- version_id
- 109
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · open-benchmarks · MGSM Thai
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MGSM Thai · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 109/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/open-benchmarks/mgsm-thai/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- MGSM Thai · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.948
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-30T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.948
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 130
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-mini-2025-01-31
- metric_population
- 46
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-30T00:00:00.000Z
- source model id
- o3 mini
Original result ↗Published prices
1–15 / 15
$0.55
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$1.10
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$4.40
- Inference provider
- cloudflareWorkersAI
- Platform
- OpenAI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.28
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.55
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$2.20
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
- Billing conditions
- mode
- batch
$0.55
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$1.10
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$4.40
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- endpoint
- azure | openai/o3-mini
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
$0.55
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$1.10
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$4.40
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- request
- Currency
- USD
- Billing conditions
- charge
- web_search
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Web search
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–3 / 3
- source model id
- azure-openai/o3-mini@2025-01-31
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 200000
- max output tokens
- 100000
- source model id
- openai/o3-mini
- hosting
- third-party
- context length
- 200000
- max output tokens
- 100000
- source model id
- openai/o3-mini
- hosting
- vercel-gateway
- context length
- 200000
- max output tokens
- 100000
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/o3-mini-2025-01-31
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–11 / 11
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2023-10-31
- normalized value
- 2023-10-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2023-10-01
- normalized value
- 2023-10-01
- precision
- day
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- September 2023
- normalized value
- 2023-09
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models