OpenAI: o3
o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Us…
openai/o3-2025-04-16
o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Use it to think through multi-step problems that involve analysis across text, code, and images.
Specifications & capabilities
- Input
- image, text, file
- Output
- text
- Context window · tokens
- 200000
- Capabilities
- fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗Evaluations
41–60 / 328
925.053
- source id
- kaggleBenchmarks
- leaderboard name
- Chess Suite · Text Input
- dimension
- general
- metric
- Chess Suite · Text Input
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1446
- name
- Text Input
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 86
- name
- Chess Suite
- type
- SUITE
- owner
- kaggle
- version
- 2
- version_id
- 437
- maintainer_tier
- Kaggle-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · kaggle · Chess Suite
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Chess Suite · Text Input
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 437/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/kaggle/chess-suite/versions/2
- measurements
- name
- Text Input
- unit
- score
- scale
- 1
- metric
- Chess Suite · Text Input
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 925.0527736757293
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- unevenConfidenceInterval
- plus
- 39.13897846125769
- minus
- 52.372522708570386
- hasUnevenConfidenceInterval
- true
- taskVersionId
- 0
- evaluationDate
- 2026-11-11T20:03:32.6840746Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 925.0527736757293
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- unevenConfidenceInterval
- plus
- 39.13897846125769
- minus
- 52.372522708570386
- hasUnevenConfidenceInterval
- true
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1446
- task_url
- /benchmarks/kaggle/chess-text/versions/2
- sort_order
- DESCENDING
- source_name
- Text Input
- display_type
- counts
- task_version
- 1
- task_display_type
- COUNTS
- normalization_status
- normalized
- published_at
- 2026-02-02T15:46:09.865215700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 19
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2026-11-11T20:03:32.684Z
- source model id
- o3
- confidence lower
- 872.6802509671589
- confidence upper
- 964.191752136987
Original result ↗67.434%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · F-Score
- dimension
- reasoning
- metric
- AssetOpsBench · F-Score
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1040
- name
- F-Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · F-Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- F-Score
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · F-Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.6743400475415989
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-22T14:28:10.443283Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.6743400475415989
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1040
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-fscore-benchmark-task
- sort_order
- DESCENDING
- source_name
- F-Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-22T14:28:10.443Z
- source model id
- o3
Original result ↗58.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Portuguese · EM Portuguese
- dimension
- reasoning
- metric
- MultiLoKo Portuguese · EM Portuguese
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 48
- name
- EM Portuguese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 71
- name
- MultiLoKo Portuguese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 57
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Portuguese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Portuguese · EM Portuguese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 57/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-portuguese/versions/1
- measurements
- name
- EM Portuguese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Portuguese · EM Portuguese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.588
- standardDeviation
- 0
- confidenceInterval
- 0.0610120135077043
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0610120135077043
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.588
- standardDeviation
- 0
- confidenceInterval
- 0.0610120135077043
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0610120135077043
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 48
- sort_order
- DESCENDING
- source_name
- EM Portuguese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.5269879864922957
- confidence upper
- 0.6490120135077042
Original result ↗54%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Italian · EM Italian
- dimension
- reasoning
- metric
- MultiLoKo Italian · EM Italian
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41
- name
- EM Italian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 64
- name
- MultiLoKo Italian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 50
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Italian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Italian · EM Italian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 50/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-italian/versions/1
- measurements
- name
- EM Italian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Italian · EM Italian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.54
- standardDeviation
- 0
- confidenceInterval
- 0.0617808504655097
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0617808504655097
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.54
- standardDeviation
- 0
- confidenceInterval
- 0.0617808504655097
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0617808504655097
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41
- sort_order
- DESCENDING
- source_name
- EM Italian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.47821914953449035
- confidence upper
- 0.6017808504655098
Original result ↗52.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Spanish · EM Spanish
- dimension
- reasoning
- metric
- MultiLoKo Spanish · EM Spanish
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 52
- name
- EM Spanish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 75
- name
- MultiLoKo Spanish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 61
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Spanish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Spanish · EM Spanish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 61/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-spanish/versions/1
- measurements
- name
- EM Spanish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Spanish · EM Spanish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.528
- standardDeviation
- 0
- confidenceInterval
- 0.0618822430575397
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618822430575397
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.528
- standardDeviation
- 0
- confidenceInterval
- 0.0618822430575397
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618822430575397
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 52
- sort_order
- DESCENDING
- source_name
- EM Spanish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.4661177569424603
- confidence upper
- 0.5898822430575397
Original result ↗51.667%
- source id
- kaggleBenchmarks
- leaderboard name
- ITBench · Average Score
- dimension
- reasoning
- metric
- ITBench · Average Score
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1064
- name
- Average Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- rawPercentages
- benchmark
- id
- 246
- name
- ITBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 225
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · ITBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- ITBench · Average Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 225/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/itbench/versions/1
- measurements
- name
- Average Score
- unit
- percent
- scale
- 100
- metric
- ITBench · Average Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 51.66666666666667
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-12-01T22:27:57.070731Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 51.66666666666667
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1064
- sort_order
- DESCENDING
- source_name
- Average Score
- display_type
- rawPercentages
- task_version
- 1
- task_display_type
- RAW_PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-12-02T12:36:17.407125500Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 13
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-12-01T22:27:57.070Z
- source model id
- o3
Original result ↗46.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Romanian · EM Romanian
- dimension
- reasoning
- metric
- MultiLoKo Romanian · EM Romanian
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 49
- name
- EM Romanian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 72
- name
- MultiLoKo Romanian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 58
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Romanian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Romanian · EM Romanian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 58/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-romanian/versions/1
- measurements
- name
- EM Romanian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Romanian · EM Romanian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.468
- standardDeviation
- 0
- confidenceInterval
- 0.0618524389605176
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618524389605176
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.468
- standardDeviation
- 0
- confidenceInterval
- 0.0618524389605176
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618524389605176
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 49
- sort_order
- DESCENDING
- source_name
- EM Romanian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.4061475610394824
- confidence upper
- 0.5298524389605176
Original result ↗45.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo German · EM German
- dimension
- reasoning
- metric
- MultiLoKo German · EM German
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 37
- name
- EM German
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 60
- name
- MultiLoKo German
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 46
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo German
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo German · EM German
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 46/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-german/versions/1
- measurements
- name
- EM German
- unit
- percent
- scale
- 1
- metric
- MultiLoKo German · EM German
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.456
- standardDeviation
- 0
- confidenceInterval
- 0.0617390521759661
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0617390521759661
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.456
- standardDeviation
- 0
- confidenceInterval
- 0.0617390521759661
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0617390521759661
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 37
- sort_order
- DESCENDING
- source_name
- EM German
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.3942609478240339
- confidence upper
- 0.5177390521759662
Original result ↗45.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Hindi · EM Hindi
- dimension
- reasoning
- metric
- MultiLoKo Hindi · EM Hindi
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 39
- name
- EM Hindi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 62
- name
- MultiLoKo Hindi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 48
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Hindi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Hindi · EM Hindi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 48/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-hindi/versions/1
- measurements
- name
- EM Hindi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Hindi · EM Hindi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.456
- standardDeviation
- 0
- confidenceInterval
- 0.0617390521759661
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0617390521759661
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.456
- standardDeviation
- 0
- confidenceInterval
- 0.0617390521759661
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0617390521759661
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 39
- sort_order
- DESCENDING
- source_name
- EM Hindi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.3942609478240339
- confidence upper
- 0.5177390521759662
Original result ↗44.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Swedish · EM Swedish
- dimension
- reasoning
- metric
- MultiLoKo Swedish · EM Swedish
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 53
- name
- EM Swedish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 76
- name
- MultiLoKo Swedish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 62
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Swedish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Swedish · EM Swedish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 62/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-swedish/versions/1
- measurements
- name
- EM Swedish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Swedish · EM Swedish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.444
- standardDeviation
- 0
- confidenceInterval
- 0.0615895410053309
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0615895410053309
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.444
- standardDeviation
- 0
- confidenceInterval
- 0.0615895410053309
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0615895410053309
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 53
- sort_order
- DESCENDING
- source_name
- EM Swedish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.3824104589946691
- confidence upper
- 0.5055895410053309
Original result ↗40.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Indonesian · EM Indonesian
- dimension
- reasoning
- metric
- MultiLoKo Indonesian · EM Indonesian
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40
- name
- EM Indonesian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 63
- name
- MultiLoKo Indonesian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 49
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Indonesian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Indonesian · EM Indonesian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 49/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-indonesian/versions/1
- measurements
- name
- EM Indonesian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Indonesian · EM Indonesian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.404
- standardDeviation
- 0
- confidenceInterval
- 0.0608263699617859
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0608263699617859
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.404
- standardDeviation
- 0
- confidenceInterval
- 0.0608263699617859
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0608263699617859
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40
- sort_order
- DESCENDING
- source_name
- EM Indonesian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.3431736300382141
- confidence upper
- 0.46482636996178595
Original result ↗36.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Marathi · EM Marathi
- dimension
- reasoning
- metric
- MultiLoKo Marathi · EM Marathi
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 46
- name
- EM Marathi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 69
- name
- MultiLoKo Marathi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 55
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Marathi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Marathi · EM Marathi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 55/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-marathi/versions/1
- measurements
- name
- EM Marathi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Marathi · EM Marathi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.364
- standardDeviation
- 0
- confidenceInterval
- 0.0596427056001309
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0596427056001309
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.364
- standardDeviation
- 0
- confidenceInterval
- 0.0596427056001309
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0596427056001309
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 46
- sort_order
- DESCENDING
- source_name
- EM Marathi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.3043572943998691
- confidence upper
- 0.4236427056001309
Original result ↗67.248
- source id
- kaggleBenchmarks
- leaderboard name
- IndicGenBench · XorQA-In
- dimension
- general
- metric
- IndicGenBench · XorQA-In
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 817
- name
- XorQA-In
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 212
- name
- IndicGenBench
- type
- SUITE
- owner
- deepmind
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- version_id
- 191
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · IndicGenBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- IndicGenBench · XorQA-In
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 191/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
- measurements
- name
- XorQA-In
- unit
- score
- scale
- 1
- metric
- IndicGenBench · XorQA-In
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 67.24829913136512
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-28T22:12:41.475374Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 67.24829913136512
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 817
- task_url
- /benchmarks/tasks/aminmohamedmohami/xorqa-in-task
- sort_order
- DESCENDING
- source_name
- XorQA-In
- display_type
- counts
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-04T15:09:28.942941700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 13
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-28T22:12:41.475Z
- source model id
- o3
Original result ↗21.989
- source id
- kaggleBenchmarks
- leaderboard name
- IndicGenBench · CrossSum-In
- dimension
- general
- metric
- IndicGenBench · CrossSum-In
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 815
- name
- CrossSum-In
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 212
- name
- IndicGenBench
- type
- SUITE
- owner
- deepmind
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- version_id
- 191
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · IndicGenBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- IndicGenBench · CrossSum-In
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 191/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
- measurements
- name
- CrossSum-In
- unit
- score
- scale
- 1
- metric
- IndicGenBench · CrossSum-In
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 21.98939111350083
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-28T22:12:54.970365Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 21.98939111350083
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 815
- task_url
- /benchmarks/tasks/aminmohamedmohami/crosssum-in-task
- sort_order
- DESCENDING
- source_name
- CrossSum-In
- display_type
- counts
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-04T15:09:28.942941700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 13
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-28T22:12:54.970Z
- source model id
- o3
Original result ↗69.391%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · Elimination Accuracy
- dimension
- reasoning
- metric
- AssetOpsBench · Elimination Accuracy
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1041
- name
- Elimination Accuracy
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · Elimination Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- Elimination Accuracy
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · Elimination Accuracy
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.6939127105666156
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-22T14:59:07.849171Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.6939127105666156
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1041
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-eliminationaccuracy-benchmark-task
- sort_order
- DESCENDING
- source_name
- Elimination Accuracy
- display_type
- percentages
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-22T14:59:07.849Z
- source model id
- o3
Original result ↗65.167%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · Pert. Simple Accuracy
- dimension
- reasoning
- metric
- AssetOpsBench · Pert. Simple Accuracy
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1043
- name
- Pert. Simple Accuracy
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · Pert. Simple Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- Pert. Simple Accuracy
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · Pert. Simple Accuracy
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.651668541432321
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-23T14:35:27.008723Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.651668541432321
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1043
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-perturbationsimple-benchmark-task
- sort_order
- DESCENDING
- source_name
- Pert. Simple Accuracy
- display_type
- percentages
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-23T14:35:27.008Z
- source model id
- o3
Original result ↗48.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Czech · EM Czech
- dimension
- reasoning
- metric
- MultiLoKo Czech · EM Czech
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 33
- name
- EM Czech
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 56
- name
- MultiLoKo Czech
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 42
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Czech
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Czech · EM Czech
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 42/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-czech/versions/1
- measurements
- name
- EM Czech
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Czech · EM Czech
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.488
- standardDeviation
- 0
- confidenceInterval
- 0.0619616505623712
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619616505623712
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.488
- standardDeviation
- 0
- confidenceInterval
- 0.0619616505623712
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619616505623712
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 33
- sort_order
- DESCENDING
- source_name
- EM Czech
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.4260383494376288
- confidence upper
- 0.5499616505623712
Original result ↗46.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Turkish · EM Turkish
- dimension
- reasoning
- metric
- MultiLoKo Turkish · EM Turkish
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 57
- name
- EM Turkish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 80
- name
- MultiLoKo Turkish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 66
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Turkish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Turkish · EM Turkish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 66/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-turkish/versions/1
- measurements
- name
- EM Turkish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Turkish · EM Turkish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.464
- standardDeviation
- 0
- confidenceInterval
- 0.0618186436131337
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618186436131337
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.464
- standardDeviation
- 0
- confidenceInterval
- 0.0618186436131337
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618186436131337
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 57
- sort_order
- DESCENDING
- source_name
- EM Turkish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.4021813563868663
- confidence upper
- 0.5258186436131337
Original result ↗355.756
- source id
- kaggleBenchmarks
- leaderboard name
- Word Association · Score
- dimension
- general
- metric
- Word Association · Score
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 27256
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 4537
- name
- Word Association
- type
- GAME
- owner
- kaggle
- version
- 1
- version_id
- 4560
- maintainer_tier
- Kaggle-published
- candidate
- candidate_class
- modelVersion
- collection
- kaggleGameArena
- record_kind
- aggregate
- game_arena
- true
- attribution
- Kaggle · kaggle · Word Association
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Word Association · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 4560/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/kaggle/word-association/versions/1
- measurements
- name
- Score
- unit
- score
- scale
- 1
- metric
- Word Association · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 355.75620420762516
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- unevenConfidenceInterval
- plus
- 28.902142596400495
- minus
- 34.22480584225127
- hasUnevenConfidenceInterval
- true
- taskVersionId
- 0
- evaluationDate
- 2026-11-11T19:58:55.7420417Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 355.75620420762516
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- unevenConfidenceInterval
- plus
- 28.902142596400495
- minus
- 34.22480584225127
- hasUnevenConfidenceInterval
- true
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 27256
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- counts
- task_version
- 1
- task_display_type
- COUNTS
- normalization_status
- normalized
- published_at
- 2026-05-26T19:37:29.256086900Z
- candidate_class
- modelVersion
- methodology_url
- https://www.kaggle.com/game-arena-faq
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- game_arena_rating
- Game-specific rating; compare within this game and version
- metric_population
- 40
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2026-11-11T19:58:55.742Z
- source model id
- o3
- confidence lower
- 321.5313983653739
- confidence upper
- 384.65834680402565
Original result ↗48.231
- source id
- kaggleBenchmarks
- leaderboard name
- IndicGenBench · Flores-In
- dimension
- general
- metric
- IndicGenBench · Flores-In
- rank
- 7
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 816
- name
- Flores-In
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 212
- name
- IndicGenBench
- type
- SUITE
- owner
- deepmind
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- version_id
- 191
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · deepmind · IndicGenBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- IndicGenBench · Flores-In
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 191/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/deepmind/indic-gen-bench/versions/1
- measurements
- name
- Flores-In
- unit
- score
- scale
- 1
- metric
- IndicGenBench · Flores-In
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 48.231036546273856
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-10-28T22:12:17.510585Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 48.231036546273856
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 816
- task_url
- /benchmarks/tasks/aminmohamedmohami/flores-in-task
- sort_order
- DESCENDING
- source_name
- Flores-In
- display_type
- counts
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-04T15:09:28.942941700Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 13
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-10-28T22:12:17.510Z
- source model id
- o3
Original result ↗Published prices
1–8 / 8
$0.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$2.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$8.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
$0.25
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$1.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$4.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
- Billing conditions
- mode
- batch
Provider specifications & access
1–1 / 1
- source model id
- azure-openai/o3@2025-04-16
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 200000
- max output tokens
- 100000
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/o3-2025-04-16
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–9 / 9
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- May 2024
- normalized value
- 2024-05
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models