OpenAI: o3
o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Us…
openai/o3-2025-04-16
o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Use it to think through multi-step problems that involve analysis across text, code, and images.
Specifications & capabilities
- Input
- image, text, file
- Output
- text
- Context window · tokens
- 200000
- Capabilities
- fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗Evaluations
21–40 / 328
0.497
- source id
- llmStatsArchive
- leaderboard name
- BrowseComp · Normalized (reported)
- dimension
- general
- metric
- BrowseComp · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- BrowseComp
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-28T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2504.12516
- updated_at
- 2025-07-28T00:00:00.000000+00:00
- description
- BrowseComp is a benchmark comprising 1,266 questions that challenge AI agents to persistently navigate the internet in search of hard-to-find, entangled information. The benchmark measures agents' ability to exercise persistence in information gathering, demonstrate creativity in web navigation, and find concise, verifiable answers. Despite the difficulty of the questions, BrowseComp is simple and easy-to-use, as predicted answers are short and easily verifiable against reference answers.
- benchmark_id
- browsecomp
- multilingual
- false
- analysis_method
- accuracy (with python + browsing)
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- BrowseComp · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o3-2025-04-16/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/introducing-o3-and-o4-mini/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o3-2025-04-16
Original result ↗0.497
- source id
- llmStatsArchive
- leaderboard name
- BrowseComp · Raw
- dimension
- general
- metric
- BrowseComp · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- BrowseComp
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-28T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2504.12516
- updated_at
- 2025-07-28T00:00:00.000000+00:00
- description
- BrowseComp is a benchmark comprising 1,266 questions that challenge AI agents to persistently navigate the internet in search of hard-to-find, entangled information. The benchmark measures agents' ability to exercise persistence in information gathering, demonstrate creativity in web navigation, and find concise, verifiable answers. Despite the difficulty of the questions, BrowseComp is simple and easy-to-use, as predicted answers are short and easily verifiable against reference answers.
- benchmark_id
- browsecomp
- multilingual
- false
- analysis_method
- accuracy (with python + browsing)
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- BrowseComp · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o3-2025-04-16/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/introducing-o3-and-o4-mini/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o3-2025-04-16
Original result ↗46.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Korean · EM Korean
- dimension
- reasoning
- metric
- MultiLoKo Korean · EM Korean
- rank
- 2
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 44
- name
- EM Korean
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 67
- name
- MultiLoKo Korean
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 53
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Korean
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Korean · EM Korean
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 53/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-korean/versions/1
- measurements
- name
- EM Korean
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Korean · EM Korean
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.468
- standardDeviation
- 0
- confidenceInterval
- 0.0618524389605176
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618524389605176
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.468
- standardDeviation
- 0
- confidenceInterval
- 0.0618524389605176
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618524389605176
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 44
- sort_order
- DESCENDING
- source_name
- EM Korean
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.4061475610394824
- confidence upper
- 0.5298524389605176
Original result ↗35.98%
- source id
- sweBench
- leaderboard name
- SWE-bench Multimodal
- dimension
- coding
- metric
- Resolved tasks
- rank
- 2
- evaluated at
- 2025-11-17T00:00:00.000Z
Original result ↗0.065
- source id
- llmStatsArchive
- leaderboard name
- ARC-AGI v2 · Normalized (reported)
- dimension
- general
- metric
- ARC-AGI v2 · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- ARC-AGI v2
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- categories
- reasoning
- vision
- spatial_reasoning
- created_at
- 2025-07-19T19:56:13.916360+00:00
- paper_link
- https://arxiv.org/abs/2505.11831
- updated_at
- 2025-07-19T19:56:13.916360+00:00
- description
- ARC-AGI-2 is an upgraded benchmark for measuring abstract reasoning and problem-solving abilities in AI systems through visual grid transformation tasks. It evaluates fluid intelligence via input-output grid pairs (1x1 to 30x30) using colored cells (0-9), requiring models to identify underlying transformation rules from demonstration examples and apply them to test cases. Designed to be easy for humans but challenging for AI, focusing on core cognitive abilities like spatial reasoning, pattern recognition, and compositional generalization.
- benchmark_id
- arc-agi-v2
- multilingual
- false
- analysis_method
- accuracy
- is_self_reported
- false
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- ARC-AGI v2 · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o3-2025-04-16/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://x.com/xai/status/1943158495588815072
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o3-2025-04-16
Original result ↗0.065
- source id
- llmStatsArchive
- leaderboard name
- ARC-AGI v2 · Raw
- dimension
- general
- metric
- ARC-AGI v2 · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- ARC-AGI v2
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- categories
- reasoning
- vision
- spatial_reasoning
- created_at
- 2025-07-19T19:56:13.916360+00:00
- paper_link
- https://arxiv.org/abs/2505.11831
- updated_at
- 2025-07-19T19:56:13.916360+00:00
- description
- ARC-AGI-2 is an upgraded benchmark for measuring abstract reasoning and problem-solving abilities in AI systems through visual grid transformation tasks. It evaluates fluid intelligence via input-output grid pairs (1x1 to 30x30) using colored cells (0-9), requiring models to identify underlying transformation rules from demonstration examples and apply them to test cases. Designed to be easy for humans but challenging for AI, focusing on core cognitive abilities like spatial reasoning, pattern recognition, and compositional generalization.
- benchmark_id
- arc-agi-v2
- multilingual
- false
- analysis_method
- accuracy
- is_self_reported
- false
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- ARC-AGI v2 · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o3-2025-04-16/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://x.com/xai/status/1943158495588815072
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o3-2025-04-16
Original result ↗934
- source id
- kaggleBenchmarks
- leaderboard name
- Chess Openings · Score
- dimension
- general
- metric
- Chess Openings · Score
- rank
- 3
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1869
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- counts
- benchmark
- id
- 183
- name
- Chess Openings
- type
- GAME
- owner
- kaggle
- version
- 2
- version_id
- 436
- maintainer_tier
- Kaggle-published
- candidate
- candidate_class
- modelVersion
- collection
- kaggleGameArena
- record_kind
- aggregate
- game_arena
- true
- attribution
- Kaggle · kaggle · Chess Openings
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Chess Openings · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 436/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/kaggle/chess-text-openings/versions/2
- measurements
- name
- Score
- unit
- score
- scale
- 1
- metric
- Chess Openings · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 934
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- unevenConfidenceInterval
- plus
- 98
- minus
- 95
- hasUnevenConfidenceInterval
- true
- taskVersionId
- 0
- evaluationDate
- 2026-01-13T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 934
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- unevenConfidenceInterval
- plus
- 98
- minus
- 95
- hasUnevenConfidenceInterval
- true
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1869
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- counts
- task_version
- 1
- task_display_type
- COUNTS
- normalization_status
- normalized
- published_at
- 2026-02-02T15:46:28.981771500Z
- candidate_class
- modelVersion
- methodology_url
- https://www.kaggle.com/game-arena-faq
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- game_arena_rating
- Game-specific rating; compare within this game and version
- metric_population
- 11
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2026-01-13T00:00:00.000Z
- source model id
- o3
- confidence lower
- 839
- confidence upper
- 1032
Original result ↗69.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo English · EM English
- dimension
- reasoning
- metric
- MultiLoKo English · EM English
- rank
- 3
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 27
- name
- EM English
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 52
- name
- MultiLoKo English
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 38
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo English
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo English · EM English
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 38/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-english/versions/1
- measurements
- name
- EM English
- unit
- percent
- scale
- 1
- metric
- MultiLoKo English · EM English
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.696
- standardDeviation
- 0
- confidenceInterval
- 0.0570189871224575
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0570189871224575
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.696
- standardDeviation
- 0
- confidenceInterval
- 0.0570189871224575
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0570189871224575
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 27
- sort_order
- DESCENDING
- source_name
- EM English
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.6389810128775425
- confidence upper
- 0.7530189871224574
Original result ↗63.705%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · Selection Accuracy
- dimension
- reasoning
- metric
- AssetOpsBench · Selection Accuracy
- rank
- 3
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 1042
- name
- Selection Accuracy
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · Selection Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- Selection Accuracy
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · Selection Accuracy
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.6370530877573131
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-22T14:59:47.329153Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.6370530877573131
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 1042
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-selectionaccuracy-benchmark-task
- sort_order
- DESCENDING
- source_name
- Selection Accuracy
- display_type
- percentages
- task_version
- 1
- task_display_type
- RAW_VALUES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-22T14:59:47.329Z
- source model id
- o3
Original result ↗52%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Tagalog · EM Tagalog
- dimension
- reasoning
- metric
- MultiLoKo Tagalog · EM Tagalog
- rank
- 3
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 54
- name
- EM Tagalog
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 77
- name
- MultiLoKo Tagalog
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 63
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Tagalog
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Tagalog · EM Tagalog
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 63/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-tagalog/versions/1
- measurements
- name
- EM Tagalog
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Tagalog · EM Tagalog
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.52
- standardDeviation
- 0
- confidenceInterval
- 0.0619298997785481
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619298997785481
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.52
- standardDeviation
- 0
- confidenceInterval
- 0.0619298997785481
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619298997785481
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 54
- sort_order
- DESCENDING
- source_name
- EM Tagalog
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.4580701002214519
- confidence upper
- 0.5819298997785481
Original result ↗67.579%
- source id
- kaggleBenchmarks
- leaderboard name
- AssetOpsBench · Score
- dimension
- reasoning
- metric
- AssetOpsBench · Score
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 487619
- name
- Score
- version
- 5
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 243
- name
- AssetOpsBench
- type
- INDIVIDUAL
- owner
- ibm-research
- version
- 1
- categories
- evaluation
- general knowledge and reasoning
- version_id
- 222
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · ibm-research · AssetOpsBench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- AssetOpsBench · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 222/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- AssetOpsBench · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.675790526184227
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-11-22T04:59:59.816908Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.675790526184227
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 487619
- task_url
- /benchmarks/tasks/benchmarkler/failuresensoriq-main-benchmark-task
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 5
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-12-01T22:15:26.261663300Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 25
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-11-22T04:59:59.816Z
- source model id
- o3
Original result ↗54.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo French · EM French
- dimension
- reasoning
- metric
- MultiLoKo French · EM French
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 36
- name
- EM French
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 59
- name
- MultiLoKo French
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 45
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo French
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo French · EM French
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 45/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-french/versions/1
- measurements
- name
- EM French
- unit
- percent
- scale
- 1
- metric
- MultiLoKo French · EM French
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.548
- standardDeviation
- 0
- confidenceInterval
- 0.0616932406038344
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0616932406038344
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.548
- standardDeviation
- 0
- confidenceInterval
- 0.0616932406038344
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0616932406038344
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 36
- sort_order
- DESCENDING
- source_name
- EM French
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.48630675939616563
- confidence upper
- 0.6096932406038345
Original result ↗47.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Dutch · EM Dutch
- dimension
- reasoning
- metric
- MultiLoKo Dutch · EM Dutch
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 34
- name
- EM Dutch
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 57
- name
- MultiLoKo Dutch
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 43
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Dutch
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Dutch · EM Dutch
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 43/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-dutch/versions/1
- measurements
- name
- EM Dutch
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Dutch · EM Dutch
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.476
- standardDeviation
- 0
- confidenceInterval
- 0.0619080616686651
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619080616686651
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.476
- standardDeviation
- 0
- confidenceInterval
- 0.0619080616686651
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619080616686651
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 34
- sort_order
- DESCENDING
- source_name
- EM Dutch
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.4140919383313349
- confidence upper
- 0.5379080616686651
Original result ↗41.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Hebrew · EM Hebrew
- dimension
- reasoning
- metric
- MultiLoKo Hebrew · EM Hebrew
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 38
- name
- EM Hebrew
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 61
- name
- MultiLoKo Hebrew
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 47
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Hebrew
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Hebrew · EM Hebrew
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 47/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-hebrew/versions/1
- measurements
- name
- EM Hebrew
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Hebrew · EM Hebrew
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.412
- standardDeviation
- 0
- confidenceInterval
- 0.0610120135077044
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0610120135077044
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.412
- standardDeviation
- 0
- confidenceInterval
- 0.0610120135077044
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0610120135077044
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 38
- sort_order
- DESCENDING
- source_name
- EM Hebrew
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.35098798649229557
- confidence upper
- 0.4730120135077044
Original result ↗40.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Japanese · EM Japanese
- dimension
- reasoning
- metric
- MultiLoKo Japanese · EM Japanese
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 42
- name
- EM Japanese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 65
- name
- MultiLoKo Japanese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 51
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Japanese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Japanese · EM Japanese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 51/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-japanese/versions/1
- measurements
- name
- EM Japanese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Japanese · EM Japanese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.408
- standardDeviation
- 0
- confidenceInterval
- 0.0609212802792317
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0609212802792317
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.408
- standardDeviation
- 0
- confidenceInterval
- 0.0609212802792317
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0609212802792317
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 42
- sort_order
- DESCENDING
- source_name
- EM Japanese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.34707871972076826
- confidence upper
- 0.4689212802792317
Original result ↗38.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Arabic · EM Arabic
- dimension
- reasoning
- metric
- MultiLoKo Arabic · EM Arabic
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 30
- name
- EM Arabic
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 53
- name
- MultiLoKo Arabic
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 39
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Arabic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Arabic · EM Arabic
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 39/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-arabic/versions/1
- measurements
- name
- EM Arabic
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Arabic · EM Arabic
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.388
- standardDeviation
- 0
- confidenceInterval
- 0.0604045510115502
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0604045510115502
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.388
- standardDeviation
- 0
- confidenceInterval
- 0.0604045510115502
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0604045510115502
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 30
- sort_order
- DESCENDING
- source_name
- EM Arabic
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.32759544898844983
- confidence upper
- 0.4484045510115502
Original result ↗36.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Bengali · EM Bengali
- dimension
- reasoning
- metric
- MultiLoKo Bengali · EM Bengali
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 31
- name
- EM Bengali
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 54
- name
- MultiLoKo Bengali
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 40
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Bengali
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Bengali · EM Bengali
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 40/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-bengali/versions/1
- measurements
- name
- EM Bengali
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Bengali · EM Bengali
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.368
- standardDeviation
- 0
- confidenceInterval
- 0.0597806365533776
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0597806365533776
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.368
- standardDeviation
- 0
- confidenceInterval
- 0.0597806365533776
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0597806365533776
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 31
- sort_order
- DESCENDING
- source_name
- EM Bengali
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.3082193634466224
- confidence upper
- 0.4277806365533776
Original result ↗33.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Polish · EM Polish
- dimension
- reasoning
- metric
- MultiLoKo Polish · EM Polish
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 47
- name
- EM Polish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 70
- name
- MultiLoKo Polish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 56
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Polish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Polish · EM Polish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 56/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-polish/versions/1
- measurements
- name
- EM Polish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Polish · EM Polish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.332
- standardDeviation
- 0
- confidenceInterval
- 0.0583761379818247
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0583761379818247
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.332
- standardDeviation
- 0
- confidenceInterval
- 0.0583761379818247
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0583761379818247
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 47
- sort_order
- DESCENDING
- source_name
- EM Polish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.27362386201817535
- confidence upper
- 0.3903761379818247
Original result ↗31.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Malay · EM Malay
- dimension
- reasoning
- metric
- MultiLoKo Malay · EM Malay
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 45
- name
- EM Malay
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 68
- name
- MultiLoKo Malay
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 54
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Malay
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Malay · EM Malay
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 54/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-malay/versions/1
- measurements
- name
- EM Malay
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Malay · EM Malay
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.316
- standardDeviation
- 0
- confidenceInterval
- 0.0576301405634277
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0576301405634277
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.316
- standardDeviation
- 0
- confidenceInterval
- 0.0576301405634277
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0576301405634277
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 45
- sort_order
- DESCENDING
- source_name
- EM Malay
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.2583698594365723
- confidence upper
- 0.3736301405634277
Original result ↗30.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Thai · EM Thai
- dimension
- reasoning
- metric
- MultiLoKo Thai · EM Thai
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 55
- name
- EM Thai
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 78
- name
- MultiLoKo Thai
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 64
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Thai
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Thai · EM Thai
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 64/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-thai/versions/1
- measurements
- name
- EM Thai
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Thai · EM Thai
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.304
- standardDeviation
- 0
- confidenceInterval
- 0.0570189871224575
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0570189871224575
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.304
- standardDeviation
- 0
- confidenceInterval
- 0.0570189871224575
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0570189871224575
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 55
- sort_order
- DESCENDING
- source_name
- EM Thai
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o3-2025-04-16
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o3
- confidence lower
- 0.24698101287754248
- confidence upper
- 0.3610189871224575
Original result ↗Published prices
1–8 / 8
$0.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$2.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$8.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
$0.25
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$1.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$4.00
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- web_search
- Currency
- USD
- Billing conditions
- mode
- batch
Provider specifications & access
1–1 / 1
- source model id
- azure-openai/o3@2025-04-16
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 200000
- max output tokens
- 100000
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/o3-2025-04-16
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–9 / 9
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-06-30
- normalized value
- 2024-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- May 2024
- normalized value
- 2024-05
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models