OpenAI: o1
The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason...
openai/o1-2024-12-17
The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason...
Specifications & capabilities
- Input
- text, image, file
- Output
- text
- Context window · tokens
- 200000
- Knowledge cutoff
- 2023-09
- Capabilities
- multimodal, reasoning, toolCalling, visionInput
Evaluations
41–60 / 200
40%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Korean · EM Korean
- dimension
- reasoning
- metric
- MultiLoKo Korean · EM Korean
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 44
- name
- EM Korean
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 67
- name
- MultiLoKo Korean
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 53
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Korean
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Korean · EM Korean
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 53/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-korean/versions/1
- measurements
- name
- EM Korean
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Korean · EM Korean
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.4
- standardDeviation
- 0
- confidenceInterval
- 0.0607272629703196
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0607272629703196
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.4
- standardDeviation
- 0
- confidenceInterval
- 0.0607272629703196
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0607272629703196
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 44
- sort_order
- DESCENDING
- source_name
- EM Korean
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.3392727370296804
- confidence upper
- 0.46072726297031963
Original result ↗36.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Arabic · EM Arabic
- dimension
- reasoning
- metric
- MultiLoKo Arabic · EM Arabic
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 30
- name
- EM Arabic
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 53
- name
- MultiLoKo Arabic
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 39
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Arabic
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Arabic · EM Arabic
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 39/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-arabic/versions/1
- measurements
- name
- EM Arabic
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Arabic · EM Arabic
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.368
- standardDeviation
- 0
- confidenceInterval
- 0.0597806365533776
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0597806365533776
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.368
- standardDeviation
- 0
- confidenceInterval
- 0.0597806365533776
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0597806365533776
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 30
- sort_order
- DESCENDING
- source_name
- EM Arabic
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.3082193634466224
- confidence upper
- 0.4277806365533776
Original result ↗32%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Polish · EM Polish
- dimension
- reasoning
- metric
- MultiLoKo Polish · EM Polish
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 47
- name
- EM Polish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 70
- name
- MultiLoKo Polish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 56
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Polish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Polish · EM Polish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 56/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-polish/versions/1
- measurements
- name
- EM Polish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Polish · EM Polish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.32
- standardDeviation
- 0
- confidenceInterval
- 0.0578239202885118
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0578239202885118
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.32
- standardDeviation
- 0
- confidenceInterval
- 0.0578239202885118
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0578239202885118
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 47
- sort_order
- DESCENDING
- source_name
- EM Polish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.2621760797114882
- confidence upper
- 0.3778239202885118
Original result ↗28%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Thai · EM Thai
- dimension
- reasoning
- metric
- MultiLoKo Thai · EM Thai
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 55
- name
- EM Thai
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 78
- name
- MultiLoKo Thai
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 64
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Thai
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Thai · EM Thai
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 64/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-thai/versions/1
- measurements
- name
- EM Thai
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Thai · EM Thai
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.28
- standardDeviation
- 0
- confidenceInterval
- 0.0556574558617957
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0556574558617957
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.28
- standardDeviation
- 0
- confidenceInterval
- 0.0556574558617957
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0556574558617957
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 55
- sort_order
- DESCENDING
- source_name
- EM Thai
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.22434254413820431
- confidence upper
- 0.33565745586179574
Original result ↗6.319%
- source id
- kaggleBenchmarks
- leaderboard name
- BrowseComp · Score
- dimension
- knowledge
- metric
- BrowseComp · Score
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 26
- name
- Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 49
- name
- BrowseComp
- type
- INDIVIDUAL
- owner
- openai
- version
- 1
- categories
- general knowledge and reasoning
- factuality and grounding
- version_id
- 35
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · openai · BrowseComp
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- BrowseComp · Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 35/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/openai/browsecomp/versions/1
- measurements
- name
- Score
- unit
- percent
- scale
- 1
- metric
- BrowseComp · Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.0631911532385466
- standardDeviation
- 0
- confidenceInterval
- 0.0134024708249485
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0134024708249485
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-13T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.0631911532385466
- standardDeviation
- 0
- confidenceInterval
- 0.0134024708249485
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0134024708249485
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 26
- sort_order
- DESCENDING
- source_name
- Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T14:25:30.827974100Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 42
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-13T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.0497886824135981
- confidence upper
- 0.0765936240634951
Original result ↗0.78
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Normalized (reported)
- dimension
- general
- metric
- GPQA · Normalized (reported)
- rank
- 6
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/openai-o3-mini/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 10
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.78
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Raw
- dimension
- general
- metric
- GPQA · Raw
- rank
- 6
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/openai-o3-mini/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 10
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗52%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Italian · EM Italian
- dimension
- reasoning
- metric
- MultiLoKo Italian · EM Italian
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 41
- name
- EM Italian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 64
- name
- MultiLoKo Italian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 50
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Italian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Italian · EM Italian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 50/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-italian/versions/1
- measurements
- name
- EM Italian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Italian · EM Italian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.52
- standardDeviation
- 0
- confidenceInterval
- 0.0619298997785481
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619298997785481
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.52
- standardDeviation
- 0
- confidenceInterval
- 0.0619298997785481
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619298997785481
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 41
- sort_order
- DESCENDING
- source_name
- EM Italian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.4580701002214519
- confidence upper
- 0.5819298997785481
Original result ↗51.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo French · EM French
- dimension
- reasoning
- metric
- MultiLoKo French · EM French
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 36
- name
- EM French
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 59
- name
- MultiLoKo French
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 45
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo French
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo French · EM French
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 45/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-french/versions/1
- measurements
- name
- EM French
- unit
- percent
- scale
- 1
- metric
- MultiLoKo French · EM French
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.516
- standardDeviation
- 0
- confidenceInterval
- 0.0619477615968626
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619477615968626
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.516
- standardDeviation
- 0
- confidenceInterval
- 0.0619477615968626
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619477615968626
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 36
- sort_order
- DESCENDING
- source_name
- EM French
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.4540522384031374
- confidence upper
- 0.5779477615968626
Original result ↗47.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Tagalog · EM Tagalog
- dimension
- reasoning
- metric
- MultiLoKo Tagalog · EM Tagalog
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 54
- name
- EM Tagalog
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 77
- name
- MultiLoKo Tagalog
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 63
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Tagalog
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Tagalog · EM Tagalog
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 63/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-tagalog/versions/1
- measurements
- name
- EM Tagalog
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Tagalog · EM Tagalog
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.472
- standardDeviation
- 0
- confidenceInterval
- 0.0618822430575397
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618822430575397
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.472
- standardDeviation
- 0
- confidenceInterval
- 0.0618822430575397
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618822430575397
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 54
- sort_order
- DESCENDING
- source_name
- EM Tagalog
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.4101177569424603
- confidence upper
- 0.5338822430575396
Original result ↗44%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Dutch · EM Dutch
- dimension
- reasoning
- metric
- MultiLoKo Dutch · EM Dutch
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 34
- name
- EM Dutch
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 57
- name
- MultiLoKo Dutch
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 43
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Dutch
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Dutch · EM Dutch
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 43/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-dutch/versions/1
- measurements
- name
- EM Dutch
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Dutch · EM Dutch
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.44
- standardDeviation
- 0
- confidenceInterval
- 0.0615316326264477
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0615316326264477
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.44
- standardDeviation
- 0
- confidenceInterval
- 0.0615316326264477
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0615316326264477
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 34
- sort_order
- DESCENDING
- source_name
- EM Dutch
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.3784683673735523
- confidence upper
- 0.5015316326264477
Original result ↗43.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo German · EM German
- dimension
- reasoning
- metric
- MultiLoKo German · EM German
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 37
- name
- EM German
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 60
- name
- MultiLoKo German
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 46
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo German
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo German · EM German
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 46/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-german/versions/1
- measurements
- name
- EM German
- unit
- percent
- scale
- 1
- metric
- MultiLoKo German · EM German
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.436
- standardDeviation
- 0
- confidenceInterval
- 0.061469670239687
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.061469670239687
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.436
- standardDeviation
- 0
- confidenceInterval
- 0.061469670239687
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.061469670239687
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 37
- sort_order
- DESCENDING
- source_name
- EM German
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.374530329760313
- confidence upper
- 0.497469670239687
Original result ↗39.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Vietnamese · EM Vietnamese
- dimension
- reasoning
- metric
- MultiLoKo Vietnamese · EM Vietnamese
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 59
- name
- EM Vietnamese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 82
- name
- MultiLoKo Vietnamese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 68
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Vietnamese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Vietnamese · EM Vietnamese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 68/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-vietnamese/versions/1
- measurements
- name
- EM Vietnamese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Vietnamese · EM Vietnamese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.396
- standardDeviation
- 0
- confidenceInterval
- 0.0606239387228808
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0606239387228808
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.396
- standardDeviation
- 0
- confidenceInterval
- 0.0606239387228808
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0606239387228808
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 59
- sort_order
- DESCENDING
- source_name
- EM Vietnamese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.3353760612771192
- confidence upper
- 0.4566239387228808
Original result ↗39.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Japanese · EM Japanese
- dimension
- reasoning
- metric
- MultiLoKo Japanese · EM Japanese
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 42
- name
- EM Japanese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 65
- name
- MultiLoKo Japanese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 51
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Japanese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Japanese · EM Japanese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 51/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-japanese/versions/1
- measurements
- name
- EM Japanese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Japanese · EM Japanese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.392
- standardDeviation
- 0
- confidenceInterval
- 0.06051637561814
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.06051637561814
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.392
- standardDeviation
- 0
- confidenceInterval
- 0.06051637561814
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.06051637561814
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 42
- sort_order
- DESCENDING
- source_name
- EM Japanese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.33148362438186
- confidence upper
- 0.45251637561814
Original result ↗38.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Indonesian · EM Indonesian
- dimension
- reasoning
- metric
- MultiLoKo Indonesian · EM Indonesian
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 40
- name
- EM Indonesian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 63
- name
- MultiLoKo Indonesian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 49
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Indonesian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Indonesian · EM Indonesian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 49/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-indonesian/versions/1
- measurements
- name
- EM Indonesian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Indonesian · EM Indonesian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.384
- standardDeviation
- 0
- confidenceInterval
- 0.0602884411900746
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0602884411900746
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.384
- standardDeviation
- 0
- confidenceInterval
- 0.0602884411900746
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0602884411900746
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 40
- sort_order
- DESCENDING
- source_name
- EM Indonesian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.3237115588099254
- confidence upper
- 0.4442884411900746
Original result ↗36%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Marathi · EM Marathi
- dimension
- reasoning
- metric
- MultiLoKo Marathi · EM Marathi
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 46
- name
- EM Marathi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 69
- name
- MultiLoKo Marathi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 55
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Marathi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Marathi · EM Marathi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 55/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-marathi/versions/1
- measurements
- name
- EM Marathi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Marathi · EM Marathi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.36
- standardDeviation
- 0
- confidenceInterval
- 0.0595003231012379
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0595003231012379
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.36
- standardDeviation
- 0
- confidenceInterval
- 0.0595003231012379
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0595003231012379
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 46
- sort_order
- DESCENDING
- source_name
- EM Marathi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.3004996768987621
- confidence upper
- 0.41950032310123786
Original result ↗26.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Malay · EM Malay
- dimension
- reasoning
- metric
- MultiLoKo Malay · EM Malay
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 45
- name
- EM Malay
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 68
- name
- MultiLoKo Malay
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 54
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Malay
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Malay · EM Malay
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 54/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-malay/versions/1
- measurements
- name
- EM Malay
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Malay · EM Malay
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.264
- standardDeviation
- 0
- confidenceInterval
- 0.0546410400595066
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0546410400595066
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.264
- standardDeviation
- 0
- confidenceInterval
- 0.0546410400595066
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0546410400595066
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 45
- sort_order
- DESCENDING
- source_name
- EM Malay
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.20935895994049342
- confidence upper
- 0.3186410400595066
Original result ↗24.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Farsi · EM Farsi
- dimension
- reasoning
- metric
- MultiLoKo Farsi · EM Farsi
- rank
- 6
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 35
- name
- EM Farsi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 58
- name
- MultiLoKo Farsi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 44
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Farsi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Farsi · EM Farsi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 44/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-farsi/versions/1
- measurements
- name
- EM Farsi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Farsi · EM Farsi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.244
- standardDeviation
- 0
- confidenceInterval
- 0.053239491353713
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.053239491353713
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.244
- standardDeviation
- 0
- confidenceInterval
- 0.053239491353713
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.053239491353713
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 35
- sort_order
- DESCENDING
- source_name
- EM Farsi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 50
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.190760508646287
- confidence upper
- 0.29723949135371297
Original result ↗39.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Hindi · EM Hindi
- dimension
- reasoning
- metric
- MultiLoKo Hindi · EM Hindi
- rank
- 8
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 39
- name
- EM Hindi
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 62
- name
- MultiLoKo Hindi
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 48
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Hindi
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Hindi · EM Hindi
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 48/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-hindi/versions/1
- measurements
- name
- EM Hindi
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Hindi · EM Hindi
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.396
- standardDeviation
- 0
- confidenceInterval
- 0.0606239387228808
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0606239387228808
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.396
- standardDeviation
- 0
- confidenceInterval
- 0.0606239387228808
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0606239387228808
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 39
- sort_order
- DESCENDING
- source_name
- EM Hindi
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.3353760612771192
- confidence upper
- 0.4566239387228808
Original result ↗61.134%
- source id
- liveBench
- leaderboard name
- LiveBench Language
- dimension
- intelligence
- metric
- Average score
- rank
- 9
- sample size
- 140
- evaluated at
- 2024-12-23T17:34:52.876Z
- source model id
- o1-2024-12-17-low
Original result ↗Provider specifications & access
1–1 / 1
- source model id
- azure-openai/o1@2024-12-17
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 200000
- max output tokens
- 100000
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/o1-2024-12-17
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–3 / 3
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- September 2023
- normalized value
- 2023-09
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models