OpenAI: o1
The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason...
openai/o1-2024-12-17
The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason...
Specifications & capabilities
- Input
- text, image, file
- Output
- text
- Context window · tokens
- 200000
- Knowledge cutoff
- 2023-09
- Capabilities
- multimodal, reasoning, toolCalling, visionInput
Evaluations
21–40 / 200
0.893
- source id
- llmStatsArchive
- leaderboard name
- MGSM · Normalized (reported)
- dimension
- general
- metric
- MGSM · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MGSM
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.669061+00:00
- paper_link
- https://arxiv.org/abs/2210.03057
- updated_at
- 2025-07-19T19:56:13.669061+00:00
- description
- MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
- benchmark_id
- mgsm
- multilingual
- true
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MGSM · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/o1-and-new-tools-for-developers/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.893
- source id
- llmStatsArchive
- leaderboard name
- MGSM · Raw
- dimension
- general
- metric
- MGSM · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MGSM
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.669061+00:00
- paper_link
- https://arxiv.org/abs/2210.03057
- updated_at
- 2025-07-19T19:56:13.669061+00:00
- description
- MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
- benchmark_id
- mgsm
- multilingual
- true
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MGSM · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/o1-and-new-tools-for-developers/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.47
- source id
- llmStatsArchive
- leaderboard name
- SimpleQA · Normalized (reported)
- dimension
- general
- metric
- SimpleQA · Normalized (reported)
- rank
- 3
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- SimpleQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2411.04368
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.
- benchmark_id
- simpleqa
- multilingual
- false
- analysis_method
- accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- SimpleQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/introducing-gpt-4-5/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.47
- source id
- llmStatsArchive
- leaderboard name
- SimpleQA · Raw
- dimension
- general
- metric
- SimpleQA · Raw
- rank
- 3
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- SimpleQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2411.04368
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.
- benchmark_id
- simpleqa
- multilingual
- false
- analysis_method
- accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- SimpleQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/introducing-gpt-4-5/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.881
- source id
- llmStatsArchive
- leaderboard name
- HumanEval · Raw
- dimension
- general
- metric
- HumanEval · Raw
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- HumanEval
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.595263+00:00
- paper_link
- https://arxiv.org/abs/2107.03374
- updated_at
- 2025-07-19T19:56:12.595263+00:00
- description
- A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
- benchmark_id
- humaneval
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- HumanEval · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 10
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.881
- source id
- llmStatsArchive
- leaderboard name
- HumanEval · Normalized (reported)
- dimension
- general
- metric
- HumanEval · Normalized (reported)
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- HumanEval
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.595263+00:00
- paper_link
- https://arxiv.org/abs/2107.03374
- updated_at
- 2025-07-19T19:56:12.595263+00:00
- description
- A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
- benchmark_id
- humaneval
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- HumanEval · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 10
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.743
- source id
- llmStatsArchive
- leaderboard name
- AIME 2024 · Normalized (reported)
- dimension
- general
- metric
- AIME 2024 · Normalized (reported)
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AIME 2024
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.941652+00:00
- paper_link
- https://arxiv.org/html/2503.21380v2
- updated_at
- 2025-09-30T00:00:00.000000+00:00
- description
- American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
- benchmark_id
- aime-2024
- multilingual
- false
- analysis_method
- accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AIME 2024 · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/gpt-4-1/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.743
- source id
- llmStatsArchive
- leaderboard name
- AIME 2024 · Raw
- dimension
- general
- metric
- AIME 2024 · Raw
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AIME 2024
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.941652+00:00
- paper_link
- https://arxiv.org/html/2503.21380v2
- updated_at
- 2025-09-30T00:00:00.000000+00:00
- description
- American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
- benchmark_id
- aime-2024
- multilingual
- false
- analysis_method
- accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AIME 2024 · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/gpt-4-1/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗66.4%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo English · EM English
- dimension
- reasoning
- metric
- MultiLoKo English · EM English
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 27
- name
- EM English
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 52
- name
- MultiLoKo English
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 38
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo English
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo English · EM English
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 38/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-english/versions/1
- measurements
- name
- EM English
- unit
- percent
- scale
- 1
- metric
- MultiLoKo English · EM English
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.664
- standardDeviation
- 0
- confidenceInterval
- 0.0585506559735121
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0585506559735121
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.664
- standardDeviation
- 0
- confidenceInterval
- 0.0585506559735121
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0585506559735121
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 27
- sort_order
- DESCENDING
- source_name
- EM English
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.605449344026488
- confidence upper
- 0.7225506559735121
Original result ↗64.755%
- source id
- liveBench
- leaderboard name
- LiveBench Language
- dimension
- intelligence
- metric
- Average score
- rank
- 4
- sample size
- 140
- evaluated at
- 2025-03-04T16:25:32.323Z
- source model id
- o1-2024-12-17-medium
Original result ↗59.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Portuguese · EM Portuguese
- dimension
- reasoning
- metric
- MultiLoKo Portuguese · EM Portuguese
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 48
- name
- EM Portuguese
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 71
- name
- MultiLoKo Portuguese
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 57
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Portuguese
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Portuguese · EM Portuguese
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 57/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-portuguese/versions/1
- measurements
- name
- EM Portuguese
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Portuguese · EM Portuguese
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.592
- standardDeviation
- 0
- confidenceInterval
- 0.0609212802792317
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0609212802792317
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.592
- standardDeviation
- 0
- confidenceInterval
- 0.0609212802792317
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0609212802792317
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 48
- sort_order
- DESCENDING
- source_name
- EM Portuguese
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.5310787197207683
- confidence upper
- 0.6529212802792317
Original result ↗54.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Spanish · EM Spanish
- dimension
- reasoning
- metric
- MultiLoKo Spanish · EM Spanish
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 52
- name
- EM Spanish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 75
- name
- MultiLoKo Spanish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 61
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Spanish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Spanish · EM Spanish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 61/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-spanish/versions/1
- measurements
- name
- EM Spanish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Spanish · EM Spanish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.548
- standardDeviation
- 0
- confidenceInterval
- 0.0616932406038344
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0616932406038344
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.548
- standardDeviation
- 0
- confidenceInterval
- 0.0616932406038344
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0616932406038344
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 52
- sort_order
- DESCENDING
- source_name
- EM Spanish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.48630675939616563
- confidence upper
- 0.6096932406038345
Original result ↗47.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Romanian · EM Romanian
- dimension
- reasoning
- metric
- MultiLoKo Romanian · EM Romanian
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 49
- name
- EM Romanian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 72
- name
- MultiLoKo Romanian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 58
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Romanian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Romanian · EM Romanian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 58/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-romanian/versions/1
- measurements
- name
- EM Romanian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Romanian · EM Romanian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.472
- standardDeviation
- 0
- confidenceInterval
- 0.0618822430575397
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618822430575397
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.472
- standardDeviation
- 0
- confidenceInterval
- 0.0618822430575397
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0618822430575397
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 49
- sort_order
- DESCENDING
- source_name
- EM Romanian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.4101177569424603
- confidence upper
- 0.5338822430575396
Original result ↗44.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Swedish · EM Swedish
- dimension
- reasoning
- metric
- MultiLoKo Swedish · EM Swedish
- rank
- 4
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 53
- name
- EM Swedish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 76
- name
- MultiLoKo Swedish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 62
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Swedish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Swedish · EM Swedish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 62/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-swedish/versions/1
- measurements
- name
- EM Swedish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Swedish · EM Swedish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.448
- standardDeviation
- 0
- confidenceInterval
- 0.0616434068014536
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0616434068014536
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.448
- standardDeviation
- 0
- confidenceInterval
- 0.0616434068014536
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0616434068014536
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 53
- sort_order
- DESCENDING
- source_name
- EM Swedish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.3863565931985464
- confidence upper
- 0.5096434068014536
Original result ↗67.451%
- source id
- liveBench
- leaderboard name
- LiveBench Overall
- dimension
- general
- metric
- Average score
- rank
- 5
- sample size
- 418
- evaluated at
- 2025-02-06T20:21:56.570Z
- source model id
- o1-2024-12-17-high
Original result ↗63.539%
- source id
- liveBench
- leaderboard name
- LiveBench Language
- dimension
- intelligence
- metric
- Average score
- rank
- 5
- sample size
- 240
- evaluated at
- 2025-02-06T20:21:56.570Z
- source model id
- o1-2024-12-17-high
Original result ↗49.2%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Czech · EM Czech
- dimension
- reasoning
- metric
- MultiLoKo Czech · EM Czech
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 33
- name
- EM Czech
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 56
- name
- MultiLoKo Czech
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 42
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Czech
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Czech · EM Czech
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 42/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-czech/versions/1
- measurements
- name
- EM Czech
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Czech · EM Czech
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.492
- standardDeviation
- 0
- confidenceInterval
- 0.0619715693462415
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619715693462415
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.492
- standardDeviation
- 0
- confidenceInterval
- 0.0619715693462415
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619715693462415
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 33
- sort_order
- DESCENDING
- source_name
- EM Czech
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.4300284306537585
- confidence upper
- 0.5539715693462415
Original result ↗48%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Russian · EM Russian
- dimension
- reasoning
- metric
- MultiLoKo Russian · EM Russian
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 50
- name
- EM Russian
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 73
- name
- MultiLoKo Russian
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 59
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Russian
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Russian · EM Russian
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 59/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-russian/versions/1
- measurements
- name
- EM Russian
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Russian · EM Russian
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.48
- standardDeviation
- 0
- confidenceInterval
- 0.0619298997785481
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619298997785481
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.48
- standardDeviation
- 0
- confidenceInterval
- 0.0619298997785481
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619298997785481
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 50
- sort_order
- DESCENDING
- source_name
- EM Russian
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.41807010022145186
- confidence upper
- 0.5419298997785481
Original result ↗47.6%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Turkish · EM Turkish
- dimension
- reasoning
- metric
- MultiLoKo Turkish · EM Turkish
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 57
- name
- EM Turkish
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 80
- name
- MultiLoKo Turkish
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 66
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Turkish
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Turkish · EM Turkish
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 66/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-turkish/versions/1
- measurements
- name
- EM Turkish
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Turkish · EM Turkish
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.476
- standardDeviation
- 0
- confidenceInterval
- 0.0619080616686651
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619080616686651
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.476
- standardDeviation
- 0
- confidenceInterval
- 0.0619080616686651
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0619080616686651
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 57
- sort_order
- DESCENDING
- source_name
- EM Turkish
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.4140919383313349
- confidence upper
- 0.5379080616686651
Original result ↗40.8%
- source id
- kaggleBenchmarks
- leaderboard name
- MultiLoKo Hebrew · EM Hebrew
- dimension
- reasoning
- metric
- MultiLoKo Hebrew · EM Hebrew
- rank
- 5
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 38
- name
- EM Hebrew
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 61
- name
- MultiLoKo Hebrew
- type
- INDIVIDUAL
- owner
- metaresearch
- version
- 1
- categories
- multilingual and cross-lingual capabilities
- general knowledge and reasoning
- version_id
- 47
- maintainer_tier
- Kaggle-maintained
- maintenance_level
- KAGGLE_MAINTAINED
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · metaresearch · MultiLoKo Hebrew
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- MultiLoKo Hebrew · EM Hebrew
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 47/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/metaresearch/multiloko-hebrew/versions/1
- measurements
- name
- EM Hebrew
- unit
- percent
- scale
- 1
- metric
- MultiLoKo Hebrew · EM Hebrew
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.408
- standardDeviation
- 0
- confidenceInterval
- 0.0609212802792317
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0609212802792317
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2025-06-16T00:00:00Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.408
- standardDeviation
- 0
- confidenceInterval
- 0.0609212802792317
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 0.0609212802792317
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 38
- sort_order
- DESCENDING
- source_name
- EM Hebrew
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2025-07-15T11:49:31.790Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 51
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2025-06-16T00:00:00.000Z
- source model id
- o1
- confidence lower
- 0.34707871972076826
- confidence upper
- 0.4689212802792317
Original result ↗Provider specifications & access
1–1 / 1
- source model id
- azure-openai/o1@2024-12-17
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 200000
- max output tokens
- 100000
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/o1-2024-12-17
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–3 / 3
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- September 2023
- normalized value
- 2023-09
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models