OpenAI: o1
The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason...
openai/o1-2024-12-17
The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason...
Specifications & capabilities
- Input
- text, image, file
- Output
- text
- Context window · tokens
- 200000
- Knowledge cutoff
- 2023-09
- Capabilities
- multimodal, reasoning, toolCalling, visionInput
Evaluations
1–20 / 200
0.971
- source id
- llmStatsArchive
- leaderboard name
- GSM8k · Normalized (reported)
- dimension
- general
- metric
- GSM8k · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GSM8k
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.397385+00:00
- paper_link
- https://arxiv.org/abs/2110.14168
- updated_at
- 2025-07-19T19:56:11.397385+00:00
- description
- Grade School Math 8K, a dataset of 8.5K high-quality linguistically diverse grade school math word problems requiring multi-step reasoning and elementary arithmetic operations.
- benchmark_id
- gsm8k
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GSM8k · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.971
- source id
- llmStatsArchive
- leaderboard name
- GSM8k · Raw
- dimension
- general
- metric
- GSM8k · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GSM8k
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.397385+00:00
- paper_link
- https://arxiv.org/abs/2110.14168
- updated_at
- 2025-07-19T19:56:11.397385+00:00
- description
- Grade School Math 8K, a dataset of 8.5K high-quality linguistically diverse grade school math word problems requiring multi-step reasoning and elementary arithmetic operations.
- benchmark_id
- gsm8k
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GSM8k · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.964
- source id
- llmStatsArchive
- leaderboard name
- MATH · Raw
- dimension
- general
- metric
- MATH · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MATH
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.804258+00:00
- paper_link
- https://arxiv.org/abs/2103.03874
- updated_at
- 2025-07-19T19:56:11.804258+00:00
- description
- MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
- benchmark_id
- math
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MATH · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.964
- source id
- llmStatsArchive
- leaderboard name
- MATH · Normalized (reported)
- dimension
- general
- metric
- MATH · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MATH
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.804258+00:00
- paper_link
- https://arxiv.org/abs/2103.03874
- updated_at
- 2025-07-19T19:56:11.804258+00:00
- description
- MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
- benchmark_id
- math
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MATH · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.928
- source id
- llmStatsArchive
- leaderboard name
- GPQA Physics · Normalized (reported)
- dimension
- general
- metric
- GPQA Physics · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA Physics
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:15.400663+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:15.400663+00:00
- description
- Physics subset of GPQA, containing challenging multiple-choice questions written by domain experts in physics. These Google-proof questions require graduate-level knowledge and reasoning.
- benchmark_id
- gpqa-physics
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA Physics · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.928
- source id
- llmStatsArchive
- leaderboard name
- GPQA Physics · Raw
- dimension
- general
- metric
- GPQA Physics · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA Physics
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:15.400663+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:15.400663+00:00
- description
- Physics subset of GPQA, containing challenging multiple-choice questions written by domain experts in physics. These Google-proof questions require graduate-level knowledge and reasoning.
- benchmark_id
- gpqa-physics
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA Physics · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.918
- source id
- llmStatsArchive
- leaderboard name
- MMLU · Normalized (reported)
- dimension
- general
- metric
- MMLU · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- general
- reasoning
- language
- math
- created_at
- 2025-07-19T19:56:11.200416+00:00
- paper_link
- https://arxiv.org/abs/2009.03300
- updated_at
- 2025-07-19T19:56:11.200416+00:00
- description
- Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
- benchmark_id
- mmlu
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.918
- source id
- llmStatsArchive
- leaderboard name
- MMLU · Raw
- dimension
- general
- metric
- MMLU · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- general
- reasoning
- language
- math
- created_at
- 2025-07-19T19:56:11.200416+00:00
- paper_link
- https://arxiv.org/abs/2009.03300
- updated_at
- 2025-07-19T19:56:11.200416+00:00
- description
- Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
- benchmark_id
- mmlu
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.877
- source id
- llmStatsArchive
- leaderboard name
- MMMLU · Normalized (reported)
- dimension
- general
- metric
- MMMLU · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMMLU
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:14.144789+00:00
- paper_link
- https://arxiv.org/abs/2009.03300
- updated_at
- 2025-07-19T19:56:14.144789+00:00
- description
- Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili, Yoruba, and Chinese. Contains approximately 15,908 multiple-choice questions per language covering 57 subjects.
- benchmark_id
- mmmlu
- multilingual
- true
- analysis_method
- accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMMLU · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/gpt-4-1/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.877
- source id
- llmStatsArchive
- leaderboard name
- MMMLU · Raw
- dimension
- general
- metric
- MMMLU · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMMLU
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:14.144789+00:00
- paper_link
- https://arxiv.org/abs/2009.03300
- updated_at
- 2025-07-19T19:56:14.144789+00:00
- description
- Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili, Yoruba, and Chinese. Contains approximately 15,908 multiple-choice questions per language covering 57 subjects.
- benchmark_id
- mmmlu
- multilingual
- true
- analysis_method
- accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMMLU · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/gpt-4-1/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.776
- source id
- llmStatsArchive
- leaderboard name
- MMMU · Raw
- dimension
- general
- metric
- MMMU · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMMU
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- categories
- multimodal
- reasoning
- general
- created_at
- 2025-07-19T19:56:12.130105+00:00
- paper_link
- https://arxiv.org/abs/2311.16502
- updated_at
- 2025-07-19T19:56:12.130105+00:00
- description
- MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
- benchmark_id
- mmmu
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMMU · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/gpt-4-1/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.776
- source id
- llmStatsArchive
- leaderboard name
- MMMU · Normalized (reported)
- dimension
- general
- metric
- MMMU · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMMU
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- categories
- multimodal
- reasoning
- general
- created_at
- 2025-07-19T19:56:12.130105+00:00
- paper_link
- https://arxiv.org/abs/2311.16502
- updated_at
- 2025-07-19T19:56:12.130105+00:00
- description
- MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
- benchmark_id
- mmmu
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMMU · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/gpt-4-1/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.718
- source id
- llmStatsArchive
- leaderboard name
- MathVista · Normalized (reported)
- dimension
- general
- metric
- MathVista · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MathVista
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.069611+00:00
- paper_link
- https://arxiv.org/abs/2310.02255
- updated_at
- 2025-07-19T19:56:12.069611+00:00
- description
- MathVista evaluates mathematical reasoning of foundation models in visual contexts. It consists of 6,141 examples derived from 28 existing multimodal datasets and 3 newly created datasets (IQTest, FunctionQA, and PaperQA), combining challenges from diverse mathematical and visual tasks to assess models' ability to understand complex figures and perform rigorous reasoning.
- benchmark_id
- mathvista
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MathVista · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/gpt-4-1/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.718
- source id
- llmStatsArchive
- leaderboard name
- MathVista · Raw
- dimension
- general
- metric
- MathVista · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MathVista
- language
- en
- modality
- multimodal
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.069611+00:00
- paper_link
- https://arxiv.org/abs/2310.02255
- updated_at
- 2025-07-19T19:56:12.069611+00:00
- description
- MathVista evaluates mathematical reasoning of foundation models in visual contexts. It consists of 6,141 examples derived from 28 existing multimodal datasets and 3 newly created datasets (IQTest, FunctionQA, and PaperQA), combining challenges from diverse mathematical and visual tasks to assess models' ability to understand complex figures and perform rigorous reasoning.
- benchmark_id
- mathvista
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MathVista · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/gpt-4-1/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.692
- source id
- llmStatsArchive
- leaderboard name
- GPQA Biology · Normalized (reported)
- dimension
- general
- metric
- GPQA Biology · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA Biology
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:15.391187+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:15.391187+00:00
- description
- Biology subset of GPQA, containing challenging multiple-choice questions written by domain experts in biology. These Google-proof questions require graduate-level knowledge and reasoning.
- benchmark_id
- gpqa-biology
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA Biology · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.692
- source id
- llmStatsArchive
- leaderboard name
- GPQA Biology · Raw
- dimension
- general
- metric
- GPQA Biology · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA Biology
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:15.391187+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:15.391187+00:00
- description
- Biology subset of GPQA, containing challenging multiple-choice questions written by domain experts in biology. These Google-proof questions require graduate-level knowledge and reasoning.
- benchmark_id
- gpqa-biology
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA Biology · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.647
- source id
- llmStatsArchive
- leaderboard name
- GPQA Chemistry · Raw
- dimension
- general
- metric
- GPQA Chemistry · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA Chemistry
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:15.395806+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:15.395806+00:00
- description
- Chemistry subset of GPQA, containing challenging multiple-choice questions written by domain experts in chemistry. These Google-proof questions require graduate-level knowledge and reasoning.
- benchmark_id
- gpqa-chemistry
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA Chemistry · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.647
- source id
- llmStatsArchive
- leaderboard name
- GPQA Chemistry · Normalized (reported)
- dimension
- general
- metric
- GPQA Chemistry · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA Chemistry
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:15.395806+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:15.395806+00:00
- description
- Chemistry subset of GPQA, containing challenging multiple-choice questions written by domain experts in chemistry. These Google-proof questions require graduate-level knowledge and reasoning.
- benchmark_id
- gpqa-chemistry
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA Chemistry · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/learning-to-reason-with-llms/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.055
- source id
- llmStatsArchive
- leaderboard name
- FrontierMath · Normalized (reported)
- dimension
- general
- metric
- FrontierMath · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- FrontierMath
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:15.179213+00:00
- paper_link
- https://arxiv.org/abs/2411.04872
- updated_at
- 2025-07-19T19:56:15.179213+00:00
- description
- A benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians, covering most major branches of modern mathematics from number theory and real analysis to algebraic geometry and category theory.
- benchmark_id
- frontiermath
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- FrontierMath · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/o1-and-new-tools-for-developers/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗0.055
- source id
- llmStatsArchive
- leaderboard name
- FrontierMath · Raw
- dimension
- general
- metric
- FrontierMath · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- FrontierMath
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:15.179213+00:00
- paper_link
- https://arxiv.org/abs/2411.04872
- updated_at
- 2025-07-19T19:56:15.179213+00:00
- description
- A benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians, covering most major branches of modern mathematics from number theory and real analysis to algebraic geometry and category theory.
- benchmark_id
- frontiermath
- multilingual
- false
- analysis_method
- pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- FrontierMath · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/openai/models/o1-2024-12-17/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://openai.com/index/o1-and-new-tools-for-developers/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- openai/o1-2024-12-17
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- o1-2024-12-17
Original result ↗Provider specifications & access
1–1 / 1
- source model id
- azure-openai/o1@2024-12-17
- hosting
- foundry-managed-compute
- license
- custom
- context length
- 200000
- max output tokens
- 100000
Safety assessments
1–1 / 1
35 / 100
- source id
- stanfordFMTI
- canonical model id
- openai/o1-2024-12-17
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- OpenAI
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 35 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–3 / 3
- field
- context_length
- value
- 200000
- normalized value
- 200000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- max_output_tokens
- value
- 100000
- normalized value
- 100000
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- training_data_cutoff
- value
- September 2023
- normalized value
- 2023-09
- precision
- month
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
Explore more models