OpenAI: o1

The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason...

openai/o1-2024-12-17

The latest and strongest model family from OpenAI, o1 is designed to spend more time thinking before responding. The o1 model series is trained with large-scale reinforcement learning to reason...

Specifications & capabilities

Input
text, image, file
Output
text
Context window · tokens
200000
Knowledge cutoff
2023-09
Capabilities
multimodal, reasoning, toolCalling, visionInput

Evaluations

1–20 / 200

LLM Stats · GitHub archive

0.971

source id
llmStatsArchive
leaderboard name
GSM8k · Normalized (reported)
dimension
general
metric
GSM8k · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GSM8k
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.397385+00:00
paper_link
https://arxiv.org/abs/2110.14168
updated_at
2025-07-19T19:56:11.397385+00:00
description
Grade School Math 8K, a dataset of 8.5K high-quality linguistically diverse grade school math word problems requiring multi-step reasoning and elementary arithmetic operations.
benchmark_id
gsm8k
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GSM8k · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.971

source id
llmStatsArchive
leaderboard name
GSM8k · Raw
dimension
general
metric
GSM8k · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GSM8k
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.397385+00:00
paper_link
https://arxiv.org/abs/2110.14168
updated_at
2025-07-19T19:56:11.397385+00:00
description
Grade School Math 8K, a dataset of 8.5K high-quality linguistically diverse grade school math word problems requiring multi-step reasoning and elementary arithmetic operations.
benchmark_id
gsm8k
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GSM8k · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.964

source id
llmStatsArchive
leaderboard name
MATH · Raw
dimension
general
metric
MATH · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MATH
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.804258+00:00
paper_link
https://arxiv.org/abs/2103.03874
updated_at
2025-07-19T19:56:11.804258+00:00
description
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
benchmark_id
math
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MATH · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.964

source id
llmStatsArchive
leaderboard name
MATH · Normalized (reported)
dimension
general
metric
MATH · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MATH
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.804258+00:00
paper_link
https://arxiv.org/abs/2103.03874
updated_at
2025-07-19T19:56:11.804258+00:00
description
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
benchmark_id
math
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MATH · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.928

source id
llmStatsArchive
leaderboard name
GPQA Physics · Normalized (reported)
dimension
general
metric
GPQA Physics · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA Physics
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • physics
created_at
2025-07-19T19:56:15.400663+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:15.400663+00:00
description
Physics subset of GPQA, containing challenging multiple-choice questions written by domain experts in physics. These Google-proof questions require graduate-level knowledge and reasoning.
benchmark_id
gpqa-physics
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA Physics · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.928

source id
llmStatsArchive
leaderboard name
GPQA Physics · Raw
dimension
general
metric
GPQA Physics · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA Physics
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • physics
created_at
2025-07-19T19:56:15.400663+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:15.400663+00:00
description
Physics subset of GPQA, containing challenging multiple-choice questions written by domain experts in physics. These Google-proof questions require graduate-level knowledge and reasoning.
benchmark_id
gpqa-physics
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA Physics · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.918

source id
llmStatsArchive
leaderboard name
MMLU · Normalized (reported)
dimension
general
metric
MMLU · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
  • language
  • math
created_at
2025-07-19T19:56:11.200416+00:00
paper_link
https://arxiv.org/abs/2009.03300
updated_at
2025-07-19T19:56:11.200416+00:00
description
Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
benchmark_id
mmlu
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.918

source id
llmStatsArchive
leaderboard name
MMLU · Raw
dimension
general
metric
MMLU · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
  • language
  • math
created_at
2025-07-19T19:56:11.200416+00:00
paper_link
https://arxiv.org/abs/2009.03300
updated_at
2025-07-19T19:56:11.200416+00:00
description
Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
benchmark_id
mmlu
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.877

source id
llmStatsArchive
leaderboard name
MMMLU · Normalized (reported)
dimension
general
metric
MMMLU · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMLU
language
en
modality
text
verified
false
max_score
1
categories
  • language
  • reasoning
  • math
  • general
created_at
2025-07-19T19:56:14.144789+00:00
paper_link
https://arxiv.org/abs/2009.03300
updated_at
2025-07-19T19:56:14.144789+00:00
description
Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili, Yoruba, and Chinese. Contains approximately 15,908 multiple-choice questions per language covering 57 subjects.
benchmark_id
mmmlu
multilingual
true
analysis_method
accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMLU · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/gpt-4-1/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.877

source id
llmStatsArchive
leaderboard name
MMMLU · Raw
dimension
general
metric
MMMLU · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMLU
language
en
modality
text
verified
false
max_score
1
categories
  • language
  • reasoning
  • math
  • general
created_at
2025-07-19T19:56:14.144789+00:00
paper_link
https://arxiv.org/abs/2009.03300
updated_at
2025-07-19T19:56:14.144789+00:00
description
Multilingual Massive Multitask Language Understanding dataset released by OpenAI, featuring professionally translated MMLU test questions across 14 languages including Arabic, Bengali, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Portuguese, Swahili, Yoruba, and Chinese. Contains approximately 15,908 multiple-choice questions per language covering 57 subjects.
benchmark_id
mmmlu
multilingual
true
analysis_method
accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMLU · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/gpt-4-1/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.776

source id
llmStatsArchive
leaderboard name
MMMU · Raw
dimension
general
metric
MMMU · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMU
language
en
modality
multimodal
verified
false
max_score
1
categories
  • multimodal
  • reasoning
  • general
created_at
2025-07-19T19:56:12.130105+00:00
paper_link
https://arxiv.org/abs/2311.16502
updated_at
2025-07-19T19:56:12.130105+00:00
description
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
benchmark_id
mmmu
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMU · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/gpt-4-1/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.776

source id
llmStatsArchive
leaderboard name
MMMU · Normalized (reported)
dimension
general
metric
MMMU · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMU
language
en
modality
multimodal
verified
false
max_score
1
categories
  • multimodal
  • reasoning
  • general
created_at
2025-07-19T19:56:12.130105+00:00
paper_link
https://arxiv.org/abs/2311.16502
updated_at
2025-07-19T19:56:12.130105+00:00
description
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
benchmark_id
mmmu
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMU · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/gpt-4-1/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.718

source id
llmStatsArchive
leaderboard name
MathVista · Normalized (reported)
dimension
general
metric
MathVista · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MathVista
language
en
modality
multimodal
verified
false
max_score
1
categories
  • math
  • vision
  • multimodal
created_at
2025-07-19T19:56:12.069611+00:00
paper_link
https://arxiv.org/abs/2310.02255
updated_at
2025-07-19T19:56:12.069611+00:00
description
MathVista evaluates mathematical reasoning of foundation models in visual contexts. It consists of 6,141 examples derived from 28 existing multimodal datasets and 3 newly created datasets (IQTest, FunctionQA, and PaperQA), combining challenges from diverse mathematical and visual tasks to assess models' ability to understand complex figures and perform rigorous reasoning.
benchmark_id
mathvista
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MathVista · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/gpt-4-1/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.718

source id
llmStatsArchive
leaderboard name
MathVista · Raw
dimension
general
metric
MathVista · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MathVista
language
en
modality
multimodal
verified
false
max_score
1
categories
  • math
  • vision
  • multimodal
created_at
2025-07-19T19:56:12.069611+00:00
paper_link
https://arxiv.org/abs/2310.02255
updated_at
2025-07-19T19:56:12.069611+00:00
description
MathVista evaluates mathematical reasoning of foundation models in visual contexts. It consists of 6,141 examples derived from 28 existing multimodal datasets and 3 newly created datasets (IQTest, FunctionQA, and PaperQA), combining challenges from diverse mathematical and visual tasks to assess models' ability to understand complex figures and perform rigorous reasoning.
benchmark_id
mathvista
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MathVista · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/gpt-4-1/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.692

source id
llmStatsArchive
leaderboard name
GPQA Biology · Normalized (reported)
dimension
general
metric
GPQA Biology · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA Biology
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
created_at
2025-07-19T19:56:15.391187+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:15.391187+00:00
description
Biology subset of GPQA, containing challenging multiple-choice questions written by domain experts in biology. These Google-proof questions require graduate-level knowledge and reasoning.
benchmark_id
gpqa-biology
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA Biology · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.692

source id
llmStatsArchive
leaderboard name
GPQA Biology · Raw
dimension
general
metric
GPQA Biology · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA Biology
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
created_at
2025-07-19T19:56:15.391187+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:15.391187+00:00
description
Biology subset of GPQA, containing challenging multiple-choice questions written by domain experts in biology. These Google-proof questions require graduate-level knowledge and reasoning.
benchmark_id
gpqa-biology
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA Biology · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.647

source id
llmStatsArchive
leaderboard name
GPQA Chemistry · Raw
dimension
general
metric
GPQA Chemistry · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA Chemistry
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • chemistry
created_at
2025-07-19T19:56:15.395806+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:15.395806+00:00
description
Chemistry subset of GPQA, containing challenging multiple-choice questions written by domain experts in chemistry. These Google-proof questions require graduate-level knowledge and reasoning.
benchmark_id
gpqa-chemistry
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA Chemistry · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.647

source id
llmStatsArchive
leaderboard name
GPQA Chemistry · Normalized (reported)
dimension
general
metric
GPQA Chemistry · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA Chemistry
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • chemistry
created_at
2025-07-19T19:56:15.395806+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:15.395806+00:00
description
Chemistry subset of GPQA, containing challenging multiple-choice questions written by domain experts in chemistry. These Google-proof questions require graduate-level knowledge and reasoning.
benchmark_id
gpqa-chemistry
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA Chemistry · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/learning-to-reason-with-llms/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.055

source id
llmStatsArchive
leaderboard name
FrontierMath · Normalized (reported)
dimension
general
metric
FrontierMath · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
FrontierMath
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:15.179213+00:00
paper_link
https://arxiv.org/abs/2411.04872
updated_at
2025-07-19T19:56:15.179213+00:00
description
A benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians, covering most major branches of modern mathematics from number theory and real analysis to algebraic geometry and category theory.
benchmark_id
frontiermath
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
FrontierMath · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/o1-and-new-tools-for-developers/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

LLM Stats · GitHub archive

0.055

source id
llmStatsArchive
leaderboard name
FrontierMath · Raw
dimension
general
metric
FrontierMath · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
FrontierMath
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:15.179213+00:00
paper_link
https://arxiv.org/abs/2411.04872
updated_at
2025-07-19T19:56:15.179213+00:00
description
A benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians, covering most major branches of modern mathematics from number theory and real analysis to algebraic geometry and category theory.
benchmark_id
frontiermath
multilingual
false
analysis_method
pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
FrontierMath · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o1-2024-12-17/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/o1-and-new-tools-for-developers/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o1-2024-12-17
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o1-2024-12-17
Original result ↗

Provider specifications & access

1–1 / 1

OpenAI o1 · azure-openai v2024-12-17 ↗

source model id
azure-openai/o1@2024-12-17
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
custom
context length
200000
max output tokens
100000
architecture
input_modalities
  • text
  • image
output_modalities
  • text
capabilities
  • reasoning
  • toolCalling

Safety assessments

1–1 / 1

Foundation Model Transparency Index 2025 ↗

35 / 100

source id
stanfordFMTI
canonical model id
openai/o1-2024-12-17
taxonomy
transparency
hazard category
transparency
hazard name
Organization transparency
policy area
OpenAI
assessment type
organization-level disclosure
metric
disclosure_score
result label
35 / 100
sample size
100
updated at
2026-11-09T18:08:05.227Z

Source specification observations

1–3 / 3

azureFoundry ↗

field
context_length
value
200000
normalized value
200000
scope
source-model
source id
azureFoundry
confidence
1

azureFoundry ↗

field
max_output_tokens
value
100000
normalized value
100000
scope
source-model
source id
azureFoundry
confidence
1

azureFoundry ↗

field
knowledge_cutoff
reported field
training_data_cutoff
value
September 2023
normalized value
2023-09
precision
month
scope
source-model
source id
azureFoundry
confidence
1

Original sources

1–8 / 8

Microsoft Foundry · Model page ↗

Microsoft Foundry · Model documentation ↗

GitHub · Catalog ↗

GitHub · Catalog ↗

Kaggle · Catalog ↗

LMArena · Catalog ↗

Microsoft Foundry · Catalog ↗

livebench.ai · Catalog ↗

Source links

1–7 / 7

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace