OpenAI: GPT-4o (2024-05-13)

GPT-4o ("o" for "omni") is OpenAI's latest AI model, supporting both text and image inputs with text outputs. It maintains the intelligence level of [GPT-4 Turbo](/models/openai/gpt-4-turbo) while be…

openai/gpt-4o-2024-05-13

GPT-4o ("o" for "omni") is OpenAI's latest AI model, supporting both text and image inputs with text outputs. It maintains the intelligence level of [GPT-4 Turbo](/models/openai/gpt-4-turbo) while being twice as fast and 50% more cost-effective. GPT-4o also offers improved performance in processing non-English languages and enhanced visual capabilities.

For benchmarking against other models, it was briefly called ["im-also-a-good-gpt2-chatbot"](https://twitter.com/LiamFedus/status/1790064963966370209)

#multimodal

Specifications & capabilities

Input
text, image, file
Output
text
Context window · tokens
128000
Knowledge cutoff
2023-10
Capabilities
audioInput, fileInput, multimodal, structuredOutputs, toolCalling, visionInput
Description source ↗

Evaluations

1–20 / 85

LLM Stats · GitHub archive

0.905

source id
llmStatsArchive
leaderboard name
MGSM · Normalized (reported)
dimension
general
metric
MGSM · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MGSM
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:13.669061+00:00
paper_link
https://arxiv.org/abs/2210.03057
updated_at
2025-07-19T19:56:13.669061+00:00
description
MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
benchmark_id
mgsm
multilingual
true
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MGSM · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.905

source id
llmStatsArchive
leaderboard name
MGSM · Raw
dimension
general
metric
MGSM · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MGSM
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:13.669061+00:00
paper_link
https://arxiv.org/abs/2210.03057
updated_at
2025-07-19T19:56:13.669061+00:00
description
MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
benchmark_id
mgsm
multilingual
true
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MGSM · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.887

source id
llmStatsArchive
leaderboard name
MMLU · Raw
dimension
general
metric
MMLU · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
  • language
  • math
created_at
2025-07-19T19:56:11.200416+00:00
paper_link
https://arxiv.org/abs/2009.03300
updated_at
2025-07-19T19:56:11.200416+00:00
description
Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
benchmark_id
mmlu
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
8
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.887

source id
llmStatsArchive
leaderboard name
MMLU · Normalized (reported)
dimension
general
metric
MMLU · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
  • language
  • math
created_at
2025-07-19T19:56:11.200416+00:00
paper_link
https://arxiv.org/abs/2009.03300
updated_at
2025-07-19T19:56:11.200416+00:00
description
Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
benchmark_id
mmlu
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
8
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.834

source id
llmStatsArchive
leaderboard name
DROP · Raw
dimension
general
metric
DROP · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
DROP
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • math
created_at
2025-07-19T19:56:12.981569+00:00
paper_link
https://arxiv.org/abs/1903.00161
updated_at
2025-07-19T19:56:12.981569+00:00
description
DROP (Discrete Reasoning Over Paragraphs) is a reading comprehension benchmark requiring discrete reasoning over paragraph content. It contains crowdsourced, adversarially-created questions that require resolving references and performing discrete operations like addition, counting, or sorting, demanding comprehensive paragraph understanding beyond paraphrase-and-entity-typing shortcuts.
benchmark_id
drop
multilingual
false
analysis_method
F1 Score
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
DROP · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.834

source id
llmStatsArchive
leaderboard name
DROP · Normalized (reported)
dimension
general
metric
DROP · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
DROP
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • math
created_at
2025-07-19T19:56:12.981569+00:00
paper_link
https://arxiv.org/abs/1903.00161
updated_at
2025-07-19T19:56:12.981569+00:00
description
DROP (Discrete Reasoning Over Paragraphs) is a reading comprehension benchmark requiring discrete reasoning over paragraph content. It contains crowdsourced, adversarially-created questions that require resolving references and performing discrete operations like addition, counting, or sorting, demanding comprehensive paragraph understanding beyond paraphrase-and-entity-typing shortcuts.
benchmark_id
drop
multilingual
false
analysis_method
F1 Score
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
DROP · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.902

source id
llmStatsArchive
leaderboard name
HumanEval · Raw
dimension
general
metric
HumanEval · Raw
rank
3
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
HumanEval
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • code
created_at
2025-07-19T19:56:12.595263+00:00
paper_link
https://arxiv.org/abs/2107.03374
updated_at
2025-07-19T19:56:12.595263+00:00
description
A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
benchmark_id
humaneval
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
HumanEval · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
9
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.902

source id
llmStatsArchive
leaderboard name
HumanEval · Normalized (reported)
dimension
general
metric
HumanEval · Normalized (reported)
rank
3
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
HumanEval
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • code
created_at
2025-07-19T19:56:12.595263+00:00
paper_link
https://arxiv.org/abs/2107.03374
updated_at
2025-07-19T19:56:12.595263+00:00
description
A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
benchmark_id
humaneval
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
HumanEval · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
9
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.766

source id
llmStatsArchive
leaderboard name
MATH · Normalized (reported)
dimension
general
metric
MATH · Normalized (reported)
rank
3
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MATH
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.804258+00:00
paper_link
https://arxiv.org/abs/2103.03874
updated_at
2025-07-19T19:56:11.804258+00:00
description
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
benchmark_id
math
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MATH · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
7
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.766

source id
llmStatsArchive
leaderboard name
MATH · Raw
dimension
general
metric
MATH · Raw
rank
3
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MATH
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.804258+00:00
paper_link
https://arxiv.org/abs/2103.03874
updated_at
2025-07-19T19:56:11.804258+00:00
description
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
benchmark_id
math
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MATH · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
7
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.638

source id
llmStatsArchive
leaderboard name
MathVista · Raw
dimension
general
metric
MathVista · Raw
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MathVista
language
en
modality
multimodal
verified
false
max_score
1
categories
  • math
  • vision
  • multimodal
created_at
2025-07-19T19:56:12.069611+00:00
paper_link
https://arxiv.org/abs/2310.02255
updated_at
2025-07-19T19:56:12.069611+00:00
description
MathVista evaluates mathematical reasoning of foundation models in visual contexts. It consists of 6,141 examples derived from 28 existing multimodal datasets and 3 newly created datasets (IQTest, FunctionQA, and PaperQA), combining challenges from diverse mathematical and visual tasks to assess models' ability to understand complex figures and perform rigorous reasoning.
benchmark_id
mathvista
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MathVista · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
6
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.638

source id
llmStatsArchive
leaderboard name
MathVista · Normalized (reported)
dimension
general
metric
MathVista · Normalized (reported)
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MathVista
language
en
modality
multimodal
verified
false
max_score
1
categories
  • math
  • vision
  • multimodal
created_at
2025-07-19T19:56:12.069611+00:00
paper_link
https://arxiv.org/abs/2310.02255
updated_at
2025-07-19T19:56:12.069611+00:00
description
MathVista evaluates mathematical reasoning of foundation models in visual contexts. It consists of 6,141 examples derived from 28 existing multimodal datasets and 3 newly created datasets (IQTest, FunctionQA, and PaperQA), combining challenges from diverse mathematical and visual tasks to assess models' ability to understand complex figures and perform rigorous reasoning.
benchmark_id
mathvista
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MathVista · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
6
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.536

source id
llmStatsArchive
leaderboard name
GPQA · Raw
dimension
general
metric
GPQA · Raw
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
created_at
2025-07-19T19:56:11.588605+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:11.588605+00:00
description
A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
benchmark_id
gpqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
11
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LLM Stats · GitHub archive

0.536

source id
llmStatsArchive
leaderboard name
GPQA · Normalized (reported)
dimension
general
metric
GPQA · Normalized (reported)
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
created_at
2025-07-19T19:56:11.588605+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:11.588605+00:00
description
A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
benchmark_id
gpqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/gpt-4o-2024-05-13/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/blog/gpt-4o
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/gpt-4o-2024-05-13
metric_population
11
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
gpt-4o-2024-05-13
Original result ↗

LiveBench

51.205%

source id
liveBench
leaderboard name
LiveBench Language
dimension
intelligence
metric
Average score
rank
20
sample size
190
evaluated at
2024-12-10T19:31:44.598Z
source model id
gpt-4o-2024-05-13
Original result ↗

LiveBench

54.359%

source id
liveBench
leaderboard name
LiveBench Overall
dimension
general
metric
Average score
rank
28
sample size
394
evaluated at
2025-02-06T18:32:40.620Z
source model id
gpt-4o-2024-05-13
Original result ↗

LiveBench

50%

source id
liveBench
leaderboard name
LiveBench Coding
dimension
coding
metric
Average score
rank
46
sample size
128
evaluated at
2025-02-06T18:32:40.620Z
source model id
gpt-4o-2024-05-13
Original result ↗

LiveBench

69.583%

source id
liveBench
leaderboard name
LiveBench Instruction Following
dimension
general
metric
Average score
rank
77
sample size
76
evaluated at
2024-12-10T19:31:50.200Z
source model id
gpt-4o-2024-05-13
Original result ↗

SWE-bench

58.2%

source id
sweBench
leaderboard name
SWE-bench Verified
dimension
coding
metric
Resolved tasks
rank
83
evaluated at
2025-04-10T00:00:00.000Z
Original result ↗

LMArena

1,157.746

source id
lmarena
leaderboard name
LMArena Vision Style Control Chinese
dimension
multimodal
metric
Arena rating
rank
107
sample size
1472
evaluated at
2026-11-09T00:00:00.000Z
source model id
GPT-4o (2024-05-13)
confidence lower
1119.2438468895662
confidence upper
1196.247676001647
Original result ↗

Published prices

1–2 / 2

Provider specifications & access

1–1 / 1

OpenAI GPT-4o · azure-openai v2024-05-13 ↗

source model id
azure-openai/gpt-4o@2024-05-13
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
custom
context length
131072
max output tokens
4096
architecture
input_modalities
  • text
  • image
  • audio
output_modalities
  • text
capabilities

    Safety assessments

    1–1 / 1

    Foundation Model Transparency Index 2025 ↗

    35 / 100

    source id
    stanfordFMTI
    canonical model id
    openai/gpt-4o-2024-05-13
    taxonomy
    transparency
    hazard category
    transparency
    hazard name
    Organization transparency
    policy area
    OpenAI
    assessment type
    organization-level disclosure
    metric
    disclosure_score
    result label
    35 / 100
    sample size
    100
    updated at
    2026-11-09T18:08:05.227Z

    Source specification observations

    1–6 / 6

    azureFoundry ↗

    field
    context_length
    value
    131072
    normalized value
    131072
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    openRouter ↗

    field
    context_length
    value
    128000
    normalized value
    128000
    scope
    source-model
    source id
    openRouter
    confidence
    1

    openRouter ↗

    field
    knowledge_cutoff
    reported field
    knowledge_cutoff
    value
    2023-10-31
    normalized value
    2023-10-31
    precision
    day
    scope
    source-model
    source id
    openRouter
    confidence
    1

    azureFoundry ↗

    field
    max_output_tokens
    value
    4096
    normalized value
    4096
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    openRouter ↗

    field
    max_output_tokens
    value
    4096
    normalized value
    4096
    scope
    source-model
    source id
    openRouter
    confidence
    1

    azureFoundry ↗

    field
    knowledge_cutoff
    reported field
    training_data_cutoff
    value
    Oct 2023
    normalized value
    2023-10
    precision
    month
    scope
    source-model
    source id
    azureFoundry
    confidence
    1

    Original sources

    1–10 / 10

    Microsoft Foundry · Model page ↗

    OpenRouter · Model page ↗

    Microsoft Foundry · Model documentation ↗

    GitHub · Catalog ↗

    GitHub · Catalog ↗

    LMArena · Catalog ↗

    Microsoft Foundry · Catalog ↗

    OpenRouter · Catalog ↗

    livebench.ai · Catalog ↗

    www.swebench.com · Catalog ↗

    Source links

    1–8 / 8

    Explore more models

    Models

    Loading
    EVIDENCE / COMPARISON

    Benchmarks

    Explore every benchmark

    Loading

    These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

    Install TensorTrace