DeepSeek: DeepSeek V3

DeepSeek-V3 is the latest model from the DeepSeek team, building upon the instruction following and coding abilities of the previous versions. Pre-trained on nearly 15 trillion tokens, the reported e…

deepseek/deepseek-chat-v3

DeepSeek-V3 is the latest model from the DeepSeek team, building upon the instruction following and coding abilities of the previous versions. Pre-trained on nearly 15 trillion tokens, the reported evaluations reveal that the model outperforms other open-source models and rivals leading closed-source models.

For model details, please visit [the DeepSeek-V3 repo](https://github.com/deepseek-ai/DeepSeek-V3) for more information, or see the [launch announcement](https://api-docs.deepseek.com/news/news1226).

Specifications & capabilities

Input
text
Output
text
Context window · tokens
163840
Knowledge cutoff
2024-07-31
Published model ID
DeepSeek-V3
Capabilities
structuredOutputs, toolCalling
Description source ↗

Evaluations

21–40 / 55

LLM Stats · GitHub archive

0.865

source id
llmStatsArchive
leaderboard name
C-Eval · Raw
dimension
general
metric
C-Eval · Raw
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
C-Eval
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
created_at
2025-07-19T19:56:11.917478+00:00
paper_link
https://arxiv.org/abs/2305.08322
updated_at
2025-07-19T19:56:11.917478+00:00
description
C-Eval is a comprehensive Chinese evaluation suite designed to assess advanced knowledge and reasoning abilities of foundation models in a Chinese context. It comprises 13,948 multiple-choice questions across 52 diverse disciplines spanning humanities, science, and engineering, with four difficulty levels: middle school, high school, college, and professional. The benchmark includes C-Eval Hard, a subset of very challenging subjects requiring advanced reasoning abilities.
benchmark_id
c-eval
multilingual
true
analysis_method
Exact Match
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
C-Eval · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.865

source id
llmStatsArchive
leaderboard name
C-Eval · Normalized (reported)
dimension
general
metric
C-Eval · Normalized (reported)
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
C-Eval
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
created_at
2025-07-19T19:56:11.917478+00:00
paper_link
https://arxiv.org/abs/2305.08322
updated_at
2025-07-19T19:56:11.917478+00:00
description
C-Eval is a comprehensive Chinese evaluation suite designed to assess advanced knowledge and reasoning abilities of foundation models in a Chinese context. It comprises 13,948 multiple-choice questions across 52 diverse disciplines spanning humanities, science, and engineering, with four difficulty levels: middle school, high school, college, and professional. The benchmark includes C-Eval Hard, a subset of very challenging subjects requiring advanced reasoning abilities.
benchmark_id
c-eval
multilingual
true
analysis_method
Exact Match
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
C-Eval · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.759

source id
llmStatsArchive
leaderboard name
MMLU-Pro · Normalized (reported)
dimension
general
metric
MMLU-Pro · Normalized (reported)
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU-Pro
language
en
modality
text
verified
false
max_score
1
categories
  • language
  • reasoning
  • math
  • general
created_at
2025-07-19T19:56:11.408351+00:00
paper_link
https://arxiv.org/abs/2406.01574
updated_at
2025-07-19T19:56:11.408351+00:00
description
A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
benchmark_id
mmlu-pro
multilingual
false
analysis_method
Exact Match
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU-Pro · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.759

source id
llmStatsArchive
leaderboard name
MMLU-Pro · Raw
dimension
general
metric
MMLU-Pro · Raw
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU-Pro
language
en
modality
text
verified
false
max_score
1
categories
  • language
  • reasoning
  • math
  • general
created_at
2025-07-19T19:56:11.408351+00:00
paper_link
https://arxiv.org/abs/2406.01574
updated_at
2025-07-19T19:56:11.408351+00:00
description
A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
benchmark_id
mmlu-pro
multilingual
false
analysis_method
Exact Match
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU-Pro · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.648

source id
llmStatsArchive
leaderboard name
CSimpleQA · Normalized (reported)
dimension
general
metric
CSimpleQA · Normalized (reported)
rank
3
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
CSimpleQA
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • language
created_at
2025-07-19T19:56:11.931358+00:00
paper_link
https://arxiv.org/abs/2411.07140
updated_at
2025-07-19T19:56:11.931358+00:00
description
Chinese SimpleQA is the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions. It contains 3,000 high-quality questions spanning 6 major topics with 99 diverse subtopics, designed to assess Chinese factual knowledge across humanities, science, engineering, culture, and society.
benchmark_id
csimpleqa
multilingual
true
analysis_method
Correct
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
CSimpleQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.648

source id
llmStatsArchive
leaderboard name
CSimpleQA · Raw
dimension
general
metric
CSimpleQA · Raw
rank
3
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
CSimpleQA
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • language
created_at
2025-07-19T19:56:11.931358+00:00
paper_link
https://arxiv.org/abs/2411.07140
updated_at
2025-07-19T19:56:11.931358+00:00
description
Chinese SimpleQA is the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions. It contains 3,000 high-quality questions spanning 6 major topics with 99 diverse subtopics, designed to assess Chinese factual knowledge across humanities, science, engineering, culture, and society.
benchmark_id
csimpleqa
multilingual
true
analysis_method
Correct
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
CSimpleQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.496

source id
llmStatsArchive
leaderboard name
Aider-Polyglot · Normalized (reported)
dimension
general
metric
Aider-Polyglot · Normalized (reported)
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
Aider-Polyglot
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • code
created_at
2025-09-05T00:00:00.000000+00:00
updated_at
2025-09-30T00:00:00.000000+00:00
description
A coding benchmark that evaluates LLMs on 225 challenging Exercism programming exercises across C++, Go, Java, JavaScript, Python, and Rust. Models receive two attempts to solve each problem, with test error feedback provided after the first attempt if it fails. The benchmark measures both initial problem-solving ability and capacity to edit code based on error feedback, providing an end-to-end evaluation of code generation and editing capabilities across multiple programming languages.
benchmark_id
aider-polyglot
multilingual
false
implementation_link
https://github.com/Aider-AI/polyglot-benchmark
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
Aider-Polyglot · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
5
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.496

source id
llmStatsArchive
leaderboard name
Aider-Polyglot · Raw
dimension
general
metric
Aider-Polyglot · Raw
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
Aider-Polyglot
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • code
created_at
2025-09-05T00:00:00.000000+00:00
updated_at
2025-09-30T00:00:00.000000+00:00
description
A coding benchmark that evaluates LLMs on 225 challenging Exercism programming exercises across C++, Go, Java, JavaScript, Python, and Rust. Models receive two attempts to solve each problem, with test error feedback provided after the first attempt if it fails. The benchmark measures both initial problem-solving ability and capacity to edit code based on error feedback, providing an end-to-end evaluation of code generation and editing capabilities across multiple programming languages.
benchmark_id
aider-polyglot
multilingual
false
implementation_link
https://github.com/Aider-AI/polyglot-benchmark
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
Aider-Polyglot · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
5
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.249

source id
llmStatsArchive
leaderboard name
SimpleQA · Raw
dimension
general
metric
SimpleQA · Raw
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
SimpleQA
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
created_at
2025-09-05T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2411.04368
updated_at
2025-09-05T00:00:00.000000+00:00
description
SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.
benchmark_id
simpleqa
multilingual
false
analysis_method
Correct
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
SimpleQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
4
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.249

source id
llmStatsArchive
leaderboard name
SimpleQA · Normalized (reported)
dimension
general
metric
SimpleQA · Normalized (reported)
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
SimpleQA
language
en
modality
text
verified
false
max_score
1
categories
  • general
  • reasoning
created_at
2025-09-05T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2411.04368
updated_at
2025-09-05T00:00:00.000000+00:00
description
SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.
benchmark_id
simpleqa
multilingual
false
analysis_method
Correct
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
SimpleQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
4
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.591

source id
llmStatsArchive
leaderboard name
GPQA · Normalized (reported)
dimension
general
metric
GPQA · Normalized (reported)
rank
5
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
created_at
2025-07-19T19:56:11.588605+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:11.588605+00:00
description
A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
benchmark_id
gpqa
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
5
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.591

source id
llmStatsArchive
leaderboard name
GPQA · Raw
dimension
general
metric
GPQA · Raw
rank
5
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
created_at
2025-07-19T19:56:11.588605+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:11.588605+00:00
description
A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
benchmark_id
gpqa
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
5
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.392

source id
llmStatsArchive
leaderboard name
AIME 2024 · Raw
dimension
general
metric
AIME 2024 · Raw
rank
5
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
AIME 2024
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.941652+00:00
paper_link
https://arxiv.org/html/2503.21380v2
updated_at
2025-09-30T00:00:00.000000+00:00
description
American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
benchmark_id
aime-2024
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
AIME 2024 · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
5
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.392

source id
llmStatsArchive
leaderboard name
AIME 2024 · Normalized (reported)
dimension
general
metric
AIME 2024 · Normalized (reported)
rank
5
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
AIME 2024
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.941652+00:00
paper_link
https://arxiv.org/html/2503.21380v2
updated_at
2025-09-30T00:00:00.000000+00:00
description
American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
benchmark_id
aime-2024
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
AIME 2024 · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
5
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.376

source id
llmStatsArchive
leaderboard name
LiveCodeBench · Raw
dimension
general
metric
LiveCodeBench · Raw
rank
7
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
LiveCodeBench
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
  • code
created_at
2025-07-19T19:56:13.292229+00:00
paper_link
https://arxiv.org/abs/2403.07974
updated_at
2025-07-19T19:56:13.292229+00:00
description
LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
benchmark_id
livecodebench
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
LiveCodeBench · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
9
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

LLM Stats · GitHub archive

0.376

source id
llmStatsArchive
leaderboard name
LiveCodeBench · Normalized (reported)
dimension
general
metric
LiveCodeBench · Normalized (reported)
rank
7
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
LiveCodeBench
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
  • code
created_at
2025-07-19T19:56:13.292229+00:00
paper_link
https://arxiv.org/abs/2403.07974
updated_at
2025-07-19T19:56:13.292229+00:00
description
LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
benchmark_id
livecodebench
multilingual
false
analysis_method
Pass@1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
LiveCodeBench · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/deepseek/models/deepseek-v3/benchmarks.json
archive_kind
llmStats
evidence_url
https://github.com/deepseek-ai/DeepSeek-V3
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
deepseek/deepseek-v3
metric_population
9
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
deepseek-v3
Original result ↗

HELM Capabilities

83.052%

source id
helm
leaderboard name
HELM WildBench
dimension
agentic
metric
wildbench_score_rescaled
rank
19
sample size
1000
source model id
deepseek-ai/deepseek-v3
Original result ↗

LiveBench

61.719%

source id
liveBench
leaderboard name
LiveBench Coding
dimension
coding
metric
Average score
rank
22
sample size
128
evaluated at
2025-02-06T18:31:24.389Z
source model id
deepseek-v3
Original result ↗

OpenRouter Benchmarks

40.38%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · StreamLake
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
25
sample size
149
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
StreamLake
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-02T15:44:48.231Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
149
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.40380291275167784
accuracy_std_dev
0.008840018715415086
cost_per_task_usd
0.07104265771812081
output_tokens_per_task
8668.979865771813
latency_seconds_per_task
235.54413422818794
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-chat-v3
source model id
deepseek/deepseek-chat-v3
Original result ↗

LiveBench

54.668%

source id
liveBench
leaderboard name
LiveBench Overall
dimension
general
metric
Average score
rank
26
sample size
418
evaluated at
2025-04-01T21:57:19.993Z
source model id
deepseek-v3
Original result ↗

Published prices

1–4 / 4

deepseek/deepseek-chat ↗

$0.26

Inference provider
openRouter
Platform
OpenRouter
Unit
input_tokens_per_million
Currency
USD
Billing conditions

deepseek/deepseek-chat ↗

$1.03

Inference provider
openRouter
Platform
OpenRouter
Unit
output_tokens_per_million
Currency
USD
Billing conditions

deepseek-ai/DeepSeek-V3 ↗

$0.25

Inference provider
siliconFlow
Platform
SiliconFlow
Unit
input_tokens_per_million
Currency
USD
Billing conditions
Billing label
Input tokens (per 1M)

deepseek-ai/DeepSeek-V3 ↗

$1.00

Inference provider
siliconFlow
Platform
SiliconFlow
Unit
output_tokens_per_million
Currency
USD
Billing conditions
Billing label
Output tokens (per 1M)

Provider specifications & access

1–3 / 3

DeepSeek-V3 ↗

source model id
deepseek-ai/DeepSeek-V3
hosting
siliconflow-hosted
platforms
  • SiliconFlow
max output tokens
163840
architecture
input_modalities
  • text
output_modalities
  • text
capabilities
  • structuredOutputs
  • toolCalling

deepseek-ai/DeepSeek-V3 ↗

source model id
publishers/deepseek-ai/models/deepseek-v3@deepseek-v3
hosting
vertex-self-deployed
platforms
  • Google Cloud Vertex AI
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    deepseek-ai/DeepSeek-V3 ↗

    source model id
    publishers/hf-deepseek-ai/models/deepseek-v3@001
    hosting
    vertex-self-deployed
    platforms
    • Google Cloud Vertex AI
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      Safety assessments

      1–1 / 1

      Foundation Model Transparency Index 2025 ↗

      32 / 100

      source id
      stanfordFMTI
      canonical model id
      deepseek/deepseek-chat-v3
      taxonomy
      transparency
      hazard category
      transparency
      hazard name
      Organization transparency
      policy area
      DeepSeek
      assessment type
      organization-level disclosure
      metric
      disclosure_score
      result label
      32 / 100
      sample size
      100
      updated at
      2026-11-09T18:08:05.227Z

      Source specification observations

      1–5 / 5

      openRouter ↗

      field
      context_length
      value
      163840
      normalized value
      163840
      scope
      source-model
      source id
      openRouter
      confidence
      1

      siliconFlow ↗

      field
      context_length
      value
      164K
      scope
      source-model
      source id
      siliconFlow
      confidence
      1

      openRouter ↗

      field
      knowledge_cutoff
      reported field
      knowledge_cutoff
      value
      2024-07-31
      normalized value
      2024-07-31
      precision
      day
      scope
      source-model
      source id
      openRouter
      confidence
      1

      openRouter ↗

      field
      max_output_tokens
      value
      16000
      normalized value
      16000
      scope
      source-model
      source id
      openRouter
      confidence
      1

      siliconFlow ↗

      field
      max_output_tokens
      value
      164K
      scope
      source-model
      source id
      siliconFlow
      confidence
      1

      Original sources

      1–20 / 20

      SiliconFlow · Model page ↗

      Hugging Face · Model repository ↗

      Google Cloud · Model page ↗

      Google Cloud · Model page ↗

      OpenRouter · Model page ↗

      SiliconFlow · Playground ↗

      Google Cloud · Model documentation ↗

      SiliconFlow · API documentation ↗

      arXiv · Model reference ↗

      Epoch AI · Catalog ↗

      GitHub · Catalog ↗

      GitHub · Catalog ↗

      Google Cloud · Catalog ↗

      Hugging Face · Catalog ↗

      Hugging Face · Catalog ↗

      OpenRouter · Catalog ↗

      OpenRouter · Catalog ↗

      SiliconFlow · Catalog ↗

      crfm.stanford.edu · Catalog ↗

      livebench.ai · Catalog ↗

      Source links

      1–11 / 11

      Hugging Face model card

      1–1 / 1

      Explore more models

      Models

      Loading
      EVIDENCE / COMPARISON

      Benchmarks

      Explore every benchmark

      Loading

      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

      Install TensorTrace