DeepSeek: DeepSeek V3
DeepSeek-V3 is the latest model from the DeepSeek team, building upon the instruction following and coding abilities of the previous versions. Pre-trained on nearly 15 trillion tokens, the reported e…
deepseek/deepseek-chat-v3
DeepSeek-V3 is the latest model from the DeepSeek team, building upon the instruction following and coding abilities of the previous versions. Pre-trained on nearly 15 trillion tokens, the reported evaluations reveal that the model outperforms other open-source models and rivals leading closed-source models.
For model details, please visit [the DeepSeek-V3 repo](https://github.com/deepseek-ai/DeepSeek-V3) for more information, or see the [launch announcement](https://api-docs.deepseek.com/news/news1226).
Specifications & capabilities
- Input
- text
- Output
- text
- Context window · tokens
- 163840
- Knowledge cutoff
- 2024-07-31
- Published model ID
- DeepSeek-V3
- Capabilities
- structuredOutputs, toolCalling
Description source ↗Evaluations
21–40 / 55
0.865
- source id
- llmStatsArchive
- leaderboard name
- C-Eval · Raw
- dimension
- general
- metric
- C-Eval · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- C-Eval
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.917478+00:00
- paper_link
- https://arxiv.org/abs/2305.08322
- updated_at
- 2025-07-19T19:56:11.917478+00:00
- description
- C-Eval is a comprehensive Chinese evaluation suite designed to assess advanced knowledge and reasoning abilities of foundation models in a Chinese context. It comprises 13,948 multiple-choice questions across 52 diverse disciplines spanning humanities, science, and engineering, with four difficulty levels: middle school, high school, college, and professional. The benchmark includes C-Eval Hard, a subset of very challenging subjects requiring advanced reasoning abilities.
- benchmark_id
- c-eval
- multilingual
- true
- analysis_method
- Exact Match
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- C-Eval · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.865
- source id
- llmStatsArchive
- leaderboard name
- C-Eval · Normalized (reported)
- dimension
- general
- metric
- C-Eval · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- C-Eval
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.917478+00:00
- paper_link
- https://arxiv.org/abs/2305.08322
- updated_at
- 2025-07-19T19:56:11.917478+00:00
- description
- C-Eval is a comprehensive Chinese evaluation suite designed to assess advanced knowledge and reasoning abilities of foundation models in a Chinese context. It comprises 13,948 multiple-choice questions across 52 diverse disciplines spanning humanities, science, and engineering, with four difficulty levels: middle school, high school, college, and professional. The benchmark includes C-Eval Hard, a subset of very challenging subjects requiring advanced reasoning abilities.
- benchmark_id
- c-eval
- multilingual
- true
- analysis_method
- Exact Match
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- C-Eval · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.759
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Pro · Normalized (reported)
- dimension
- general
- metric
- MMLU-Pro · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Pro
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:11.408351+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:11.408351+00:00
- description
- A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
- benchmark_id
- mmlu-pro
- multilingual
- false
- analysis_method
- Exact Match
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Pro · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.759
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Pro · Raw
- dimension
- general
- metric
- MMLU-Pro · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Pro
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:11.408351+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:11.408351+00:00
- description
- A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
- benchmark_id
- mmlu-pro
- multilingual
- false
- analysis_method
- Exact Match
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Pro · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.648
- source id
- llmStatsArchive
- leaderboard name
- CSimpleQA · Normalized (reported)
- dimension
- general
- metric
- CSimpleQA · Normalized (reported)
- rank
- 3
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- CSimpleQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.931358+00:00
- paper_link
- https://arxiv.org/abs/2411.07140
- updated_at
- 2025-07-19T19:56:11.931358+00:00
- description
- Chinese SimpleQA is the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions. It contains 3,000 high-quality questions spanning 6 major topics with 99 diverse subtopics, designed to assess Chinese factual knowledge across humanities, science, engineering, culture, and society.
- benchmark_id
- csimpleqa
- multilingual
- true
- analysis_method
- Correct
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- CSimpleQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.648
- source id
- llmStatsArchive
- leaderboard name
- CSimpleQA · Raw
- dimension
- general
- metric
- CSimpleQA · Raw
- rank
- 3
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- CSimpleQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.931358+00:00
- paper_link
- https://arxiv.org/abs/2411.07140
- updated_at
- 2025-07-19T19:56:11.931358+00:00
- description
- Chinese SimpleQA is the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions. It contains 3,000 high-quality questions spanning 6 major topics with 99 diverse subtopics, designed to assess Chinese factual knowledge across humanities, science, engineering, culture, and society.
- benchmark_id
- csimpleqa
- multilingual
- true
- analysis_method
- Correct
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- CSimpleQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.496
- source id
- llmStatsArchive
- leaderboard name
- Aider-Polyglot · Normalized (reported)
- dimension
- general
- metric
- Aider-Polyglot · Normalized (reported)
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Aider-Polyglot
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- updated_at
- 2025-09-30T00:00:00.000000+00:00
- description
- A coding benchmark that evaluates LLMs on 225 challenging Exercism programming exercises across C++, Go, Java, JavaScript, Python, and Rust. Models receive two attempts to solve each problem, with test error feedback provided after the first attempt if it fails. The benchmark measures both initial problem-solving ability and capacity to edit code based on error feedback, providing an end-to-end evaluation of code generation and editing capabilities across multiple programming languages.
- benchmark_id
- aider-polyglot
- multilingual
- false
- implementation_link
- https://github.com/Aider-AI/polyglot-benchmark
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Aider-Polyglot · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.496
- source id
- llmStatsArchive
- leaderboard name
- Aider-Polyglot · Raw
- dimension
- general
- metric
- Aider-Polyglot · Raw
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Aider-Polyglot
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- updated_at
- 2025-09-30T00:00:00.000000+00:00
- description
- A coding benchmark that evaluates LLMs on 225 challenging Exercism programming exercises across C++, Go, Java, JavaScript, Python, and Rust. Models receive two attempts to solve each problem, with test error feedback provided after the first attempt if it fails. The benchmark measures both initial problem-solving ability and capacity to edit code based on error feedback, providing an end-to-end evaluation of code generation and editing capabilities across multiple programming languages.
- benchmark_id
- aider-polyglot
- multilingual
- false
- implementation_link
- https://github.com/Aider-AI/polyglot-benchmark
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Aider-Polyglot · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.249
- source id
- llmStatsArchive
- leaderboard name
- SimpleQA · Raw
- dimension
- general
- metric
- SimpleQA · Raw
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- SimpleQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2411.04368
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.
- benchmark_id
- simpleqa
- multilingual
- false
- analysis_method
- Correct
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- SimpleQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.249
- source id
- llmStatsArchive
- leaderboard name
- SimpleQA · Normalized (reported)
- dimension
- general
- metric
- SimpleQA · Normalized (reported)
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- SimpleQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2411.04368
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- SimpleQA is a factuality benchmark developed by OpenAI that measures the short-form factual accuracy of large language models. The benchmark contains 4,326 short, fact-seeking questions that are adversarially collected and designed to have single, indisputable answers. Questions cover diverse topics from science and technology to entertainment, and the benchmark also measures model calibration by evaluating whether models know what they know.
- benchmark_id
- simpleqa
- multilingual
- false
- analysis_method
- Correct
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- SimpleQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.591
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Normalized (reported)
- dimension
- general
- metric
- GPQA · Normalized (reported)
- rank
- 5
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.591
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Raw
- dimension
- general
- metric
- GPQA · Raw
- rank
- 5
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.392
- source id
- llmStatsArchive
- leaderboard name
- AIME 2024 · Raw
- dimension
- general
- metric
- AIME 2024 · Raw
- rank
- 5
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AIME 2024
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.941652+00:00
- paper_link
- https://arxiv.org/html/2503.21380v2
- updated_at
- 2025-09-30T00:00:00.000000+00:00
- description
- American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
- benchmark_id
- aime-2024
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AIME 2024 · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.392
- source id
- llmStatsArchive
- leaderboard name
- AIME 2024 · Normalized (reported)
- dimension
- general
- metric
- AIME 2024 · Normalized (reported)
- rank
- 5
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AIME 2024
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.941652+00:00
- paper_link
- https://arxiv.org/html/2503.21380v2
- updated_at
- 2025-09-30T00:00:00.000000+00:00
- description
- American Invitational Mathematics Examination 2024, consisting of 30 challenging mathematical reasoning problems from AIME I and AIME II competitions. Each problem requires an integer answer between 0-999 and tests advanced mathematical reasoning across algebra, geometry, combinatorics, and number theory. Used as a benchmark for evaluating mathematical reasoning capabilities in large language models at Olympiad-level difficulty.
- benchmark_id
- aime-2024
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AIME 2024 · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.376
- source id
- llmStatsArchive
- leaderboard name
- LiveCodeBench · Raw
- dimension
- general
- metric
- LiveCodeBench · Raw
- rank
- 7
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- LiveCodeBench
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.292229+00:00
- paper_link
- https://arxiv.org/abs/2403.07974
- updated_at
- 2025-07-19T19:56:13.292229+00:00
- description
- LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
- benchmark_id
- livecodebench
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- LiveCodeBench · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 9
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗0.376
- source id
- llmStatsArchive
- leaderboard name
- LiveCodeBench · Normalized (reported)
- dimension
- general
- metric
- LiveCodeBench · Normalized (reported)
- rank
- 7
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- LiveCodeBench
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.292229+00:00
- paper_link
- https://arxiv.org/abs/2403.07974
- updated_at
- 2025-07-19T19:56:13.292229+00:00
- description
- LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
- benchmark_id
- livecodebench
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- LiveCodeBench · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/deepseek/models/deepseek-v3/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://github.com/deepseek-ai/DeepSeek-V3
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- deepseek/deepseek-v3
- metric_population
- 9
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- deepseek-v3
Original result ↗83.052%
- source id
- helm
- leaderboard name
- HELM WildBench
- dimension
- agentic
- metric
- wildbench_score_rescaled
- rank
- 19
- sample size
- 1000
- source model id
- deepseek-ai/deepseek-v3
Original result ↗61.719%
- source id
- liveBench
- leaderboard name
- LiveBench Coding
- dimension
- coding
- metric
- Average score
- rank
- 22
- sample size
- 128
- evaluated at
- 2025-02-06T18:31:24.389Z
- source model id
- deepseek-v3
Original result ↗40.38%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · StreamLake
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 25
- sample size
- 149
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- StreamLake
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-02T15:44:48.231Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 149
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.40380291275167784
- accuracy_std_dev
- 0.008840018715415086
- cost_per_task_usd
- 0.07104265771812081
- output_tokens_per_task
- 8668.979865771813
- latency_seconds_per_task
- 235.54413422818794
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 25
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-chat-v3
- source model id
- deepseek/deepseek-chat-v3
Original result ↗54.668%
- source id
- liveBench
- leaderboard name
- LiveBench Overall
- dimension
- general
- metric
- Average score
- rank
- 26
- sample size
- 418
- evaluated at
- 2025-04-01T21:57:19.993Z
- source model id
- deepseek-v3
Original result ↗Published prices
1–4 / 4
$0.26
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$1.03
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.25
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
$1.00
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
Provider specifications & access
1–3 / 3
- source model id
- deepseek-ai/DeepSeek-V3
- hosting
- siliconflow-hosted
- max output tokens
- 163840
- capabilities
- structuredOutputs
- toolCalling
- source model id
- publishers/deepseek-ai/models/deepseek-v3@deepseek-v3
- hosting
- vertex-self-deployed
- source model id
- publishers/hf-deepseek-ai/models/deepseek-v3@001
- hosting
- vertex-self-deployed
Safety assessments
1–1 / 1
32 / 100
- source id
- stanfordFMTI
- canonical model id
- deepseek/deepseek-chat-v3
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- DeepSeek
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 32 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–5 / 5
- field
- context_length
- value
- 163840
- normalized value
- 163840
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 164K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-07-31
- normalized value
- 2024-07-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 16000
- normalized value
- 16000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 164K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
Hugging Face model card
1–1 / 1
- downloads
- 1453825
- likes
- 4462
- pipeline tag
- text-generation
- library name
- transformers
- License
Explore more models