DeepSeek: DeepSeek V4 Pro 0423
DeepSeek V4 Pro is a large-scale Mixture-of-Experts model from DeepSeek with 1.6T total parameters and 49B activated parameters, supporting a 1M-token context window. It is designed for advanced reas…
deepseek/deepseek-v4-pro-20260423
DeepSeek V4 Pro is a large-scale Mixture-of-Experts model from DeepSeek with 1.6T total parameters and 49B activated parameters, supporting a 1M-token context window. It is designed for advanced reasoning, coding, and long-horizon agent workflows, with strong performance across knowledge, math, and software engineering benchmarks.
Built on the same architecture as DeepSeek V4 Flash, it introduces a hybrid attention system for efficient long-context processing. Reasoning efforts high and xhigh are supported; xhigh maps to max reasoning. It is well suited for complex workloads such as full-codebase analysis, multi-step automation, and large-scale information synthesis, where both capability and efficiency are critical
Specifications & capabilities
- Input
- text
- Output
- text
- Context window · tokens
- 1048576
- Knowledge cutoff
- 2025-05
- Published model ID
- DeepSeek-V4-Pro
- Capabilities
- audioInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling
Description source ↗Evaluations
41–60 / 67
87.996%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Parasail
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 9
- sample size
- 933
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Parasail
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-17T16:07:16.258Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 933
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8799569142550911
- accuracy_std_dev
- 0.006792606115475872
- cost_per_task_usd
- 0.11584198606645231
- output_tokens_per_task
- 33104.76848874598
- latency_seconds_per_task
- 484.61605144694533
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 36
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗77.5%
- source id
- openCompass
- leaderboard name
- Average
- dimension
- general
- metric
- Average
- rank
- 9
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Average
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗76.095%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Novita
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 9
- sample size
- 350
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Novita
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T19:46:00.618Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 350
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7609524285714285
- accuracy_std_dev
- 0.016106319095818472
- cost_per_task_usd
- 0.12609460857142857
- output_tokens_per_task
- 20587.46
- latency_seconds_per_task
- 262.4076285714286
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 52
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗75.786%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Alibaba
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 9
- sample size
- 700
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 14
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Alibaba
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T19:54:53.711Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 700
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7578571428571429
- accuracy_std_dev
- 0.029048460957159017
- cost_per_task_usd
- 0.09093985428571429
- output_tokens_per_task
- 11033.007142857143
- latency_seconds_per_task
- 189.68876714285713
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 29
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗75.126%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Parasail
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 9
- sample size
- 199
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 4
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Parasail
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-10T19:17:08.161Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 199
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.751256015075377
- accuracy_std_dev
- 0.006852721115549665
- cost_per_task_usd
- 0.07795362814070352
- output_tokens_per_task
- 10368.422110552763
- latency_seconds_per_task
- 228.60350753768844
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 28
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗72.667%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Nebius
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 10
- sample size
- 50
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 1
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Nebius
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-03T22:19:06.647Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 50
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.726667
- cost_per_task_usd
- 0.07819403999999999
- output_tokens_per_task
- 10299.58
- latency_seconds_per_task
- 528.0142
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 23
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗87.1%
- source id
- openCompass
- leaderboard name
- Knowledge_Social_Science
- dimension
- general
- metric
- Knowledge_Social_Science
- rank
- 11
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- KnowledgeTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Knowledge_Social_Science
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗86.532%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · SiliconFlow
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 11
- sample size
- 1386
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- SiliconFlow
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:05:14.701Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1386
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8653197142857143
- accuracy_std_dev
- 0.018961344600561016
- cost_per_task_usd
- 0.16908653823953823
- output_tokens_per_task
- 53671.56421356421
- latency_seconds_per_task
- 1092.5425418470418
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 34
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗71.2%
- source id
- openCompass
- leaderboard name
- Average
- dimension
- general
- metric
- Average
- rank
- 12
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- MathTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Average
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗71.2%
- source id
- openCompass
- leaderboard name
- Math
- dimension
- general
- metric
- Math
- rank
- 12
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Math
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗62.5%
- source id
- openCompass
- leaderboard name
- Math_Competition
- dimension
- general
- metric
- Math_Competition
- rank
- 12
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- MathTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Math_Competition
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗55.5%
- source id
- openCompass
- leaderboard name
- Average
- dimension
- general
- metric
- Average
- rank
- 13
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- ReasonTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Average
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗55.5%
- source id
- openCompass
- leaderboard name
- Reasoning
- dimension
- general
- metric
- Reasoning
- rank
- 13
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Reasoning
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗41.3%
- source id
- openCompass
- leaderboard name
- Academic_Reasoning
- dimension
- general
- metric
- Academic_Reasoning
- rank
- 13
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- ReasonTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Academic_Reasoning
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗85.931%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DigitalOcean
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 14
- sample size
- 1386
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DigitalOcean
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:05:14.716Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1386
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8593071428571429
- accuracy_std_dev
- 0.07000414295174368
- cost_per_task_usd
- 0.07085715151515151
- output_tokens_per_task
- 34951.957431457435
- latency_seconds_per_task
- 1036.918955988456
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 23
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗81.986%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · BaseTen
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 15
- sample size
- 396
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- BaseTen
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-08-29T18:58:14.477Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 396
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.819865
- accuracy_std_dev
- 0.006733999999999962
- cost_per_task_usd
- 0.08986796717171718
- output_tokens_per_task
- 25486.853535353537
- latency_seconds_per_task
- 195.25559343434344
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 16
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗76.857%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · auto-routing
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 15
- sample size
- 350
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- auto-routing
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T20:15:14.788Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 350
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7685714285714286
- accuracy_std_dev
- 0.020995671729853154
- cost_per_task_usd
- 0.062093528571428566
- output_tokens_per_task
- 13057.314285714287
- latency_seconds_per_task
- 230.92066857142856
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 128
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗69.6%
- source id
- openCompass
- leaderboard name
- Common_Reasoning
- dimension
- general
- metric
- Common_Reasoning
- rank
- 15
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- ReasonTable
- catalog
- closed
- snapshot
- month
- 26-07
- fileName
- assets/llm-rank/llm-data-v2.26-07.20260824.json
- updateTime
- 2026-08-24
- columnFileName
- assets/llm-rank/llm-column-v2.26-07.20260824.json
- metric_kind
- Common_Reasoning
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 1-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 15
- benchmark_evaluation
- true
- source model id
- DeepSeek-V4-Pro
Original result ↗87.542%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Azure
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 22
- sample size
- 1386
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Azure
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:30:53.502Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1386
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8754208571428571
- accuracy_std_dev
- 0.009220961422271589
- cost_per_task_usd
- 0.1371511341991342
- output_tokens_per_task
- 35614.08585858586
- latency_seconds_per_task
- 439.8560331890332
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 43
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗69.18%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Venice
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 32
- sample size
- 1386
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Venice
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:56:31.817Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1386
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.6917989999999999
- accuracy_std_dev
- 0.053895225463061976
- cost_per_task_usd
- 0.14352880086580086
- output_tokens_per_task
- 43119.867965367965
- latency_seconds_per_task
- 700.9271991341992
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 39
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-pro-20260423
- source model id
- deepseek/deepseek-v4-pro-20260423
Original result ↗Published prices
1–20 / 36
$0.15
- Inference provider
- cloudflareWorkersAI
- Platform
- Fireworks AI
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$1.74
- Inference provider
- cloudflareWorkersAI
- Platform
- Fireworks AI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$3.48
- Inference provider
- cloudflareWorkersAI
- Platform
- Fireworks AI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
- Pricing via
- Cloudflare AI Gateway
$0.25
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.90
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$1.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.14
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing label
- Cached input tokens (per 1M)
$1.50
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- Input tokens (per 1M)
$3.14
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- Output tokens (per 1M)
$0.20
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4-pro
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$2.40
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4-pro
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$4.80
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4-pro
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.15
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | deepseek/deepseek-v4-pro
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$1.74
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | deepseek/deepseek-v4-pro
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$3.48
- Inference provider
- vercelAIGatewayCatalog
- Platform
- azure
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- azure | deepseek/deepseek-v4-pro
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.13
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- baseten | deepseek/deepseek-v4-pro
- model_type
- language
- quantization
- fp4
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.13
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- scope
- zone
- geo_region
- us
- endpoint
- baseten | deepseek/deepseek-v4-pro
- model_type
- language
- quantization
- fp4
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$1.32
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- baseten | deepseek/deepseek-v4-pro
- model_type
- language
- quantization
- fp4
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$1.32
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- scope
- zone
- geo_region
- us
- endpoint
- baseten | deepseek/deepseek-v4-pro
- model_type
- language
- quantization
- fp4
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$3.96
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- baseten | deepseek/deepseek-v4-pro
- model_type
- language
- quantization
- fp4
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–4 / 4
- source model id
- deepseek-ai/DeepSeek-V4-Pro
- hosting
- siliconflow-hosted
- max output tokens
- 393216
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- source model id
- deepseek/deepseek-v4-pro
- hosting
- third-party
- context length
- 131072
- source model id
- deepseek/deepseek-v4-pro
- hosting
- vercel-gateway
- context length
- 1048600
- max output tokens
- 1048600
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- source model id
- publishers/deepseek-ai/models/deepseek-v4@deepseek-v4-pro
- hosting
- vertex-self-deployed
Safety assessments
1–1 / 1
32 / 100
- source id
- stanfordFMTI
- canonical model id
- deepseek/deepseek-v4-pro-20260423
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- DeepSeek
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 32 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–8 / 8
- field
- context_length
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- cloudflareWorkersAI
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1049K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- context_length
- value
- 1048600
- normalized value
- 1048600
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-05
- normalized value
- 2025-05
- precision
- month
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 393216
- normalized value
- 393216
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 393K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- max_output_tokens
- value
- 1048600
- normalized value
- 1048600
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
Explore more models