DeepSeek: DeepSeek V4 Flash 0731

DeepSeek V4 Flash 0731 is a sparse mixture-of-experts model from DeepSeek, with 13B active parameters out of 284B total. This re-post-trained revision is suited for coding, reasoning, and agent workf…

deepseek/deepseek-v4-flash-20260731

DeepSeek V4 Flash 0731 is a sparse mixture-of-experts model from DeepSeek, with 13B active parameters out of 284B total. This re-post-trained revision is suited for coding, reasoning, and agent workflows. This is the GA release of DeepSeek V4 Flash.

Specifications & capabilities

Input
text
Output
text
Context window · tokens
1048576
Knowledge cutoff
2025-05
Published model ID
DeepSeek-V4-Flash-0731
Capabilities
promptCaching, reasoning, structuredOutputs, toolCalling
Description source ↗

Evaluations

61–77 / 77

OpenRouter Benchmarks

73.529%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Relace
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
10
sample size
850
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
17
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Relace
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-02T18:06:38.075Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
850
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7352940588235294
accuracy_std_dev
0.03872239054432369
cost_per_task_usd
0.009413557647058823
output_tokens_per_task
15440.829411764706
latency_seconds_per_task
215.23848941176473
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
13
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

75.417%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Alibaba
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
11
sample size
640
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
13
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:41:35.064Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
640
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.75416675
accuracy_std_dev
0.019096569464219906
cost_per_task_usd
0.0324113640625
output_tokens_per_task
15840.8265625
latency_seconds_per_task
163.294796875
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

74.867%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Parasail
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
11
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Parasail
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:32:12.444Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7486667
accuracy_std_dev
0.02733343658616678
cost_per_task_usd
0.018688805999999995
output_tokens_per_task
15779.308
latency_seconds_per_task
207.184068
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
28
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

72%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Fireworks
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
11
sample size
100
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Fireworks
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-09-19T08:13:24.062Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
100
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.72
accuracy_std_dev
0.030000000000000027
cost_per_task_usd
0.026063740000000002
output_tokens_per_task
12968.38
latency_seconds_per_task
127.59792
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
14
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

87.429%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Together
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
13
sample size
1770
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
9
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Together
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T08:23:51.334Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1770
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8742938723163841
accuracy_std_dev
0.008148155125586256
cost_per_task_usd
0.009214897175141243
output_tokens_per_task
32530.198305084745
latency_seconds_per_task
375.89575536723163
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

85.387%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Nebius
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
13
sample size
990
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-12T15:01:54.124Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
990
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8538717999999998
accuracy_std_dev
0.04946631113151657
cost_per_task_usd
0.005526387878787878
output_tokens_per_task
19384.131313131315
latency_seconds_per_task
190.0744404040404
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

75.533%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Novita
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
13
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Novita
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:45:56.690Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7553333999999999
accuracy_std_dev
0.017650954020675496
cost_per_task_usd
0.040253192
output_tokens_per_task
16547.004
latency_seconds_per_task
172.625032
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
52
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

70.19%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Nebius
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
16
sample size
246
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-12T15:21:11.557Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
246
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7018973089430894
accuracy_std_dev
0.04349030049378826
cost_per_task_usd
0.029020682926829265
output_tokens_per_task
21482.62601626016
latency_seconds_per_task
391.9913536585366
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

67.253%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · DigitalOcean
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
17
sample size
398
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
8
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
DigitalOcean
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:55:35.800Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
398
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.6725294120603015
accuracy_std_dev
0.11099160244860588
cost_per_task_usd
0.03270493216080402
output_tokens_per_task
69160.1582914573
latency_seconds_per_task
1961.0637788944723
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
20
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

75.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · DeepInfra
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
18
sample size
550
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T13:30:46.598Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
550
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7533332727272728
accuracy_std_dev
0.023952899912084696
cost_per_task_usd
0.007874056363636363
output_tokens_per_task
15094.398181818182
latency_seconds_per_task
231.90571636363637
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
63
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

73.407%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
44
sample size
450
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
9
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T11:40:30.784Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
450
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7340741111111111
accuracy_std_dev
0.05052255713253118
cost_per_task_usd
0.009341564444444445
output_tokens_per_task
15978.137777777778
latency_seconds_per_task
247.29254444444445
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

OpenRouter Benchmarks

86.683%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
48
sample size
1980
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:17:29.008Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1980
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8668349000000002
accuracy_std_dev
0.02737903360401897
cost_per_task_usd
0.008139828787878788
output_tokens_per_task
26892.52676767677
latency_seconds_per_task
416.33891818181814
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-v4-flash-20260731
source model id
deepseek/deepseek-v4-flash-20260731
Original result ↗

Kaggle Benchmarks

1,472.3

source id
kaggleBenchmarks
leaderboard name
Code2Video Bench · ELO-Ranking
dimension
coding
metric
Code2Video Bench · ELO-Ranking
rank
13
aggregation scope
kaggle
true
status
published
license
Apache-2.0
direction
higherIsBetter
conditions
task
id
582971
name
ELO-Ranking
version
1
sort_order
DESCENDING
display_type
floatCounts
benchmark
id
7434
name
Code2Video Bench
type
INDIVIDUAL
owner
heygen
version
1
categories
  • coding
  • multimodal capabilities
version_id
7450
maintainer_tier
Community-published
candidate
candidate_class
modelVersion
record_kind
aggregate
attribution
Kaggle · heygen · Code2Video Bench
license_url
https://www.apache.org/licenses/LICENSE-2.0
metric_kind
Code2Video Bench · ELO-Ranking
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
7450/leaderboard.json
source_page
https://www.kaggle.com/benchmarks/heygen/code2video/versions/1
measurements
  • name
    ELO-Ranking
    unit
    score
    scale
    1
    metric
    Code2Video Bench · ELO-Ranking
    result
    resultCase
    numericResult
    booleanResult
    false
    numericResult
    value
    1472.3
    standardDeviation
    0
    confidenceInterval
    10.6
    hasStandardDeviation
    false
    hasConfidenceInterval
    true
    confidenceIntervalNullable
    10.6
    hasUnevenConfidenceInterval
    false
    taskVersionId
    0
    evaluationDate
    2026-10-22T04:05:24.6596936Z
    hasBooleanResult
    false
    hasNumericResult
    true
    hasTaskVersionId
    false
    hasEvaluationDate
    true
    numericResultNullable
    value
    1472.3
    standardDeviation
    0
    confidenceInterval
    10.6
    hasStandardDeviation
    false
    hasConfidenceInterval
    true
    confidenceIntervalNullable
    10.6
    hasUnevenConfidenceInterval
    false
    hasNumericResultPublic
    false
    customAdditionalResults
      hasNumericResultPrivate
      false
      task_id
      582971
      sort_order
      DESCENDING
      source_name
      ELO-Ranking
      display_type
      floatCounts
      task_version
      1
      task_display_type
      FLOAT_COUNTS
      normalization_status
      normalized
    published_at
    2026-09-21T14:05:02.139768400Z
    candidate_class
    modelVersion
    source_revision
    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
    publisher_anchor
    deepseek-ai/deepseek-v4-flash-0731
    metric_population
    19
    benchmark_evaluation
    true
    task_metadata_source
    benchmarkVersion
    candidate_metadata_source
    benchmarkVersionMappings
    evaluated at
    2026-10-22T04:05:24.659Z
    source model id
    Deepseek V4 Flash 0731
    confidence lower
    1461.7
    confidence upper
    1482.8999999999999
    Original result ↗

    Kaggle Benchmarks

    9.4%

    source id
    kaggleBenchmarks
    leaderboard name
    Code2Video Bench · Raw Score
    dimension
    coding
    metric
    Code2Video Bench · Raw Score
    rank
    13
    aggregation scope
    kaggle
    true
    status
    published
    license
    Apache-2.0
    direction
    higherIsBetter
    conditions
    task
    id
    583283
    name
    Raw Score
    version
    1
    sort_order
    DESCENDING
    display_type
    percentages
    benchmark
    id
    7434
    name
    Code2Video Bench
    type
    INDIVIDUAL
    owner
    heygen
    version
    1
    categories
    • coding
    • multimodal capabilities
    version_id
    7450
    maintainer_tier
    Community-published
    candidate
    candidate_class
    modelVersion
    record_kind
    aggregate
    attribution
    Kaggle · heygen · Code2Video Bench
    license_url
    https://www.apache.org/licenses/LICENSE-2.0
    metric_kind
    Code2Video Bench · Raw Score
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    7450/leaderboard.json
    source_page
    https://www.kaggle.com/benchmarks/heygen/code2video/versions/1
    measurements
    • name
      Raw Score
      unit
      percent
      scale
      1
      metric
      Code2Video Bench · Raw Score
      result
      resultCase
      numericResult
      booleanResult
      false
      numericResult
      value
      0.094
      standardDeviation
      0
      confidenceInterval
      0
      hasStandardDeviation
      false
      hasConfidenceInterval
      false
      hasUnevenConfidenceInterval
      false
      taskVersionId
      0
      evaluationDate
      2026-10-22T04:05:24.8136758Z
      hasBooleanResult
      false
      hasNumericResult
      true
      hasTaskVersionId
      false
      hasEvaluationDate
      true
      numericResultNullable
      value
      0.094
      standardDeviation
      0
      confidenceInterval
      0
      hasStandardDeviation
      false
      hasConfidenceInterval
      false
      hasUnevenConfidenceInterval
      false
      hasNumericResultPublic
      false
      customAdditionalResults
        hasNumericResultPrivate
        false
        task_id
        583283
        sort_order
        DESCENDING
        source_name
        Raw Score
        display_type
        percentages
        task_version
        1
        task_display_type
        PERCENTAGES
        normalization_status
        normalized
      published_at
      2026-09-21T14:05:02.139768400Z
      candidate_class
      modelVersion
      source_revision
      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
      publisher_anchor
      deepseek-ai/deepseek-v4-flash-0731
      metric_population
      19
      benchmark_evaluation
      true
      task_metadata_source
      benchmarkVersion
      candidate_metadata_source
      benchmarkVersionMappings
      evaluated at
      2026-10-22T04:05:24.813Z
      source model id
      Deepseek V4 Flash 0731
      Original result ↗

      Published prices

      1–20 / 68

      ~deepseek/deepseek-v4-flash-latest ↗

      $0.04

      Inference provider
      openRouter
      Platform
      OpenRouter
      Unit
      cache_read_tokens_per_million
      Currency
      USD
      Billing conditions
      alias_target
      name
      DeepSeek: DeepSeek V4 Flash 0731
      slug
      deepseek/deepseek-v4-flash-0731
      routing_alias
      true

      ~deepseek/deepseek-v4-flash-latest ↗

      $0.04

      Inference provider
      openRouter
      Platform
      OpenRouter
      Unit
      input_tokens_per_million
      Currency
      USD
      Billing conditions
      alias_target
      name
      DeepSeek: DeepSeek V4 Flash 0731
      slug
      deepseek/deepseek-v4-flash-0731
      routing_alias
      true

      ~deepseek/deepseek-v4-flash-latest ↗

      $0.30

      Inference provider
      openRouter
      Platform
      OpenRouter
      Unit
      output_tokens_per_million
      Currency
      USD
      Billing conditions
      alias_target
      name
      DeepSeek: DeepSeek V4 Flash 0731
      slug
      deepseek/deepseek-v4-flash-0731
      routing_alias
      true

      deepseek-ai/DeepSeek-V4-Flash-0731 ↗

      $0.01

      Inference provider
      siliconFlow
      Platform
      SiliconFlow
      Unit
      cache_read_tokens_per_million
      Currency
      USD
      Billing conditions
      tier
      peak:off:14-24
      label
      14:00 – 24:00(UTC)
      Billing label
      Cached input tokens (per 1M)

      deepseek-ai/DeepSeek-V4-Flash-0731 ↗

      $0.03

      Inference provider
      siliconFlow
      Platform
      SiliconFlow
      Unit
      cache_read_tokens_per_million
      Currency
      USD
      Billing conditions
      tier
      peak:on:0-14
      label
      00:00 – 14:00(UTC)
      Billing label
      Cached input tokens (per 1M)

      deepseek-ai/DeepSeek-V4-Flash-0731 ↗

      $0.22

      Inference provider
      siliconFlow
      Platform
      SiliconFlow
      Unit
      input_tokens_per_million
      Currency
      USD
      Billing conditions
      tier
      peak:off:14-24
      label
      14:00 – 24:00(UTC)
      Billing label
      Input tokens (per 1M)

      deepseek-ai/DeepSeek-V4-Flash-0731 ↗

      $0.44

      Inference provider
      siliconFlow
      Platform
      SiliconFlow
      Unit
      input_tokens_per_million
      Currency
      USD
      Billing conditions
      tier
      peak:on:0-14
      label
      00:00 – 14:00(UTC)
      Billing label
      Input tokens (per 1M)

      deepseek-ai/DeepSeek-V4-Flash-0731 ↗

      $0.66

      Inference provider
      siliconFlow
      Platform
      SiliconFlow
      Unit
      output_tokens_per_million
      Currency
      USD
      Billing conditions
      tier
      peak:off:14-24
      label
      14:00 – 24:00(UTC)
      Billing label
      Output tokens (per 1M)

      deepseek-ai/DeepSeek-V4-Flash-0731 ↗

      $1.32

      Inference provider
      siliconFlow
      Platform
      SiliconFlow
      Unit
      output_tokens_per_million
      Currency
      USD
      Billing conditions
      tier
      peak:on:0-14
      label
      00:00 – 14:00(UTC)
      Billing label
      Output tokens (per 1M)

      deepseek/deepseek-v4-flash-0731 ↗

      $0.02

      Inference provider
      vercelAIGatewayCatalog
      Platform
      alibaba
      Unit
      cache_read_tokens_per_million
      Currency
      USD
      Billing conditions
      endpoint
      alibaba | deepseek/deepseek-v4-flash-0731
      model_type
      language
      peak_pricing
      windows
      • end_minute_utc
        840
        start_minute_utc
        0
      multiplier
      2
      billing_scope
      via Vercel AI Gateway
      Billing label
      Cached input / 1M
      Pricing via
      Vercel AI Gateway

      deepseek/deepseek-v4-flash-0731 ↗

      $0.22

      Inference provider
      vercelAIGatewayCatalog
      Platform
      alibaba
      Unit
      input_tokens_per_million
      Currency
      USD
      Billing conditions
      endpoint
      alibaba | deepseek/deepseek-v4-flash-0731
      model_type
      language
      peak_pricing
      windows
      • end_minute_utc
        840
        start_minute_utc
        0
      multiplier
      2
      billing_scope
      via Vercel AI Gateway
      Billing label
      Input tokens / 1M
      Pricing via
      Vercel AI Gateway

      deepseek/deepseek-v4-flash-0731 ↗

      $0.66

      Inference provider
      vercelAIGatewayCatalog
      Platform
      alibaba
      Unit
      output_tokens_per_million
      Currency
      USD
      Billing conditions
      endpoint
      alibaba | deepseek/deepseek-v4-flash-0731
      model_type
      language
      peak_pricing
      windows
      • end_minute_utc
        840
        start_minute_utc
        0
      multiplier
      2
      billing_scope
      via Vercel AI Gateway
      Billing label
      Output tokens / 1M
      Pricing via
      Vercel AI Gateway

      Provider specifications & access

      1–9 / 9

      deepseek-ai/DeepSeek-V4-Flash-0731 ↗

      source model id
      azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@1
      hosting
      foundry-managed-compute
      platforms
      • Microsoft Foundry
      license
      MIT
      context length
      1048576
      architecture
      input_modalities
      • text
      output_modalities
      • text
      capabilities

        deepseek-ai/DeepSeek-V4-Flash-0731 ↗

        source model id
        azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@2
        hosting
        foundry-managed-compute
        platforms
        • Microsoft Foundry
        license
        MIT
        context length
        1048576
        architecture
        input_modalities
        • text
        output_modalities
        • text
        capabilities

          deepseek-ai/DeepSeek-V4-Flash-0731 ↗

          source model id
          azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@3
          hosting
          foundry-managed-compute
          platforms
          • Microsoft Foundry
          license
          MIT
          context length
          1048576
          architecture
          input_modalities
          • text
          output_modalities
          • text
          capabilities

            deepseek-ai/DeepSeek-V4-Flash-0731 ↗

            source model id
            azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@4
            hosting
            foundry-managed-compute
            platforms
            • Microsoft Foundry
            license
            MIT
            context length
            1048576
            architecture
            input_modalities
            • text
            output_modalities
            • text
            capabilities

              deepseek-ai/DeepSeek-V4-Flash-0731 ↗

              source model id
              azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@5
              hosting
              foundry-managed-compute
              platforms
              • Microsoft Foundry
              license
              MIT
              context length
              1048576
              architecture
              input_modalities
              • text
              output_modalities
              • text
              capabilities

                deepseek-v4-flash-0731 ↗

                source model id
                @cf/deepseek-ai/deepseek-v4-flash-0731
                hosting
                cloudflare-hosted
                platforms
                • Cloudflare Workers AI
                context length
                1048576
                architecture
                input_modalities
                • text
                output_modalities
                • text
                capabilities
                • toolCalling

                DeepSeek-V4-Flash-0731 ↗

                source model id
                deepseek-ai/DeepSeek-V4-Flash-0731
                hosting
                siliconflow-hosted
                platforms
                • SiliconFlow
                max output tokens
                393216
                architecture
                input_modalities
                • text
                output_modalities
                • text
                capabilities
                • promptCaching
                • reasoning
                • structuredOutputs
                • toolCalling

                DeepSeek V4 Flash 0731 ↗

                source model id
                deepseek/deepseek-v4-flash-0731
                hosting
                vercel-gateway
                platforms
                • Vercel AI Gateway
                context length
                1000000
                max output tokens
                384000
                architecture
                input_modalities
                • text
                output_modalities
                • text
                capabilities
                • promptCaching
                • reasoning
                • structuredOutputs
                • toolCalling
                regions
                • us

                deepseek-ai/DeepSeek-V4-Flash-0731 ↗

                source model id
                publishers/deepseek-ai/models/deepseek-v4@deepseek-v4-flash-0731
                hosting
                vertex-self-deployed
                platforms
                • Google Cloud Vertex AI
                architecture
                input_modalities
                • text
                output_modalities
                • text
                capabilities

                  Safety assessments

                  1–1 / 1

                  Foundation Model Transparency Index 2025 ↗

                  32 / 100

                  source id
                  stanfordFMTI
                  canonical model id
                  deepseek/deepseek-v4-flash-20260731
                  taxonomy
                  transparency
                  hazard category
                  transparency
                  hazard name
                  Organization transparency
                  policy area
                  DeepSeek
                  assessment type
                  organization-level disclosure
                  metric
                  disclosure_score
                  result label
                  32 / 100
                  sample size
                  100
                  updated at
                  2026-11-09T18:08:05.227Z

                  Source specification observations

                  1–16 / 16

                  azureFoundry ↗

                  field
                  context_length
                  value
                  1048576
                  normalized value
                  1048576
                  scope
                  source-model
                  source id
                  azureFoundry
                  confidence
                  1

                  azureFoundry ↗

                  field
                  context_length
                  value
                  1048576
                  normalized value
                  1048576
                  scope
                  source-model
                  source id
                  azureFoundry
                  confidence
                  1

                  azureFoundry ↗

                  field
                  context_length
                  value
                  1048576
                  normalized value
                  1048576
                  scope
                  source-model
                  source id
                  azureFoundry
                  confidence
                  1

                  azureFoundry ↗

                  field
                  context_length
                  value
                  1048576
                  normalized value
                  1048576
                  scope
                  source-model
                  source id
                  azureFoundry
                  confidence
                  1

                  azureFoundry ↗

                  field
                  context_length
                  value
                  1048576
                  normalized value
                  1048576
                  scope
                  source-model
                  source id
                  azureFoundry
                  confidence
                  1

                  openRouter ↗

                  field
                  context_length
                  value
                  1048576
                  normalized value
                  1048576
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  openRouter ↗

                  field
                  context_length
                  value
                  1048576
                  normalized value
                  1048576
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  openRouter ↗

                  field
                  context_length
                  value
                  1048576
                  normalized value
                  1048576
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  siliconFlow ↗

                  field
                  context_length
                  value
                  1049K
                  scope
                  source-model
                  source id
                  siliconFlow
                  confidence
                  1

                  vercelAIGatewayCatalog ↗

                  field
                  context_length
                  value
                  1000000
                  normalized value
                  1000000
                  scope
                  source-model
                  source id
                  vercelAIGatewayCatalog
                  confidence
                  1

                  vercelAIGatewayCatalog ↗

                  field
                  knowledge_cutoff
                  reported field
                  knowledge_cutoff
                  value
                  2025-05
                  normalized value
                  2025-05
                  precision
                  month
                  scope
                  source-model
                  source id
                  vercelAIGatewayCatalog
                  confidence
                  1

                  openRouter ↗

                  field
                  max_output_tokens
                  value
                  384000
                  normalized value
                  384000
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  openRouter ↗

                  field
                  max_output_tokens
                  value
                  384000
                  normalized value
                  384000
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  openRouter ↗

                  field
                  max_output_tokens
                  value
                  943718
                  normalized value
                  943718
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  siliconFlow ↗

                  field
                  max_output_tokens
                  value
                  393K
                  scope
                  source-model
                  source id
                  siliconFlow
                  confidence
                  1

                  vercelAIGatewayCatalog ↗

                  field
                  max_output_tokens
                  value
                  384000
                  normalized value
                  384000
                  scope
                  source-model
                  source id
                  vercelAIGatewayCatalog
                  confidence
                  1

                  Original sources

                  1–20 / 30

                  SiliconFlow · Model page ↗

                  Hugging Face · Model reference ↗

                  Cloudflare AI · Model page ↗

                  Google Cloud · Model page ↗

                  Microsoft Foundry · Model page ↗

                  Microsoft Foundry · Model page ↗

                  Microsoft Foundry · Model page ↗

                  Microsoft Foundry · Model page ↗

                  Microsoft Foundry · Model page ↗

                  OpenRouter · Model page ↗

                  OpenRouter · Model page ↗

                  SiliconFlow · Playground ↗

                  Vercel AI Gateway · Model page ↗

                  Google Cloud · Model documentation ↗

                  Microsoft Foundry · Model documentation ↗

                  SiliconFlow · API documentation ↗

                  arXiv · Paper · 2606.19348 ↗

                  GitHub · Source snapshot ↗

                  Cloudflare AI · Catalog ↗

                  Epoch AI · Catalog ↗

                  Source links

                  1–13 / 13

                  Hugging Face model card

                  1–1 / 1

                  Explore more models

                  Models

                  Loading
                  EVIDENCE / COMPARISON

                  Benchmarks

                  Explore every benchmark

                  Loading

                  These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                  Install TensorTrace