DeepSeek: DeepSeek V4 Flash 0731
DeepSeek V4 Flash 0731 is a sparse mixture-of-experts model from DeepSeek, with 13B active parameters out of 284B total. This re-post-trained revision is suited for coding, reasoning, and agent workf…
deepseek/deepseek-v4-flash-20260731
DeepSeek V4 Flash 0731 is a sparse mixture-of-experts model from DeepSeek, with 13B active parameters out of 284B total. This re-post-trained revision is suited for coding, reasoning, and agent workflows. This is the GA release of DeepSeek V4 Flash.
Specifications & capabilities
- Input
- text
- Output
- text
- Context window · tokens
- 1048576
- Knowledge cutoff
- 2025-05
- Published model ID
- DeepSeek-V4-Flash-0731
- Capabilities
- promptCaching, reasoning, structuredOutputs, toolCalling
Description source ↗Evaluations
61–77 / 77
73.529%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Relace
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 10
- sample size
- 850
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 17
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Relace
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-02T18:06:38.075Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 850
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7352940588235294
- accuracy_std_dev
- 0.03872239054432369
- cost_per_task_usd
- 0.009413557647058823
- output_tokens_per_task
- 15440.829411764706
- latency_seconds_per_task
- 215.23848941176473
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 13
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗75.417%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Alibaba
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 11
- sample size
- 640
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 13
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Alibaba
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T11:41:35.064Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 640
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.75416675
- accuracy_std_dev
- 0.019096569464219906
- cost_per_task_usd
- 0.0324113640625
- output_tokens_per_task
- 15840.8265625
- latency_seconds_per_task
- 163.294796875
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 29
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗74.867%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Parasail
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 11
- sample size
- 500
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 10
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Parasail
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T12:32:12.444Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 500
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7486667
- accuracy_std_dev
- 0.02733343658616678
- cost_per_task_usd
- 0.018688805999999995
- output_tokens_per_task
- 15779.308
- latency_seconds_per_task
- 207.184068
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 28
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗72%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Fireworks
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 11
- sample size
- 100
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Fireworks
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-09-19T08:13:24.062Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 100
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.72
- accuracy_std_dev
- 0.030000000000000027
- cost_per_task_usd
- 0.026063740000000002
- output_tokens_per_task
- 12968.38
- latency_seconds_per_task
- 127.59792
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 14
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗87.429%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Together
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 13
- sample size
- 1770
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 9
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Together
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-17T08:23:51.334Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1770
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8742938723163841
- accuracy_std_dev
- 0.008148155125586256
- cost_per_task_usd
- 0.009214897175141243
- output_tokens_per_task
- 32530.198305084745
- latency_seconds_per_task
- 375.89575536723163
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 25
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗85.387%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Nebius
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 13
- sample size
- 990
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Nebius
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-12T15:01:54.124Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 990
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8538717999999998
- accuracy_std_dev
- 0.04946631113151657
- cost_per_task_usd
- 0.005526387878787878
- output_tokens_per_task
- 19384.131313131315
- latency_seconds_per_task
- 190.0744404040404
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 25
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗75.533%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Novita
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 13
- sample size
- 500
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 10
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Novita
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T12:45:56.690Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 500
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7553333999999999
- accuracy_std_dev
- 0.017650954020675496
- cost_per_task_usd
- 0.040253192
- output_tokens_per_task
- 16547.004
- latency_seconds_per_task
- 172.625032
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 52
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗70.19%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Nebius
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 16
- sample size
- 246
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Nebius
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-12T15:21:11.557Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 246
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7018973089430894
- accuracy_std_dev
- 0.04349030049378826
- cost_per_task_usd
- 0.029020682926829265
- output_tokens_per_task
- 21482.62601626016
- latency_seconds_per_task
- 391.9913536585366
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 23
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗67.253%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · DigitalOcean
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 17
- sample size
- 398
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 8
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- DigitalOcean
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T11:55:35.800Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 398
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.6725294120603015
- accuracy_std_dev
- 0.11099160244860588
- cost_per_task_usd
- 0.03270493216080402
- output_tokens_per_task
- 69160.1582914573
- latency_seconds_per_task
- 1961.0637788944723
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 20
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗75.333%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · DeepInfra
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 18
- sample size
- 550
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 11
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- DeepInfra
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T13:30:46.598Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 550
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7533332727272728
- accuracy_std_dev
- 0.023952899912084696
- cost_per_task_usd
- 0.007874056363636363
- output_tokens_per_task
- 15094.398181818182
- latency_seconds_per_task
- 231.90571636363637
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 63
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗73.407%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · auto-routing
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 44
- sample size
- 450
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 9
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- auto-routing
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T11:40:30.784Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 450
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7340741111111111
- accuracy_std_dev
- 0.05052255713253118
- cost_per_task_usd
- 0.009341564444444445
- output_tokens_per_task
- 15978.137777777778
- latency_seconds_per_task
- 247.29254444444445
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 128
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗86.683%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · auto-routing
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 48
- sample size
- 1980
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 10
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- auto-routing
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:17:29.008Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1980
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8668349000000002
- accuracy_std_dev
- 0.02737903360401897
- cost_per_task_usd
- 0.008139828787878788
- output_tokens_per_task
- 26892.52676767677
- latency_seconds_per_task
- 416.33891818181814
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 132
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4-flash-20260731
- source model id
- deepseek/deepseek-v4-flash-20260731
Original result ↗69.1
- source id
- artificialAnalysis
- dimension
- coding
- metric
- Coding Index
Original result ↗41
- source id
- artificialAnalysis
- dimension
- agentic
- metric
- Agentic Index
Original result ↗34.3
- source id
- artificialAnalysis
- dimension
- intelligence
- metric
- Intelligence Index
Original result ↗1,472.3
- source id
- kaggleBenchmarks
- leaderboard name
- Code2Video Bench · ELO-Ranking
- dimension
- coding
- metric
- Code2Video Bench · ELO-Ranking
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 582971
- name
- ELO-Ranking
- version
- 1
- sort_order
- DESCENDING
- display_type
- floatCounts
- benchmark
- id
- 7434
- name
- Code2Video Bench
- type
- INDIVIDUAL
- owner
- heygen
- version
- 1
- categories
- coding
- multimodal capabilities
- version_id
- 7450
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · heygen · Code2Video Bench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Code2Video Bench · ELO-Ranking
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 7450/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/heygen/code2video/versions/1
- measurements
- name
- ELO-Ranking
- unit
- score
- scale
- 1
- metric
- Code2Video Bench · ELO-Ranking
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 1472.3
- standardDeviation
- 0
- confidenceInterval
- 10.6
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 10.6
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2026-10-22T04:05:24.6596936Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 1472.3
- standardDeviation
- 0
- confidenceInterval
- 10.6
- hasStandardDeviation
- false
- hasConfidenceInterval
- true
- confidenceIntervalNullable
- 10.6
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 582971
- sort_order
- DESCENDING
- source_name
- ELO-Ranking
- display_type
- floatCounts
- task_version
- 1
- task_display_type
- FLOAT_COUNTS
- normalization_status
- normalized
- published_at
- 2026-09-21T14:05:02.139768400Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- deepseek-ai/deepseek-v4-flash-0731
- metric_population
- 19
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2026-10-22T04:05:24.659Z
- source model id
- Deepseek V4 Flash 0731
- confidence lower
- 1461.7
- confidence upper
- 1482.8999999999999
Original result ↗9.4%
- source id
- kaggleBenchmarks
- leaderboard name
- Code2Video Bench · Raw Score
- dimension
- coding
- metric
- Code2Video Bench · Raw Score
- rank
- 13
- aggregation scope
- kaggle
- true
- status
- published
- license
- Apache-2.0
- direction
- higherIsBetter
- conditions
- task
- id
- 583283
- name
- Raw Score
- version
- 1
- sort_order
- DESCENDING
- display_type
- percentages
- benchmark
- id
- 7434
- name
- Code2Video Bench
- type
- INDIVIDUAL
- owner
- heygen
- version
- 1
- categories
- coding
- multimodal capabilities
- version_id
- 7450
- maintainer_tier
- Community-published
- candidate
- candidate_class
- modelVersion
- record_kind
- aggregate
- attribution
- Kaggle · heygen · Code2Video Bench
- license_url
- https://www.apache.org/licenses/LICENSE-2.0
- metric_kind
- Code2Video Bench · Raw Score
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 7450/leaderboard.json
- source_page
- https://www.kaggle.com/benchmarks/heygen/code2video/versions/1
- measurements
- name
- Raw Score
- unit
- percent
- scale
- 1
- metric
- Code2Video Bench · Raw Score
- result
- resultCase
- numericResult
- booleanResult
- false
- numericResult
- value
- 0.094
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- taskVersionId
- 0
- evaluationDate
- 2026-10-22T04:05:24.8136758Z
- hasBooleanResult
- false
- hasNumericResult
- true
- hasTaskVersionId
- false
- hasEvaluationDate
- true
- numericResultNullable
- value
- 0.094
- standardDeviation
- 0
- confidenceInterval
- 0
- hasStandardDeviation
- false
- hasConfidenceInterval
- false
- hasUnevenConfidenceInterval
- false
- hasNumericResultPublic
- false
- hasNumericResultPrivate
- false
- task_id
- 583283
- sort_order
- DESCENDING
- source_name
- Raw Score
- display_type
- percentages
- task_version
- 1
- task_display_type
- PERCENTAGES
- normalization_status
- normalized
- published_at
- 2026-09-21T14:05:02.139768400Z
- candidate_class
- modelVersion
- source_revision
- b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
- publisher_anchor
- deepseek-ai/deepseek-v4-flash-0731
- metric_population
- 19
- benchmark_evaluation
- true
- task_metadata_source
- benchmarkVersion
- candidate_metadata_source
- benchmarkVersionMappings
- evaluated at
- 2026-10-22T04:05:24.813Z
- source model id
- Deepseek V4 Flash 0731
Original result ↗Published prices
1–20 / 68
$0.01
- Inference provider
- cloudflareWorkersAI
- Platform
- Cloudflare Workers AI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- per M cached input tokens
$1.32
- Inference provider
- cloudflareWorkersAI
- Platform
- Cloudflare Workers AI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- per M output tokens
$0.02
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.06
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$0.18
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.05
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.14
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.04
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- alias_target
- name
- DeepSeek: DeepSeek V4 Flash 0731
- slug
- deepseek/deepseek-v4-flash-0731
- routing_alias
- true
$0.04
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- alias_target
- name
- DeepSeek: DeepSeek V4 Flash 0731
- slug
- deepseek/deepseek-v4-flash-0731
- routing_alias
- true
$0.30
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- alias_target
- name
- DeepSeek: DeepSeek V4 Flash 0731
- slug
- deepseek/deepseek-v4-flash-0731
- routing_alias
- true
$0.01
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- tier
- peak:off:14-24
- label
- 14:00 – 24:00(UTC)
- Billing label
- Cached input tokens (per 1M)
$0.03
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- tier
- peak:on:0-14
- label
- 00:00 – 14:00(UTC)
- Billing label
- Cached input tokens (per 1M)
$0.22
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- tier
- peak:off:14-24
- label
- 14:00 – 24:00(UTC)
- Billing label
- Input tokens (per 1M)
$0.44
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- tier
- peak:on:0-14
- label
- 00:00 – 14:00(UTC)
- Billing label
- Input tokens (per 1M)
$0.66
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- tier
- peak:off:14-24
- label
- 14:00 – 24:00(UTC)
- Billing label
- Output tokens (per 1M)
$1.32
- Inference provider
- siliconFlow
- Platform
- SiliconFlow
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- tier
- peak:on:0-14
- label
- 00:00 – 14:00(UTC)
- Billing label
- Output tokens (per 1M)
$0.02
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4-flash-0731
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 840
- start_minute_utc
- 0
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.22
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4-flash-0731
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 840
- start_minute_utc
- 0
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.66
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4-flash-0731
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 840
- start_minute_utc
- 0
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–9 / 9
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@1
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@2
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@3
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@4
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4-flash-0731@5
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- @cf/deepseek-ai/deepseek-v4-flash-0731
- hosting
- cloudflare-hosted
- context length
- 1048576
- source model id
- deepseek-ai/DeepSeek-V4-Flash-0731
- hosting
- siliconflow-hosted
- max output tokens
- 393216
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- source model id
- deepseek/deepseek-v4-flash-0731
- hosting
- vercel-gateway
- context length
- 1000000
- max output tokens
- 384000
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- source model id
- publishers/deepseek-ai/models/deepseek-v4@deepseek-v4-flash-0731
- hosting
- vertex-self-deployed
Safety assessments
1–1 / 1
32 / 100
- source id
- stanfordFMTI
- canonical model id
- deepseek/deepseek-v4-flash-20260731
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- DeepSeek
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 32 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–16 / 16
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1049K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- context_length
- value
- 1000000
- normalized value
- 1000000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-05
- normalized value
- 2025-05
- precision
- month
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 384000
- normalized value
- 384000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 384000
- normalized value
- 384000
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 943718
- normalized value
- 943718
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 393K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- max_output_tokens
- value
- 384000
- normalized value
- 384000
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
Hugging Face model card
1–1 / 1
- downloads
- 2577582
- likes
- 4099
- pipeline tag
- text-generation
- library name
- transformers
- License
- mit
Explore more models