DeepSeek: DeepSeek V4.1 Flash 1017
DeepSeek V4.1 Flash 1017 is the October 17, 2026 snapshot of DeepSeek V4.1 Flash, served by DeepSeek under the `deepseek-flash` API model name. V4.1 Flash is a sparse mixture-of-experts model from De…
deepseek/deepseek-v4.1-flash-20261017
DeepSeek V4.1 Flash 1017 is the October 17, 2026 snapshot of DeepSeek V4.1 Flash, served by DeepSeek under the deepseek-flash API model name. V4.1 Flash is a sparse mixture-of-experts model from DeepSeek, and the first built on the company's Causal Encoder-Decoder (CED) architecture. It activates 8B parameters on input and 16B on output from a 552B-parameter backbone, an asymmetric split that keeps per-token compute low relative to the model's total size. Image understanding is native to the architecture, with visual and text embeddings trained jointly from the start of pre-training rather than added afterward as in the earlier experimental V4 Flash Vision Exp.
It is suited for coding, terminal, and computer-use agents, along with long-horizon tasks that must run to completion across many steps and long-context analysis. Compressed KV caching cuts cache memory to roughly a quarter of the previous Flash generation, significantly reducing costs on agentic workloads. DeepSeek positions it as the cost-efficient tier of the V4.1 family and reports that it exceeds V4 Pro on performance, speed, and task completion time.
Specifications & capabilities
- Input
- text, image
- Output
- text
- Context window · tokens
- 1048576
- Published model ID
- DeepSeek-V4.1-Flash-1017
- Capabilities
- multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗Evaluations
1–20 / 83
92.812%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Tenstorrent
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 160
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Tenstorrent
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-04T18:00:19.969Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 160
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9281247875
- accuracy_std_dev
- 0.003304499999999988
- cost_per_task_usd
- 0.01232809375
- output_tokens_per_task
- 10178.75625
- latency_seconds_per_task
- 251.38681875
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 8
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗92.34%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Near AI
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 396
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Near AI
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T10:06:11.748Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 396
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9234005000000001
- accuracy_std_dev
- 0.005892500000000023
- cost_per_task_usd
- 0.02620280303030303
- output_tokens_per_task
- 21737.315656565657
- latency_seconds_per_task
- 592.1412247474748
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 4
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗91.863%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · SiliconFlow
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 594
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- SiliconFlow
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:50:50.713Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 594
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9186303333333333
- accuracy_std_dev
- 0.010408091158111372
- cost_per_task_usd
- 0.025292042087542085
- output_tokens_per_task
- 20957.713804713803
- latency_seconds_per_task
- 160.4023451178451
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 34
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗91.583%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DekaLLM
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 198
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 1
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DekaLLM
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-04T09:00:09.993Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 198
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.915825
- cost_per_task_usd
- 0.014663378787878788
- output_tokens_per_task
- 21419.76767676768
- latency_seconds_per_task
- 96.92635858585858
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 12
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗91.497%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · CoreWeave
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 196
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 1
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- CoreWeave
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-04T18:00:08.219Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 196
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9149660000000001
- cost_per_task_usd
- 0.022535153061224493
- output_tokens_per_task
- 18639.61224489796
- latency_seconds_per_task
- 106.1645357142857
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 27
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗91.145%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · GMICloud
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 990
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- GMICloud
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-10T06:01:40.563Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 990
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9114477999999999
- accuracy_std_dev
- 0.0039265801609033885
- cost_per_task_usd
- 0.018160035353535352
- output_tokens_per_task
- 22106.812121212122
- latency_seconds_per_task
- 129.7026080808081
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 26
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗90.909%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · StreamLake
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 396
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- StreamLake
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:36:23.829Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 396
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9090910000000001
- accuracy_std_dev
- 0.003367000000000009
- cost_per_task_usd
- 0.018701613636363638
- output_tokens_per_task
- 22059.484848484848
- latency_seconds_per_task
- 137.00363383838382
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 30
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗90.909%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · StepFun
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 1188
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- StepFun
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-02T17:24:31.225Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1188
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9090908333333333
- accuracy_std_dev
- 0.007076201038614481
- cost_per_task_usd
- 0.01709530134680135
- output_tokens_per_task
- 21725.516835016835
- latency_seconds_per_task
- 84.45594696969698
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 1
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗90.741%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Cohere
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 396
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Cohere
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:13:11.036Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 396
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9074074999999998
- accuracy_std_dev
- 0.008417500000000022
- cost_per_task_usd
- 0.014508287878787877
- output_tokens_per_task
- 21790.61111111111
- latency_seconds_per_task
- 90.08695707070707
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 7
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗90.432%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Mistral
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 1188
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Mistral
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-11T02:59:16.591Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1188
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.904321
- accuracy_std_dev
- 0.006024476685433623
- cost_per_task_usd
- 0.025462516835016838
- output_tokens_per_task
- 21063.152356902356
- latency_seconds_per_task
- 81.18970454545455
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 5
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗90.32%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Particle AI
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 792
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 4
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Particle AI
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-03T01:49:06.293Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 792
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9031985
- accuracy_std_dev
- 0.0075759444460740165
- cost_per_task_usd
- 0.01504592297979798
- output_tokens_per_task
- 17203.179292929293
- latency_seconds_per_task
- 86.73782702020202
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 2
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗89.798%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Ionstream
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 990
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Ionstream
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-10T16:47:27.230Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 990
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8979796
- accuracy_std_dev
- 0.008689115757083655
- cost_per_task_usd
- 0.014924614141414142
- output_tokens_per_task
- 23160.81515151515
- latency_seconds_per_task
- 134.09519090909092
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 5
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗89.169%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · OpenInference
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 594
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- OpenInference
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T09:01:06.311Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 594
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8916946666666665
- accuracy_std_dev
- 0.009944030347678707
- cost_per_task_usd
- 0.011122356902356903
- output_tokens_per_task
- 25444.905723905726
- latency_seconds_per_task
- 302.42450673400674
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 7
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗77.778%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · AtlasCloud
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 1
- sample size
- 150
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- AtlasCloud
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T11:43:38.949Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 150
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.777778
- accuracy_std_dev
- 0.01257094435593444
- cost_per_task_usd
- 0.016507473333333335
- output_tokens_per_task
- 10861.266666666666
- latency_seconds_per_task
- 92.51076666666667
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 29
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗72.556%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · StepFun
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 1
- sample size
- 300
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- StepFun
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-02T17:45:53.936Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 300
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7255556666666666
- accuracy_std_dev
- 0.016960557004087128
- cost_per_task_usd
- 0.012908883333333333
- output_tokens_per_task
- 10836.613333333333
- latency_seconds_per_task
- 53.54427
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 1
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗92.281%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DeepInfra
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 2
- sample size
- 570
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DeepInfra
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:40:34.158Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 570
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9228069789473685
- accuracy_std_dev
- 0.011126928806578514
- cost_per_task_usd
- 0.00580749649122807
- output_tokens_per_task
- 19521.97543859649
- latency_seconds_per_task
- 218.61469122807017
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 71
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗92.088%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DigitalOcean
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 2
- sample size
- 594
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DigitalOcean
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:09:30.885Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 594
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9208753333333334
- accuracy_std_dev
- 0.003636621356271357
- cost_per_task_usd
- 0.019656516835016836
- output_tokens_per_task
- 20830.57239057239
- latency_seconds_per_task
- 90.4379696969697
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 23
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗91.758%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · PrimaLabs
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 2
- sample size
- 182
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 2
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- PrimaLabs
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-06T15:33:21.027Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 182
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9175825384615385
- accuracy_std_dev
- 0.009135000000000004
- cost_per_task_usd
- 0.03656871978021978
- output_tokens_per_task
- 10074.60989010989
- latency_seconds_per_task
- 44.23142857142857
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 3
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗91.311%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Baidu
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 2
- sample size
- 750
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 4
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Baidu
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-10T18:06:00.757Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 750
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.91311096
- accuracy_std_dev
- 0.009099497963486788
- cost_per_task_usd
- 0.01717951733333333
- output_tokens_per_task
- 20403.964
- latency_seconds_per_task
- 130.3304
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 12
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗90.544%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Makora
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 2
- sample size
- 987
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Makora
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:31:11.848Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 987
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9054373465045593
- accuracy_std_dev
- 0.0037554479359991244
- cost_per_task_usd
- 0.026207787234042552
- output_tokens_per_task
- 21741.29989868288
- latency_seconds_per_task
- 127.14136068895644
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 12
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20261017
- source model id
- deepseek/deepseek-v4.1-flash-20261017
Original result ↗Published prices
1–20 / 110
$0.01
- Inference provider
- cloudflareWorkersAI
- Platform
- Cloudflare Workers AI
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing label
- per M cached input tokens
$1.20
- Inference provider
- cloudflareWorkersAI
- Platform
- Cloudflare Workers AI
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing label
- per M output tokens
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- alias_target
- name
- DeepSeek: DeepSeek V4.1 Flash 1017
- slug
- deepseek/deepseek-v4.1-flash-1017
- routing_alias
- true
$0.04
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- alias_target
- name
- DeepSeek: DeepSeek V4.1 Flash 1017
- slug
- deepseek/deepseek-v4.1-flash-1017
- routing_alias
- true
$0.50
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- alias_target
- name
- DeepSeek: DeepSeek V4.1 Flash 1017
- slug
- deepseek/deepseek-v4.1-flash-1017
- routing_alias
- true
$0.01
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.30
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$1.20
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.0040
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.04
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.38
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- mode
- batch
$0.02
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 840
- start_minute_utc
- 0
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.15
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 840
- start_minute_utc
- 0
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.60
- Inference provider
- vercelAIGatewayCatalog
- Platform
- alibaba
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- alibaba | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 840
- start_minute_utc
- 0
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- baseten | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- scope
- zone
- geo_region
- us
- endpoint
- baseten | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.30
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- baseten | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.30
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- scope
- zone
- geo_region
- us
- endpoint
- baseten | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$1.20
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- baseten | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$1.20
- Inference provider
- vercelAIGatewayCatalog
- Platform
- baseten
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- scope
- zone
- geo_region
- us
- endpoint
- baseten | deepseek/deepseek-v4.1-flash-1017
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–5 / 5
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4.1-flash-1017@1
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4.1-flash-1017@2
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4.1-flash-1017@3
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- @cf/deepseek-ai/deepseek-v4.1-flash-1017
- hosting
- cloudflare-hosted
- context length
- 1048576
- source model id
- deepseek/deepseek-v4.1-flash-1017
- hosting
- vercel-gateway
- context length
- 1048576
- max output tokens
- 262144
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- visionInput
Safety assessments
1–1 / 1
32 / 100
- source id
- stanfordFMTI
- canonical model id
- deepseek/deepseek-v4.1-flash-20261017
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- DeepSeek
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 32 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–11 / 11
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 943718
- normalized value
- 943718
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 393216
- normalized value
- 393216
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 943718
- normalized value
- 943718
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 262144
- normalized value
- 262144
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
Explore more models