DeepSeek: DeepSeek V4.1 Flash
DeepSeek V4.1 Flash is a sparse mixture-of-experts model from DeepSeek, and the first built on the company's Causal Encoder-Decoder (CED) architecture. It activates 8B parameters on input and 16B on …
deepseek/deepseek-v4.1-flash-20260910
DeepSeek V4.1 Flash is a sparse mixture-of-experts model from DeepSeek, and the first built on the company's Causal Encoder-Decoder (CED) architecture. It activates 8B parameters on input and 16B on output from a 552B-parameter backbone, an asymmetric split that keeps per-token compute low relative to the model's total size. Image understanding is native to the architecture, with visual and text embeddings trained jointly from the start of pre-training rather than added afterward as in the earlier experimental [V4 Flash Vision Exp](https://openrouter.ai/deepseek/deepseek-v4-flash-vision-exp).
It is suited for coding, terminal, and computer-use agents, along with long-horizon tasks that must run to completion across many steps and long-context analysis. Compressed KV caching cuts cache memory to roughly a quarter of the previous Flash generation, significantly reducing costs on agentic workloads. DeepSeek positions it as the cost-efficient tier of the V4.1 family and reports that it exceeds [V4 Pro](https://openrouter.ai/deepseek/deepseek-v4-pro-0813) on performance, speed, and task completion time.
Specifications & capabilities
- Input
- text, image
- Output
- text
- Context window · tokens
- 1048576
- Published model ID
- DeepSeek-V4.1-Flash
- Capabilities
- multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗Evaluations
1–20 / 83
90.78%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Mancer 2
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 470
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Mancer 2
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T09:05:48.501Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 470
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9078013574468085
- accuracy_std_dev
- 0.012928926018815361
- cost_per_task_usd
- 0.01807536170212766
- output_tokens_per_task
- 17947.382978723403
- latency_seconds_per_task
- 357.7867425531915
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 6
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗72.34%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Krea
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 1
- sample size
- 517
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Krea
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-09-29T20:57:20.673Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 517
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.7234044255319149
- accuracy_std_dev
- 0.1917994784299478
- cost_per_task_usd
- 0.033565591876208896
- output_tokens_per_task
- 37102.30560928433
- latency_seconds_per_task
- 242.77243907156674
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 1
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗64.556%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Krea
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 1
- sample size
- 150
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Krea
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-09-29T19:45:03.655Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 150
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.6455556666666666
- accuracy_std_dev
- 0.09964144484545016
- cost_per_task_usd
- 0.11177372666666668
- output_tokens_per_task
- 87845.5
- latency_seconds_per_task
- 710.81772
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 1
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗90.943%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · NextBit
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 2
- sample size
- 990
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- NextBit
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T09:34:35.533Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 990
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9094272
- accuracy_std_dev
- 0.00547072746168187
- cost_per_task_usd
- 0.025930728282828285
- output_tokens_per_task
- 28121.050505050505
- latency_seconds_per_task
- 177.8861121212121
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 10
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗76.254%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Crusoe
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 2
- sample size
- 299
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Crusoe
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T12:18:15.192Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 299
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7625417625418061
- accuracy_std_dev
- 0.01910679234955175
- cost_per_task_usd
- 0.03341546153846154
- output_tokens_per_task
- 21777.678929765887
- latency_seconds_per_task
- 144.9030602006689
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 13
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗90.787%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Baidu
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 3
- sample size
- 1326
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 7
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Baidu
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-10T18:05:49.922Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1326
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9078681221719456
- accuracy_std_dev
- 0.005280935005338756
- cost_per_task_usd
- 0.023841498491704374
- output_tokens_per_task
- 27449.11161387632
- latency_seconds_per_task
- 170.5565128205128
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 12
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗90.572%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · SiliconFlow
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 3
- sample size
- 1782
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 9
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- SiliconFlow
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:29:56.453Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1782
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9057238888888889
- accuracy_std_dev
- 0.009356618102764875
- cost_per_task_usd
- 0.03432713860830528
- output_tokens_per_task
- 28444.87317620651
- latency_seconds_per_task
- 196.75796408529743
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 34
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗89.957%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DigitalOcean
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 3
- sample size
- 1779
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 9
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DigitalOcean
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:33:02.133Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1779
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8995687234401349
- accuracy_std_dev
- 0.007447984730328704
- cost_per_task_usd
- 0.0197627712197864
- output_tokens_per_task
- 22859.64811691962
- latency_seconds_per_task
- 382.0163063518831
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 23
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗89.431%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · ModelRun
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 3
- sample size
- 533
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- ModelRun
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-28T05:00:14.622Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 533
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8943087542213883
- accuracy_std_dev
- 0.03942204850926279
- cost_per_task_usd
- 0.019450917448405252
- output_tokens_per_task
- 28174.943714821762
- latency_seconds_per_task
- 177.9272420262664
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 8
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗88.507%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DekaLLM
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 3
- sample size
- 989
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DekaLLM
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-10T08:22:45.418Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 989
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8850689656218401
- accuracy_std_dev
- 0.014411932672615443
- cost_per_task_usd
- 0.026149528816986855
- output_tokens_per_task
- 26995.41152679474
- latency_seconds_per_task
- 355.79747320525786
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 12
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗88.293%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Reka
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 3
- sample size
- 783
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 4
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Reka
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:15:03.390Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 783
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8829290000000001
- accuracy_std_dev
- 0.012699110785799135
- cost_per_task_usd
- 0.027103435504469985
- output_tokens_per_task
- 30539.469987228607
- latency_seconds_per_task
- 200.8993601532567
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 13
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗87.744%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Ionstream
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 3
- sample size
- 990
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 5
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Ionstream
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-21T11:23:15.101Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 990
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8774407999999999
- accuracy_std_dev
- 0.00954717737134911
- cost_per_task_usd
- 0.03921924343434343
- output_tokens_per_task
- 37354.009090909094
- latency_seconds_per_task
- 242.47004242424242
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 5
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗87.226%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · OpenInference
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 3
- sample size
- 1584
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 8
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- OpenInference
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:09:31.159Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1584
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8722639999999998
- accuracy_std_dev
- 0.01391911622553672
- cost_per_task_usd
- 0.01943295707070707
- output_tokens_per_task
- 35123.857954545456
- latency_seconds_per_task
- 314.6860820707071
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 7
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗73.333%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Mancer 2
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 3
- sample size
- 150
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Mancer 2
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T12:31:12.958Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 150
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.7333333333333333
- accuracy_std_dev
- 0.014401903168987377
- cost_per_task_usd
- 0.06451945333333334
- output_tokens_per_task
- 15144.64
- latency_seconds_per_task
- 876.0850533333333
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 5
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗90.816%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Alibaba
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 4
- sample size
- 1782
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 9
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Alibaba
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:40:16.353Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1782
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9081554444444445
- accuracy_std_dev
- 0.008876832231665518
- cost_per_task_usd
- 0.033613874298540965
- output_tokens_per_task
- 27019.509539842875
- latency_seconds_per_task
- 210.73789562289562
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 29
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗90.535%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · StreamLake
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 4
- sample size
- 1782
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 9
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- StreamLake
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T09:01:06.228Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1782
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.9053496666666667
- accuracy_std_dev
- 0.008612696635394351
- cost_per_task_usd
- 0.02068693490460157
- output_tokens_per_task
- 28854.00224466891
- latency_seconds_per_task
- 208.79350953984286
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 30
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗89.541%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · AtlasCloud
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 4
- sample size
- 1584
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 8
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- AtlasCloud
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:27:49.932Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1584
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.895412375
- accuracy_std_dev
- 0.020051295631065218
- cost_per_task_usd
- 0.02414115151515151
- output_tokens_per_task
- 27810.135101010103
- latency_seconds_per_task
- 210.27494823232323
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 30
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗89.529%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Phala
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 4
- sample size
- 1536
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 8
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Phala
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T09:17:17.877Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1536
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8952905937499999
- accuracy_std_dev
- 0.016969367452854572
- cost_per_task_usd
- 0.032397834635416664
- output_tokens_per_task
- 32195.716796875
- latency_seconds_per_task
- 236.218794921875
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 24
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗89.415%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Morph
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 4
- sample size
- 1584
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 8
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Morph
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T08:33:46.211Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1584
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8941497500000002
- accuracy_std_dev
- 0.012326495474687834
- cost_per_task_usd
- 0.010706986111111112
- output_tokens_per_task
- 20748.16351010101
- latency_seconds_per_task
- 265.82926073232323
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 10
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗88.552%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · River
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 4
- sample size
- 594
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- River
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-11-14T22:14:47.015Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 594
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.8855219999999999
- accuracy_std_dev
- 0.009912179612308596
- cost_per_task_usd
- 0.016982043771043773
- output_tokens_per_task
- 35207.07407407407
- latency_seconds_per_task
- 144.32716498316498
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 6
- benchmark_evaluation
- true
- openrouter_permaslug
- deepseek/deepseek-v4.1-flash-20260910
- source model id
- deepseek/deepseek-v4.1-flash-20260910
Original result ↗Published prices
61–80 / 96
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- particle
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- particle | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.03
- Inference provider
- vercelAIGatewayCatalog
- Platform
- particle
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- scope
- zone
- geo_region
- us
- endpoint
- particle | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.07
- Inference provider
- vercelAIGatewayCatalog
- Platform
- particle
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- particle | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.07
- Inference provider
- vercelAIGatewayCatalog
- Platform
- particle
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- scope
- zone
- geo_region
- us
- endpoint
- particle | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$1.00
- Inference provider
- vercelAIGatewayCatalog
- Platform
- particle
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- particle | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$1.00
- Inference provider
- vercelAIGatewayCatalog
- Platform
- particle
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- region
- scope
- zone
- geo_region
- us
- endpoint
- particle | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.0030
- Inference provider
- vercelAIGatewayCatalog
- Platform
- relace
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- relace | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.04
- Inference provider
- vercelAIGatewayCatalog
- Platform
- relace
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- relace | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.50
- Inference provider
- vercelAIGatewayCatalog
- Platform
- relace
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- relace | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.02
- Inference provider
- vercelAIGatewayCatalog
- Platform
- runinfra
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- runinfra | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.11
- Inference provider
- vercelAIGatewayCatalog
- Platform
- runinfra
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- runinfra | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.46
- Inference provider
- vercelAIGatewayCatalog
- Platform
- runinfra
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- runinfra | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- runware
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- runware | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.15
- Inference provider
- vercelAIGatewayCatalog
- Platform
- runware
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- runware | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.60
- Inference provider
- vercelAIGatewayCatalog
- Platform
- runware
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- runware | deepseek/deepseek-v4.1-flash
- model_type
- language
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.0030
- Inference provider
- vercelAIGatewayCatalog
- Platform
- tencent
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- tencent | deepseek/deepseek-v4.1-flash
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 240
- start_minute_utc
- 60
- end_minute_utc
- 600
- start_minute_utc
- 360
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.15
- Inference provider
- vercelAIGatewayCatalog
- Platform
- tencent
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- tencent | deepseek/deepseek-v4.1-flash
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 240
- start_minute_utc
- 60
- end_minute_utc
- 600
- start_minute_utc
- 360
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.60
- Inference provider
- vercelAIGatewayCatalog
- Platform
- tencent
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- tencent | deepseek/deepseek-v4.1-flash
- model_type
- language
- peak_pricing
- windows
- end_minute_utc
- 240
- start_minute_utc
- 60
- end_minute_utc
- 600
- start_minute_utc
- 360
- multiplier
- 2
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.01
- Inference provider
- vercelAIGatewayCatalog
- Platform
- togetherai
- Unit
- cache_read_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- togetherai | deepseek/deepseek-v4.1-flash
- model_type
- language
- quantization
- fp8
- billing_scope
- via Vercel AI Gateway
- Billing label
- Cached input / 1M
- Pricing via
- Vercel AI Gateway
$0.30
- Inference provider
- vercelAIGatewayCatalog
- Platform
- togetherai
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- togetherai | deepseek/deepseek-v4.1-flash
- model_type
- language
- quantization
- fp8
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–6 / 6
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4.1-flash@1
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- azure-huggingface/deepseek-ai--deepseek-v4.1-flash@2
- hosting
- foundry-managed-compute
- license
- MIT
- context length
- 1048576
- source model id
- deepseek-ai/deepseek-v4.1-flash
- hosting
- nvidia-catalog
- context length
- 1048576
- source model id
- deepseek-ai/DeepSeek-V4.1-Flash
- hosting
- siliconflow-hosted
- max output tokens
- 393216
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- source model id
- deepseek/deepseek-v4.1-flash
- hosting
- vercel-gateway
- context length
- 1048576
- max output tokens
- 32768
- capabilities
- promptCaching
- reasoning
- structuredOutputs
- toolCalling
- visionInput
- source model id
- publishers/deepseek-ai/models/deepseek-v4.1@deepseek-v4.1-flash
- hosting
- vertex-self-deployed
Safety assessments
1–1 / 1
32 / 100
- source id
- stanfordFMTI
- canonical model id
- deepseek/deepseek-v4.1-flash-20260910
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- DeepSeek
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 32 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–11 / 11
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- nvidiaNim
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 1049K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- context_length
- value
- 1048576
- normalized value
- 1048576
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- max_output_tokens
- value
- 393216
- normalized value
- 393216
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 393K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- max_output_tokens
- value
- 32768
- normalized value
- 32768
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
Hugging Face model card
1–1 / 1
- downloads
- 1694181
- likes
- 4978
- pipeline tag
- image-text-to-text
- library name
- transformers
- License
- mit
Explore more models