DeepSeek: R1 0528

May 28th update to the [original DeepSeek R1](/deepseek/deepseek-r1) Performance on par with [OpenAI o1](/openai/o1), but open-sourced and with fully open reasoning tokens. It's 671B parameters in si…

deepseek/deepseek-r1-0528

May 28th update to the [original DeepSeek R1](/deepseek/deepseek-r1) Performance on par with [OpenAI o1](/openai/o1), but open-sourced and with fully open reasoning tokens. It's 671B parameters in size, with 37B active in an inference pass.

Fully open-source model.

Specifications & capabilities

Input
text
Output
text
Context window · tokens
163840
Knowledge cutoff
2025-03-31
Published model ID
DeepSeek-R1-0528
Capabilities
promptCaching, reasoning, structuredOutputs, toolCalling
Description source ↗

Evaluations

1–20 / 21

OpenCompass LLM

83.5%

source id
openCompass
leaderboard name
MMLU-Pro
dimension
general
metric
MMLU-Pro
rank
13
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
MMLU-Pro
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
DeepSeek-R1-0528
Original result ↗

HELM Capabilities

66.592%

source id
helm
leaderboard name
HELM GPQA
dimension
intelligence
metric
chain_of_thought_correctness
rank
16
sample size
446
source model id
deepseek-ai/deepseek-r1-0528
Original result ↗

OpenCompass LLM

89%

source id
openCompass
leaderboard name
AIME2025
dimension
general
metric
AIME2025
rank
17
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
AIME2025
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
DeepSeek-R1-0528
Original result ↗

OpenCompass LLM

80.6%

source id
openCompass
leaderboard name
GPQA-Diamond
dimension
general
metric
GPQA-Diamond
rank
18
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
GPQA-Diamond
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
DeepSeek-R1-0528
Original result ↗

HELM Capabilities

82.809%

source id
helm
leaderboard name
HELM WildBench
dimension
agentic
metric
wildbench_score_rescaled
rank
21
sample size
1000
source model id
deepseek-ai/deepseek-r1-0528
Original result ↗

HELM Capabilities

79.3%

source id
helm
leaderboard name
HELM MMLU-Pro
dimension
intelligence
metric
chain_of_thought_correctness
rank
21
sample size
1000
source model id
deepseek-ai/deepseek-r1-0528
Original result ↗

OpenCompass LLM

14.4%

source id
openCompass
leaderboard name
HLE
dimension
general
metric
HLE
rank
21
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
HLE
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
DeepSeek-R1-0528
Original result ↗

OpenRouter Benchmarks

79.586%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · SiliconFlow
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
24
sample size
934
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
SiliconFlow
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T16:06:10.469Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
934
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.79586
accuracy_std_dev
0.006604009430641372
cost_per_task_usd
0.05025589079229122
output_tokens_per_task
22883.693790149893
latency_seconds_per_task
1087.3444850107066
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
34
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-r1-0528
source model id
deepseek/deepseek-r1-0528
Original result ↗

OpenCompass LLM

68.1%

source id
openCompass
leaderboard name
Average
dimension
general
metric
Average
rank
24
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
Average
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
DeepSeek-R1-0528
Original result ↗

OpenRouter Benchmarks

47.5%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · SiliconFlow
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
24
sample size
200
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
SiliconFlow
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T19:38:30.603Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
200
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.47499975000000005
accuracy_std_dev
0.002886895683515417
cost_per_task_usd
0.171962235
output_tokens_per_task
45557.86
latency_seconds_per_task
2391.474385
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
28
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-r1-0528
source model id
deepseek/deepseek-r1-0528
Original result ↗

OpenRouter Benchmarks

79.321%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · StreamLake
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
26
sample size
1188
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
6
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
StreamLake
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T16:03:34.700Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1188
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7932098333333334
accuracy_std_dev
0.006693092768336313
cost_per_task_usd
0.05432222053872054
output_tokens_per_task
23581.273569023568
latency_seconds_per_task
460.7204478114478
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
30
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-r1-0528
source model id
deepseek/deepseek-r1-0528
Original result ↗

HELM Capabilities

42.4%

source id
helm
leaderboard name
HELM Omni-MATH
dimension
intelligence
metric
omni_math_accuracy
rank
26
sample size
1000
source model id
deepseek-ai/deepseek-r1-0528
Original result ↗

OpenCompass LLM

61%

source id
openCompass
leaderboard name
LiveCodeBenchV6
dimension
general
metric
LiveCodeBenchV6
rank
29
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
LiveCodeBenchV6
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
DeepSeek-R1-0528
Original result ↗

OpenRouter Benchmarks

78.524%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · DeepInfra
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
32
sample size
1050
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
6
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T16:06:15.029Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1050
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7852381866666666
accuracy_std_dev
0.022517162489606328
cost_per_task_usd
0.039451490476190475
output_tokens_per_task
18229.481904761906
latency_seconds_per_task
759.0874819047618
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
71
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-r1-0528
source model id
deepseek/deepseek-r1-0528
Original result ↗

OpenRouter Benchmarks

58.4%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Novita
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
41
sample size
250
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Novita
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T19:42:10.129Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
250
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.5839998000000001
accuracy_std_dev
0.06031032194707637
cost_per_task_usd
0.21038078000000002
output_tokens_per_task
47038.516
latency_seconds_per_task
2412.162416
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
52
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-r1-0528
source model id
deepseek/deepseek-r1-0528
Original result ↗

OpenCompass LLM

80%

source id
openCompass
leaderboard name
IFEval
dimension
general
metric
IFEval
rank
45
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
IFEval
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
DeepSeek-R1-0528
Original result ↗

OpenRouter Benchmarks

65.152%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Novita
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
47
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Novita
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T16:03:44.041Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.651515
accuracy_std_dev
0.1905456270048375
cost_per_task_usd
0.058297676767676776
output_tokens_per_task
23174.20202020202
latency_seconds_per_task
1098.090579124579
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
59
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-r1-0528
source model id
deepseek/deepseek-r1-0528
Original result ↗

HELM Capabilities

78.373%

source id
helm
leaderboard name
HELM IFEval
dimension
agentic
metric
ifeval_strict_accuracy
rank
51
sample size
541
source model id
deepseek-ai/deepseek-r1-0528
Original result ↗

OpenRouter Benchmarks

76.7%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
90
sample size
990
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T16:03:24.588Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
990
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7670034
accuracy_std_dev
0.04110764542320566
cost_per_task_usd
0.052571260606060606
output_tokens_per_task
23291.960606060606
latency_seconds_per_task
891.0540545454546
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-r1-0528
source model id
deepseek/deepseek-r1-0528
Original result ↗

OpenRouter Benchmarks

49.533%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
105
sample size
250
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
5
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-17T19:35:49.791Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
250
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.4953334
accuracy_std_dev
0.03792387345512057
cost_per_task_usd
0.17013926399999998
output_tokens_per_task
40036.992
latency_seconds_per_task
2120.647312
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
deepseek/deepseek-r1-0528
source model id
deepseek/deepseek-r1-0528
Original result ↗

Published prices

1–5 / 5

Provider specifications & access

1–2 / 2

deepseek-ai/DeepSeek-R1-0528 ↗

source model id
publishers/deepseek-ai/models/deepseek-r1@deepseek-r1-0528
hosting
vertex-self-deployed
platforms
  • Google Cloud Vertex AI
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    deepseek-ai/DeepSeek-R1-0528 ↗

    source model id
    publishers/hf-deepseek-ai/models/deepseek-r1-0528@001
    hosting
    vertex-self-deployed
    platforms
    • Google Cloud Vertex AI
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      Safety assessments

      1–1 / 1

      Foundation Model Transparency Index 2025 ↗

      32 / 100

      source id
      stanfordFMTI
      canonical model id
      deepseek/deepseek-r1-0528
      taxonomy
      transparency
      hazard category
      transparency
      hazard name
      Organization transparency
      policy area
      DeepSeek
      assessment type
      organization-level disclosure
      metric
      disclosure_score
      result label
      32 / 100
      sample size
      100
      updated at
      2026-11-09T18:08:05.227Z

      Source specification observations

      1–3 / 3

      openRouter ↗

      field
      context_length
      value
      163840
      normalized value
      163840
      scope
      source-model
      source id
      openRouter
      confidence
      1

      openRouter ↗

      field
      knowledge_cutoff
      reported field
      knowledge_cutoff
      value
      2025-03-31
      normalized value
      2025-03-31
      precision
      day
      scope
      source-model
      source id
      openRouter
      confidence
      1

      openRouter ↗

      field
      max_output_tokens
      value
      32768
      normalized value
      32768
      scope
      source-model
      source id
      openRouter
      confidence
      1

      Original sources

      1–14 / 14

      Hugging Face · deepseek-ai/DeepSeek-R1-0528 ↗

      DeepInfra · Model page ↗

      Google Cloud · Model page ↗

      Google Cloud · Model page ↗

      OpenRouter · Model page ↗

      Google Cloud · Model documentation ↗

      DeepInfra · Catalog ↗

      GitHub · Catalog ↗

      Google Cloud · Catalog ↗

      Hugging Face · Catalog ↗

      Hugging Face · Catalog ↗

      OpenRouter · Catalog ↗

      OpenRouter · Catalog ↗

      crfm.stanford.edu · Catalog ↗

      Source links

      1–10 / 10

      Explore more models

      Models

      Loading
      EVIDENCE / COMPARISON

      Benchmarks

      Explore every benchmark

      Loading

      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

      Install TensorTrace