Qwen: Qwen3 Coder 480B A35B

Qwen3-Coder-480B-A35B-Instruct is a Mixture-of-Experts (MoE) code generation model developed by the Qwen team. It is optimized for agentic coding tasks such as function calling, tool use, and long-co…

qwen/qwen3-coder-480b-a35b-07-25

Qwen3-Coder-480B-A35B-Instruct is a Mixture-of-Experts (MoE) code generation model developed by the Qwen team. It is optimized for agentic coding tasks such as function calling, tool use, and long-context reasoning over repositories. The model features 480 billion total parameters, with 35 billion active per forward pass (8 out of 160 experts).

Pricing for the Alibaba endpoints varies by context length. Once a request is greater than 128k input tokens, the higher pricing is used.

Specifications & capabilities

Input
text
Output
text
Context window · tokens
262144
Knowledge cutoff
2025-06-30
Published model ID
Qwen3-Coder-480B-A35B-Instruct
Capabilities
promptCaching, structuredOutputs, toolCalling
Description source ↗

Evaluations

1–13 / 13

OpenRouter Benchmarks

40.889%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Alibaba
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
25
sample size
150
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:14:53.189Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
150
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.40888866666666673
accuracy_std_dev
0.0599587430173634
cost_per_task_usd
0.6742300533333333
output_tokens_per_task
3782.766666666667
latency_seconds_per_task
128.24233999999998
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

60.213%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Alibaba
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
28
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T17:43:44.972Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.6021326666666665
accuracy_std_dev
0.003459479954495376
cost_per_task_usd
0.011048464646464645
output_tokens_per_task
2103.8636363636365
latency_seconds_per_task
31.195183501683502
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

61.251%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Google
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
36
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Google
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T17:03:50.598Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.6125139999999999
accuracy_std_dev
0.004677949194536715
cost_per_task_usd
0.004050520202020202
output_tokens_per_task
2151.1430976430975
latency_seconds_per_task
21.444562289562292
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
40
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

41.889%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Google
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
36
sample size
150
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Google
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:12:29.903Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
150
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.4188886666666667
accuracy_std_dev
0.03424661828885039
cost_per_task_usd
0.052051466666666664
output_tokens_per_task
2546.826666666667
latency_seconds_per_task
38.040333333333336
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
36
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

59.764%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Venice
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
37
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Venice
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:59:44.383Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.5976433333333334
accuracy_std_dev
0.0036366213562713575
cost_per_task_usd
0.003224691919191919
output_tokens_per_task
2072.1902356902356
latency_seconds_per_task
76.58148821548822
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
39
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

41.556%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Venice
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
37
sample size
150
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Venice
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:14:29.926Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
150
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.41555566666666666
accuracy_std_dev
0.015713798296048267
cost_per_task_usd
0.01727216
output_tokens_per_task
2838.9866666666667
latency_seconds_per_task
121.60727333333332
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
38
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

40.889%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Novita
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
47
sample size
150
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Novita
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:19:14.095Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
150
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.40888899999999995
accuracy_std_dev
0.08368091364622321
cost_per_task_usd
0.2754822733333333
output_tokens_per_task
4063.2466666666664
latency_seconds_per_task
135.11914000000002
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
52
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

60.606%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Novita
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
51
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Novita
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T17:03:29.436Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.6060606666666667
accuracy_std_dev
0.013538129913527765
cost_per_task_usd
0.0036622003367003365
output_tokens_per_task
2163.530303030303
latency_seconds_per_task
32.7390202020202
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
59
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

42.556%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · DeepInfra
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
55
sample size
150
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:12:20.034Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
150
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.42555566666666667
accuracy_std_dev
0.02910172028515764
cost_per_task_usd
0.027489039999999996
output_tokens_per_task
2795.5733333333333
latency_seconds_per_task
68.22156666666666
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
63
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

59.852%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · DeepInfra
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
59
sample size
697
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-08T23:15:07.689Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
697
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.5985173543758967
accuracy_std_dev
0.021740745111426138
cost_per_task_usd
0.002190916786226686
output_tokens_per_task
2056.6857962697272
latency_seconds_per_task
51.577241032998565
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
71
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

46.222%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
109
sample size
150
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:07:50.332Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
150
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.4622223333333333
accuracy_std_dev
0.01911642045177101
cost_per_task_usd
0.03776963333333334
output_tokens_per_task
2732.9733333333334
latency_seconds_per_task
60.587160000000004
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

OpenRouter Benchmarks

60.438%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
117
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:57:23.266Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.6043773333333333
accuracy_std_dev
0.004955863351178606
cost_per_task_usd
0.0035131498316498316
output_tokens_per_task
2122.909090909091
latency_seconds_per_task
35.423020202020204
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-coder-480b-a35b-07-25
source model id
qwen/qwen3-coder-480b-a35b-07-25
Original result ↗

Published prices

1–5 / 5

qwen/qwen3-coder ↗

$0.10

Inference provider
openRouter
Platform
OpenRouter
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions

qwen/qwen3-coder ↗

$0.30

Inference provider
openRouter
Platform
OpenRouter
Unit
input_tokens_per_million
Currency
USD
Billing conditions

qwen/qwen3-coder ↗

$1.00

Inference provider
openRouter
Platform
OpenRouter
Unit
output_tokens_per_million
Currency
USD
Billing conditions

Provider specifications & access

1–3 / 3

Qwen/Qwen3-Coder-480B-A35B-Instruct ↗

source model id
publishers/hf-qwen/models/qwen3-coder-480b-a35b-instruct@001
hosting
vertex-self-deployed
platforms
  • Google Cloud Vertex AI
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    Qwen3-Coder-480B-A35B-Instruct ↗

    source model id
    publishers/qwen/models/qwen3-coder@qwen3-coder-480b-a35b-instruct
    hosting
    vertex-self-deployed
    platforms
    • Google Cloud Vertex AI
    architecture
    input_modalities
      output_modalities
      • text
      capabilities

        Qwen3-Coder-480B-A35B ↗

        source model id
        Qwen/Qwen3-Coder-480B-A35B-Instruct
        hosting
        siliconflow-hosted
        platforms
        • SiliconFlow
        max output tokens
        262144
        architecture
        input_modalities
        • text
        output_modalities
        • text
        capabilities
        • structuredOutputs
        • toolCalling

        Safety assessments

        1–1 / 1

        Foundation Model Transparency Index 2025 ↗

        26 / 100

        source id
        stanfordFMTI
        canonical model id
        qwen/qwen3-coder-480b-a35b-07-25
        taxonomy
        transparency
        hazard category
        transparency
        hazard name
        Organization transparency
        policy area
        Alibaba
        assessment type
        organization-level disclosure
        metric
        disclosure_score
        result label
        26 / 100
        sample size
        100
        updated at
        2026-11-09T18:08:05.227Z

        Source specification observations

        1–5 / 5

        openRouter ↗

        field
        context_length
        value
        262144
        normalized value
        262144
        scope
        source-model
        source id
        openRouter
        confidence
        1

        siliconFlow ↗

        field
        context_length
        value
        262K
        scope
        source-model
        source id
        siliconFlow
        confidence
        1

        openRouter ↗

        field
        knowledge_cutoff
        reported field
        knowledge_cutoff
        value
        2025-06-30
        normalized value
        2025-06-30
        precision
        day
        scope
        source-model
        source id
        openRouter
        confidence
        1

        openRouter ↗

        field
        max_output_tokens
        value
        65536
        normalized value
        65536
        scope
        source-model
        source id
        openRouter
        confidence
        1

        siliconFlow ↗

        field
        max_output_tokens
        value
        262K
        scope
        source-model
        source id
        siliconFlow
        confidence
        1

        Original sources

        1–15 / 15

        Qwen · Model reference ↗

        SiliconFlow · Model page ↗

        Hugging Face · Qwen/Qwen3-Coder-480B-A35B-Instruct ↗

        Google Cloud · Model page ↗

        Google Cloud · Model page ↗

        OpenRouter · Model page ↗

        SiliconFlow · Playground ↗

        SiliconFlow · API documentation ↗

        Epoch AI · Catalog ↗

        GitHub · Catalog ↗

        Google Cloud · Catalog ↗

        Hugging Face · Catalog ↗

        OpenRouter · Catalog ↗

        OpenRouter · Catalog ↗

        SiliconFlow · Catalog ↗

        Source links

        1–9 / 9

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace