Qwen: Qwen3.5 397B A17B

The Qwen3.5 series 397B-A17B native vision-language model is built on a hybrid architecture that integrates a linear attention mechanism with a sparse mixture-of-experts model, achieving higher infer…

qwen/qwen3.5-397b-a17b-20260216

The Qwen3.5 series 397B-A17B native vision-language model is built on a hybrid architecture that integrates a linear attention mechanism with a sparse mixture-of-experts model, achieving higher inference efficiency. It delivers state-of-the-art performance comparable to leading-edge models across a wide range of tasks, including language understanding, logical reasoning, code generation, agent-based tasks, image understanding, video understanding, and graphical user interface (GUI) interactions. With its robust code-generation and agent capabilities, the model exhibits strong generalization across diverse agent.

Specifications & capabilities

Input
text, image, video
Output
text
Context window · tokens
262144
Published model ID
Qwen3.5-397B-A17B
Capabilities
multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗

Evaluations

1–20 / 31

OpenRouter Benchmarks

79.952%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · DigitalOcean
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
1
sample size
350
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
DigitalOcean
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T20:31:41.691Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
350
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7995237142857143
accuracy_std_dev
0.010754053185583914
cost_per_task_usd
0.14178823428571427
output_tokens_per_task
28444.37714285714
latency_seconds_per_task
2712.9529314285714
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
20
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

79.789%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Phala
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
1
sample size
348
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Phala
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T19:01:09.612Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
348
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7978929137931035
accuracy_std_dev
0.03017695806811711
cost_per_task_usd
0.20533228735632184
output_tokens_per_task
35305.502873563215
latency_seconds_per_task
772.7763793103449
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

79.37%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Alibaba
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
1
sample size
349
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T19:12:41.306Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
349
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7936961919770773
accuracy_std_dev
0.02409553422592355
cost_per_task_usd
0.18211524641833812
output_tokens_per_task
30422.72206303725
latency_seconds_per_task
451.04138395415475
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

78.451%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Parasail
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
1
sample size
297
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
6
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Parasail
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T19:27:20.607Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
297
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7845118821548821
accuracy_std_dev
0.03356639684143456
cost_per_task_usd
0.19079432323232323
output_tokens_per_task
30741.198653198655
latency_seconds_per_task
638.8991245791246
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
28
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

77.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · GMICloud
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
1
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
GMICloud
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-08-29T16:58:30.455Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.773333
cost_per_task_usd
0.26629643999999997
output_tokens_per_task
27876.64
latency_seconds_per_task
438.74158
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

77.143%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Venice
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
1
sample size
350
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Venice
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T19:49:31.051Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
350
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.771428857142857
accuracy_std_dev
0.019095107469331168
cost_per_task_usd
0.3362465285714286
output_tokens_per_task
28432.711428571427
latency_seconds_per_task
611.0598085714286
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
38
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenCompass LLM

88.4%

source id
openCompass
leaderboard name
GPQA-Diamond
dimension
general
metric
GPQA-Diamond
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
GPQA-Diamond
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
Qwen3.5-397B-A17B
Original result ↗

OpenCompass LLM

87.6%

source id
openCompass
leaderboard name
MMLU-Pro
dimension
general
metric
MMLU-Pro
rank
2
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
MMLU-Pro
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
Qwen3.5-397B-A17B
Original result ↗

OpenRouter Benchmarks

79.143%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · auto-routing
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
3
sample size
350
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T19:47:27.263Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
350
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7914285714285715
accuracy_std_dev
0.01807021076529105
cost_per_task_usd
0.20560172000000002
output_tokens_per_task
32379.805714285714
latency_seconds_per_task
767.1289885714285
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
128
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenCompass LLM

78.4%

source id
openCompass
leaderboard name
Average
dimension
general
metric
Average
rank
4
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
Average
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
Qwen3.5-397B-A17B
Original result ↗

OpenRouter Benchmarks

76.095%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · AtlasCloud
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
4
sample size
350
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
AtlasCloud
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T19:26:18.381Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
350
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7609525714285714
accuracy_std_dev
0.07435901694742715
cost_per_task_usd
0.2507432571428571
output_tokens_per_task
29106.445714285714
latency_seconds_per_task
493.9388057142857
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenCompass LLM

83%

source id
openCompass
leaderboard name
LiveCodeBenchV6
dimension
general
metric
LiveCodeBenchV6
rank
5
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
LiveCodeBenchV6
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
Qwen3.5-397B-A17B
Original result ↗

OpenCompass LLM

27.5%

source id
openCompass
leaderboard name
HLE
dimension
general
metric
HLE
rank
6
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
HLE
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
Qwen3.5-397B-A17B
Original result ↗

OpenRouter Benchmarks

88.279%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Parasail
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
8
sample size
1294
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Parasail
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:03:21.450Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1294
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.882792129057187
accuracy_std_dev
0.012193080659094465
cost_per_task_usd
0.06525332148377125
output_tokens_per_task
18028.40030911901
latency_seconds_per_task
302.1959992272025
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
36
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

87.47%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Phala
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
9
sample size
1386
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Phala
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:15:54.664Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1386
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8746991428571429
accuracy_std_dev
0.005681157161518922
cost_per_task_usd
0.08339443073593074
output_tokens_per_task
23715.052669552668
latency_seconds_per_task
446.73863275613274
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
24
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

87.133%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · AtlasCloud
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
9
sample size
1386
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
AtlasCloud
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:09:19.242Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1386
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8713324285714285
accuracy_std_dev
0.009851679854683843
cost_per_task_usd
0.07396083838383838
output_tokens_per_task
21002.70779220779
latency_seconds_per_task
301.9372424242424
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
30
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

87.239%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Alibaba
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
10
sample size
1583
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
8
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:14:42.903Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1583
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8723940682248895
accuracy_std_dev
0.00799901581348451
cost_per_task_usd
0.0495316885660139
output_tokens_per_task
21030.309538850284
latency_seconds_per_task
303.24233101705624
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenRouter Benchmarks

85.922%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · StreamLake
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
10
sample size
1584
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
8
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
StreamLake
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:04:29.122Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1584
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.859217
accuracy_std_dev
0.016791584253726632
cost_per_task_usd
0.09269683143939395
output_tokens_per_task
25619.8952020202
latency_seconds_per_task
184.47000315656567
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
30
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

OpenCompass LLM

92.3%

source id
openCompass
leaderboard name
AIME2025
dimension
general
metric
AIME2025
rank
11
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
OverallTable
catalog
open
snapshot
month
REALTIME
fileName
assets/research-rank/research-data.REALTIME.20260309.json
updateTime
2026-03-09
columnFileName
assets/research-rank/research-column.REALTIME.20260309.json
metric_kind
AIME2025
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
0-data.json
source_revision
117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
metric_population
48
benchmark_evaluation
true
source model id
Qwen3.5-397B-A17B
Original result ↗

OpenRouter Benchmarks

87.71%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · GMICloud
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
11
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
GMICloud
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-08-29T19:12:43.653Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.877104
cost_per_task_usd
0.07509321212121212
output_tokens_per_task
20738.459595959597
latency_seconds_per_task
263.1753282828283
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
26
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.5-397b-a17b-20260216
source model id
qwen/qwen3.5-397b-a17b-20260216
Original result ↗

Published prices

1–5 / 5

alibaba/qwen3.5-397b-a17b ↗

$0.60

Inference provider
cloudflareWorkersAI
Platform
Alibaba
Unit
input_tokens_per_million
Currency
USD
Billing conditions
Billing label
Input tokens (per 1M)
Pricing via
Cloudflare AI Gateway

alibaba/qwen3.5-397b-a17b ↗

$3.60

Inference provider
cloudflareWorkersAI
Platform
Alibaba
Unit
output_tokens_per_million
Currency
USD
Billing conditions
Billing label
Output tokens (per 1M)
Pricing via
Cloudflare AI Gateway

qwen/qwen3.5-397b-a17b ↗

$0.22

Inference provider
openRouter
Platform
OpenRouter
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions

qwen/qwen3.5-397b-a17b ↗

$0.45

Inference provider
openRouter
Platform
OpenRouter
Unit
input_tokens_per_million
Currency
USD
Billing conditions

qwen/qwen3.5-397b-a17b ↗

$3.00

Inference provider
openRouter
Platform
OpenRouter
Unit
output_tokens_per_million
Currency
USD
Billing conditions

Provider specifications & access

1–8 / 8

Qwen 3.5 397B A17B ↗

source model id
alibaba/qwen3.5-397b-a17b
hosting
third-party
platforms
  • Alibaba
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    Qwen/Qwen3.5-397B-A17B ↗

    source model id
    azure-huggingface/qwen--qwen3.5-397b-a17b@1
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    license
    apache-2.0
    context length
    262144
    architecture
    input_modalities
    • text
    • image
    • video
    output_modalities
    • text
    capabilities

      Qwen/Qwen3.5-397B-A17B ↗

      source model id
      azure-huggingface/qwen--qwen3.5-397b-a17b@2
      hosting
      foundry-managed-compute
      platforms
      • Microsoft Foundry
      license
      apache-2.0
      context length
      262144
      architecture
      input_modalities
      • text
      • image
      • video
      output_modalities
      • text
      capabilities

        Qwen/Qwen3.5-397B-A17B ↗

        source model id
        azure-huggingface/qwen--qwen3.5-397b-a17b@3
        hosting
        foundry-managed-compute
        platforms
        • Microsoft Foundry
        license
        apache-2.0
        context length
        262144
        architecture
        input_modalities
        • text
        • image
        • video
        output_modalities
        • text
        capabilities

          Qwen/Qwen3.5-397B-A17B ↗

          source model id
          azure-huggingface/qwen--qwen3.5-397b-a17b@4
          hosting
          foundry-managed-compute
          platforms
          • Microsoft Foundry
          license
          apache-2.0
          context length
          262144
          architecture
          input_modalities
          • text
          • image
          • video
          output_modalities
          • text
          capabilities

            Qwen/Qwen3.5-397B-A17B ↗

            source model id
            azure-huggingface/qwen--qwen3.5-397b-a17b@5
            hosting
            foundry-managed-compute
            platforms
            • Microsoft Foundry
            license
            apache-2.0
            context length
            262144
            architecture
            input_modalities
            • text
            • image
            • video
            output_modalities
            • text
            capabilities

              Qwen3.5 ↗

              source model id
              publishers/qwen/models/qwen3-5@qwen3.5-397b-a17b
              hosting
              vertex-self-deployed
              platforms
              • Google Cloud Vertex AI
              architecture
              input_modalities
              • text
              output_modalities
              • text
              capabilities

                Qwen3.5-397B-A17B ↗

                source model id
                Qwen/Qwen3.5-397B-A17B
                hosting
                siliconflow-hosted
                platforms
                • SiliconFlow
                architecture
                input_modalities
                • text
                • image
                output_modalities
                • text
                capabilities
                • structuredOutputs
                • toolCalling

                Safety assessments

                1–1 / 1

                Foundation Model Transparency Index 2025 ↗

                26 / 100

                source id
                stanfordFMTI
                canonical model id
                qwen/qwen3.5-397b-a17b-20260216
                taxonomy
                transparency
                hazard category
                transparency
                hazard name
                Organization transparency
                policy area
                Alibaba
                assessment type
                organization-level disclosure
                metric
                disclosure_score
                result label
                26 / 100
                sample size
                100
                updated at
                2026-11-09T18:08:05.227Z

                Source specification observations

                1–8 / 8

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                openRouter ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                openRouter
                confidence
                1

                siliconFlow ↗

                field
                context_length
                value
                262K
                scope
                source-model
                source id
                siliconFlow
                confidence
                1

                openRouter ↗

                field
                max_output_tokens
                value
                81920
                normalized value
                81920
                scope
                source-model
                source id
                openRouter
                confidence
                1

                Original sources

                21–26 / 26

                Hugging Face · Catalog ↗

                Hugging Face · Catalog ↗

                Microsoft Foundry · Catalog ↗

                OpenRouter · Catalog ↗

                OpenRouter · Catalog ↗

                SiliconFlow · Catalog ↗

                Source links

                1–12 / 12

                Hugging Face model card

                1–1 / 1

                Explore more models

                Models

                Loading
                EVIDENCE / COMPARISON

                Benchmarks

                Explore every benchmark

                Loading

                These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                Install TensorTrace