Qwen: Qwen3.8 27B

Qwen3.8 27B is an open-weight dense vision-language model from Qwen. It is suited for coding, professional workflows, research, multimodal interaction, and long-running agent tasks, with flexible thi…

qwen/qwen3.8-27b-20260814

Qwen3.8 27B is an open-weight dense vision-language model from Qwen. It is suited for coding, professional workflows, research, multimodal interaction, and long-running agent tasks, with flexible thinking that can be enabled or disabled.

Specifications & capabilities

Input
text, image, video
Output
text
Context window · tokens
1024000
Published model ID
Qwen3.8-27B
Capabilities
multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, videoInput, visionInput
Description source ↗

Evaluations

41–56 / 56

OpenRouter Benchmarks

83.098%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Wafer
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
12
sample size
1980
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Wafer
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:20:09.369Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1980
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8309762
accuracy_std_dev
0.004338052761320463
cost_per_task_usd
0.04722357777777777
output_tokens_per_task
15298.06111111111
latency_seconds_per_task
258.25624646464644
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
13
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

75.333%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Alibaba
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
12
sample size
600
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
12
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:41:58.031Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
600
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7533333333333333
accuracy_std_dev
0.029186609056590034
cost_per_task_usd
0.09943221
output_tokens_per_task
12177.723333333333
latency_seconds_per_task
284.246225
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

83.269%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Alibaba
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
17
sample size
2574
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
13
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:25:00.370Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2574
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8326857692307692
accuracy_std_dev
0.011020358329744859
cost_per_task_usd
0.03860720590520591
output_tokens_per_task
14476.024864024865
latency_seconds_per_task
230.7629052059052
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

82.66%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Nebius
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
17
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-04T18:02:01.782Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8265989999999999
cost_per_task_usd
0.051792560606060606
output_tokens_per_task
17173.429292929293
latency_seconds_per_task
150.3321313131313
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

82.623%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · GMICloud
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
17
sample size
352
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
GMICloud
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-11-10T06:16:01.718Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
352
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8262311079545455
accuracy_std_dev
0.0007570000000000077
cost_per_task_usd
0.03419934943181818
output_tokens_per_task
10629.338068181818
latency_seconds_per_task
187.95064204545457
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
26
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

82.038%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · CoreWeave
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
18
sample size
2574
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
13
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
CoreWeave
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:09:30.241Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2574
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8203833076923077
accuracy_std_dev
0.018076297388844354
cost_per_task_usd
0.042996285159285164
output_tokens_per_task
14318.661227661227
latency_seconds_per_task
241.7488038073038
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
27
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

83.177%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Parasail
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
20
sample size
2344
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
12
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Parasail
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:37:28.112Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2344
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.831769061433447
accuracy_std_dev
0.006801018573632123
cost_per_task_usd
0.03448967320819113
output_tokens_per_task
14194.120733788395
latency_seconds_per_task
208.52074530716723
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
36
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

75.747%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Phala
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
21
sample size
2175
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Phala
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:19:36.628Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2175
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7574711434482759
accuracy_std_dev
0.05835625158448523
cost_per_task_usd
0.014400611954022986
output_tokens_per_task
6123.435402298851
latency_seconds_per_task
105.76068091954023
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
24
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

71.6%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Venice
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
22
sample size
500
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
10
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Venice
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T12:46:35.433Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
500
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.7159998000000001
accuracy_std_dev
0.03295793548085197
cost_per_task_usd
0.24450749400000002
output_tokens_per_task
21575.564
latency_seconds_per_task
314.057314
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
38
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

79.552%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Venice
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
24
sample size
2371
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
12
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Venice
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:22:42.983Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2371
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.7955151838886546
accuracy_std_dev
0.021167952217358864
cost_per_task_usd
0.0645802066638549
output_tokens_per_task
20212.99072121468
latency_seconds_per_task
194.46245592576972
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
39
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

83.142%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · DeepInfra
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
26
sample size
1564
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
8
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
DeepInfra
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:48:50.088Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1564
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8314150997442455
accuracy_std_dev
0.012107134010057875
cost_per_task_usd
0.02770090409207161
output_tokens_per_task
14708.68989769821
latency_seconds_per_task
583.3779558823529
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
71
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

82.706%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Novita
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
29
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Novita
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:36:47.145Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8270583636363636
accuracy_std_dev
0.00941702237510474
cost_per_task_usd
0.04288225068870524
output_tokens_per_task
14178.259871441689
latency_seconds_per_task
229.94266437098256
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
59
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

OpenRouter Benchmarks

81.941%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · auto-routing
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
73
sample size
2178
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
11
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
auto-routing
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T08:09:30.951Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
2178
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.8194060909090909
accuracy_std_dev
0.013884218840464785
cost_per_task_usd
0.033594433884297524
output_tokens_per_task
13897.44214876033
latency_seconds_per_task
211.99446005509643
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
132
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3.8-27b-20260814
source model id
qwen/qwen3.8-27b-20260814
Original result ↗

Published prices

1–20 / 33

@cf/qwen/qwen3.8-27b ↗

$0.05

Inference provider
cloudflareWorkersAI
Platform
Cloudflare Workers AI
Unit
input_tokens_per_million
Currency
USD
Billing conditions
Billing label
per M cached input tokens

@cf/qwen/qwen3.8-27b ↗

$3.20

Inference provider
cloudflareWorkersAI
Platform
Cloudflare Workers AI
Unit
output_tokens_per_million
Currency
USD
Billing conditions
Billing label
per M output tokens

qwen/qwen3.8-27b ↗

$0.04

Inference provider
openRouter
Platform
OpenRouter
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions

qwen/qwen3.8-27b ↗

$0.15

Inference provider
openRouter
Platform
OpenRouter
Unit
input_tokens_per_million
Currency
USD
Billing conditions

qwen/qwen3.8-27b ↗

$1.88

Inference provider
openRouter
Platform
OpenRouter
Unit
output_tokens_per_million
Currency
USD
Billing conditions

qwen/qwen3.8-27b:batch ↗

$0.04

Inference provider
openRouter
Platform
OpenRouter
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
mode
batch

qwen/qwen3.8-27b:batch ↗

$0.16

Inference provider
openRouter
Platform
OpenRouter
Unit
input_tokens_per_million
Currency
USD
Billing conditions
mode
batch

qwen/qwen3.8-27b:batch ↗

$2.00

Inference provider
openRouter
Platform
OpenRouter
Unit
output_tokens_per_million
Currency
USD
Billing conditions
mode
batch

alibaba/qwen3.8-27b ↗

$0.10

Inference provider
vercelAIGatewayCatalog
Platform
alibaba
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
alibaba | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$0.63

Inference provider
vercelAIGatewayCatalog
Platform
alibaba
Unit
cache_write_tokens_per_million
Currency
USD
Billing conditions
endpoint
alibaba | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cache write / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$0.50

Inference provider
vercelAIGatewayCatalog
Platform
alibaba
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
alibaba | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$3.00

Inference provider
vercelAIGatewayCatalog
Platform
alibaba
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
alibaba | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$0.99

Inference provider
vercelAIGatewayCatalog
Platform
cerebras
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
cerebras | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$0.99

Inference provider
vercelAIGatewayCatalog
Platform
cerebras
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
cerebras | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$1.49

Inference provider
vercelAIGatewayCatalog
Platform
cerebras
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
cerebras | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$0.04

Inference provider
vercelAIGatewayCatalog
Platform
deepinfra
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
deepinfra | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$0.15

Inference provider
vercelAIGatewayCatalog
Platform
deepinfra
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
deepinfra | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$1.88

Inference provider
vercelAIGatewayCatalog
Platform
deepinfra
Unit
output_tokens_per_million
Currency
USD
Billing conditions
endpoint
deepinfra | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Output tokens / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$0.08

Inference provider
vercelAIGatewayCatalog
Platform
novita
Unit
cache_read_tokens_per_million
Currency
USD
Billing conditions
endpoint
novita | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Cached input / 1M
Pricing via
Vercel AI Gateway

alibaba/qwen3.8-27b ↗

$0.42

Inference provider
vercelAIGatewayCatalog
Platform
novita
Unit
input_tokens_per_million
Currency
USD
Billing conditions
endpoint
novita | alibaba/qwen3.8-27b
model_type
language
billing_scope
via Vercel AI Gateway
Billing label
Input tokens / 1M
Pricing via
Vercel AI Gateway

Provider specifications & access

1–10 / 10

Qwen3.8 27B ↗

source model id
alibaba/qwen3.8-27b
hosting
vercel-gateway
platforms
  • Vercel AI Gateway
context length
262144
max output tokens
262144
architecture
input_modalities
  • text
  • image
  • pdf
  • video
output_modalities
  • text
capabilities
  • promptCaching
  • reasoning
  • structuredOutputs
  • toolCalling
  • videoInput
  • visionInput

Qwen/Qwen3.8-27B ↗

source model id
azure-huggingface/qwen--qwen3.8-27b@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
context length
262144
architecture
input_modalities
  • text
  • image
  • video
output_modalities
  • text
capabilities

    Qwen/Qwen3.8-27B ↗

    source model id
    azure-huggingface/qwen--qwen3.8-27b@2
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    license
    apache-2.0
    context length
    262144
    architecture
    input_modalities
    • text
    • image
    • video
    output_modalities
    • text
    capabilities

      Qwen/Qwen3.8-27B ↗

      source model id
      azure-huggingface/qwen--qwen3.8-27b@3
      hosting
      foundry-managed-compute
      platforms
      • Microsoft Foundry
      license
      apache-2.0
      context length
      262144
      architecture
      input_modalities
      • text
      • image
      • video
      output_modalities
      • text
      capabilities

        Qwen/Qwen3.8-27B ↗

        source model id
        azure-huggingface/qwen--qwen3.8-27b@4
        hosting
        foundry-managed-compute
        platforms
        • Microsoft Foundry
        license
        apache-2.0
        context length
        262144
        architecture
        input_modalities
        • text
        • image
        • video
        output_modalities
        • text
        capabilities

          Qwen/Qwen3.8-27B ↗

          source model id
          azure-huggingface/qwen--qwen3.8-27b@5
          hosting
          foundry-managed-compute
          platforms
          • Microsoft Foundry
          license
          apache-2.0
          context length
          262144
          architecture
          input_modalities
          • text
          • image
          • video
          output_modalities
          • text
          capabilities

            Qwen/Qwen3.8-27B ↗

            source model id
            azure-huggingface/qwen--qwen3.8-27b@6
            hosting
            foundry-managed-compute
            platforms
            • Microsoft Foundry
            license
            apache-2.0
            context length
            262144
            architecture
            input_modalities
            • text
            • image
            • video
            output_modalities
            • text
            capabilities

              Qwen/Qwen3.8-27B ↗

              source model id
              azure-huggingface/qwen--qwen3.8-27b@7
              hosting
              foundry-managed-compute
              platforms
              • Microsoft Foundry
              license
              apache-2.0
              context length
              262144
              architecture
              input_modalities
              • text
              • image
              • video
              output_modalities
              • text
              capabilities

                qwen3.8-27b ↗

                source model id
                @cf/qwen/qwen3.8-27b
                hosting
                cloudflare-hosted
                platforms
                • Cloudflare Workers AI
                context length
                262144
                architecture
                input_modalities
                • text
                • image
                output_modalities
                • text
                capabilities
                • toolCalling

                Qwen/Qwen3.8-27B ↗

                source model id
                publishers/hf-qwen/models/qwen3.8-27b@001
                hosting
                vertex-self-deployed
                platforms
                • Google Cloud Vertex AI
                architecture
                input_modalities
                • image
                • text
                output_modalities
                • text
                capabilities
                • multimodal
                • visionInput

                Safety assessments

                1–1 / 1

                Foundation Model Transparency Index 2025 ↗

                26 / 100

                source id
                stanfordFMTI
                canonical model id
                qwen/qwen3.8-27b-20260814
                taxonomy
                transparency
                hazard category
                transparency
                hazard name
                Organization transparency
                policy area
                Alibaba
                assessment type
                organization-level disclosure
                metric
                disclosure_score
                result label
                26 / 100
                sample size
                100
                updated at
                2026-11-09T18:08:05.227Z

                Source specification observations

                1–13 / 13

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                azureFoundry ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                azureFoundry
                confidence
                1

                openRouter ↗

                field
                context_length
                value
                1024000
                normalized value
                1024000
                scope
                source-model
                source id
                openRouter
                confidence
                1

                openRouter ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                openRouter
                confidence
                1

                vercelAIGatewayCatalog ↗

                field
                context_length
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                vercelAIGatewayCatalog
                confidence
                1

                openRouter ↗

                field
                max_output_tokens
                value
                235929
                normalized value
                235929
                scope
                source-model
                source id
                openRouter
                confidence
                1

                openRouter ↗

                field
                max_output_tokens
                value
                235929
                normalized value
                235929
                scope
                source-model
                source id
                openRouter
                confidence
                1

                vercelAIGatewayCatalog ↗

                field
                max_output_tokens
                value
                262144
                normalized value
                262144
                scope
                source-model
                source id
                vercelAIGatewayCatalog
                confidence
                1

                Original sources

                1–20 / 25

                Hugging Face · Qwen/Qwen3.8-27B ↗

                Cloudflare AI · Model page ↗

                Google Cloud · Model page ↗

                Microsoft Foundry · Model page ↗

                Microsoft Foundry · Model page ↗

                Microsoft Foundry · Model page ↗

                Microsoft Foundry · Model page ↗

                Microsoft Foundry · Model page ↗

                Microsoft Foundry · Model page ↗

                Microsoft Foundry · Model page ↗

                OpenRouter · Model page ↗

                OpenRouter · Model page ↗

                Vercel AI Gateway · Model page ↗

                Microsoft Foundry · Model documentation ↗

                GitHub · Source snapshot ↗

                Cloudflare AI · Catalog ↗

                Epoch AI · Catalog ↗

                GitHub · Catalog ↗

                Google Cloud · Catalog ↗

                Hugging Face · Catalog ↗

                Source links

                1–11 / 11

                Hugging Face model card

                1–1 / 1

                Qwen/Qwen3.8-27B ↗

                downloads
                7232703
                likes
                19592
                pipeline tag
                image-text-to-text
                library name
                transformers
                License
                apache-2.0

                Explore more models

                Models

                Loading
                EVIDENCE / COMPARISON

                Benchmarks

                Explore every benchmark

                Loading

                These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                Install TensorTrace