Qwen: Qwen3 30B A3B Instruct 2507

Qwen3-30B-A3B-Instruct-2507 is a 30.5B-parameter mixture-of-experts language model from Qwen, with 3.3B active parameters per inference. It operates in non-thinking mode and is designed for high-qual…

qwen/qwen3-30b-a3b-instruct-2507

Qwen3-30B-A3B-Instruct-2507 is a 30.5B-parameter mixture-of-experts language model from Qwen, with 3.3B active parameters per inference. It operates in non-thinking mode and is designed for high-quality instruction following, multilingual understanding, and agentic tool use. Post-trained on instruction data, it demonstrates competitive performance across reasoning (AIME, ZebraLogic), coding (MultiPL-E, LiveCodeBench), and alignment (IFEval, WritingBench) benchmarks. It outperforms its non-instruct variant on subjective and open-ended tasks while retaining strong factual and coding performance.

Specifications & capabilities

Input
text
Output
text
Context window · tokens
262144
Knowledge cutoff
2025-06-30
Published model ID
Qwen3-30B-A3B-Instruct-2507
Capabilities
promptCaching, structuredOutputs, toolCalling
Description source ↗

Evaluations

1–20 / 99

OpenRouter Benchmarks

64.141%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Ambient
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
4
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Ambient
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-09T15:00:06.492Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.641414
cost_per_task_usd
0.00284089898989899
output_tokens_per_task
9266.040404040405
latency_seconds_per_task
209.35264646464648
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
4
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

38.667%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Ambient
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
4
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Ambient
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-09T15:37:38.473Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.386667
cost_per_task_usd
0.014510460000000001
output_tokens_per_task
4106.96
latency_seconds_per_task
64.33254
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
4
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

BFCL

88.92%

source id
bfcl
leaderboard name
BFCL Non-Live AST
dimension
toolUse
metric
Non-live AST accuracy
rank
10
Original result ↗

OpenRouter Benchmarks

63.061%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · DekaLLM
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
12
sample size
282
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
DekaLLM
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-23T09:29:14.171Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
282
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.6306148156028369
accuracy_std_dev
0.05281625698168657
cost_per_task_usd
0.0037584255319148938
output_tokens_per_task
12308.468085106382
latency_seconds_per_task
208.45259574468085
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
12
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

36%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · PrimeIntellect
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
17
sample size
50
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
PrimeIntellect
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-04T03:00:10.237Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
50
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.36
cost_per_task_usd
0.041000339999999996
output_tokens_per_task
4212.74
latency_seconds_per_task
322.3453
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
21
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

38%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · CoreWeave
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
18
sample size
100
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
CoreWeave
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-08-29T16:13:12.633Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
100
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.38
accuracy_std_dev
0.0066669999999999785
cost_per_task_usd
0.02802411
output_tokens_per_task
4093.08
latency_seconds_per_task
79.73777
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
22
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

62.121%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · PrimeIntellect
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
19
sample size
198
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
1
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
PrimeIntellect
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-04T03:09:40.101Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
198
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.621212
cost_per_task_usd
0.00552190909090909
output_tokens_per_task
10415.944444444445
latency_seconds_per_task
174.05564141414143
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
22
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

61.911%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Nebius
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
22
sample size
792
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-23T08:08:11.470Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
792
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.6191077500000001
accuracy_std_dev
0.015215861565731303
cost_per_task_usd
0.004869287878787879
output_tokens_per_task
15960.142676767677
latency_seconds_per_task
286.46726893939393
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

36.667%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Nebius
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
22
sample size
200
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Nebius
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-23T08:14:36.871Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
200
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.36666650000000006
accuracy_std_dev
0.021602468996621663
cost_per_task_usd
0.033141745
output_tokens_per_task
4403.68
latency_seconds_per_task
134.75370999999998
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
23
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

BFCL

93.75%

source id
bfcl
leaderboard name
BFCL Relevance Detection
dimension
toolUse
metric
Relevance detection
rank
24
Original result ↗

OpenRouter Benchmarks

41.361%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · StreamLake
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
24
sample size
191
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
StreamLake
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:17:21.424Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
191
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.4136123821989529
accuracy_std_dev
0.024976865859981302
cost_per_task_usd
0.009809837696335078
output_tokens_per_task
2670.895287958115
latency_seconds_per_task
75.93989528795812
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
25
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

62.458%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · CoreWeave
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
25
sample size
396
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
2
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
CoreWeave
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-08-29T18:20:19.448Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
396
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.6245795
accuracy_std_dev
0.011784500000000031
cost_per_task_usd
0.004032424242424243
output_tokens_per_task
13170.60101010101
latency_seconds_per_task
154.42669696969696
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
27
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

64.646%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Alibaba
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
27
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:51:25.541Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.646465
accuracy_std_dev
0.007273551310513057
cost_per_task_usd
0.004810424242424242
output_tokens_per_task
9051.29797979798
latency_seconds_per_task
80.19498484848485
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

40.667%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · SiliconFlow
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
27
sample size
150
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
SiliconFlow
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:13:56.972Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
150
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.4066666666666667
accuracy_std_dev
0.02494438257849294
cost_per_task_usd
0.028486626666666664
output_tokens_per_task
4212.966666666666
latency_seconds_per_task
245.94252666666668
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
28
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

35.111%

source id
openRouterBenchmarks
leaderboard name
tau2-bench Verified Airline · Accuracy · Alibaba
dimension
agentic
metric
tau2-bench Verified Airline · Accuracy
rank
28
sample size
150
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
tau_bench_verified_airline
name
tau2-bench Verified Airline
record_kind
aggregate
provider_name
Alibaba
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:05:34.166Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
tau2-bench Verified Airline · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
150
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/tau2-bench-airline
measurements
accuracy
0.351111
accuracy_std_dev
0.013698950130089059
cost_per_task_usd
0.057490099999999995
output_tokens_per_task
4393.28
latency_seconds_per_task
70.34937333333333
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
29
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

63.187%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · SiliconFlow
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
30
sample size
594
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
3
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
SiliconFlow
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T17:03:50.406Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
594
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.6318743333333333
accuracy_std_dev
0.008054489114083432
cost_per_task_usd
0.0027108804713804713
output_tokens_per_task
8805.387205387206
latency_seconds_per_task
284.8023080808081
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
34
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

OpenRouter Benchmarks

62.879%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · StreamLake
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
30
sample size
792
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
4
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
StreamLake
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-03T16:47:32.559Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
792
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.62878825
accuracy_std_dev
0.0068900206957236525
cost_per_task_usd
0.001630263888888889
output_tokens_per_task
8242.138888888889
latency_seconds_per_task
77.29025
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
30
benchmark_evaluation
true
openrouter_permaslug
qwen/qwen3-30b-a3b-instruct-2507
source model id
qwen/qwen3-30b-a3b-instruct-2507
Original result ↗

BFCL

85.77%

source id
bfcl
leaderboard name
BFCL Non-Live AST
dimension
toolUse
metric
Non-live AST accuracy
rank
37
Original result ↗

Published prices

1–5 / 5

Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

$0.09

Inference provider
siliconFlow
Platform
SiliconFlow
Unit
input_tokens_per_million
Currency
USD
Billing conditions
Billing label
Input tokens (per 1M)

Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

$0.30

Inference provider
siliconFlow
Platform
SiliconFlow
Unit
output_tokens_per_million
Currency
USD
Billing conditions
Billing label
Output tokens (per 1M)

Provider specifications & access

1–9 / 9

Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

source model id
azure-huggingface/qwen--qwen3-30b-a3b-instruct-2507@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
context length
262144
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

    source model id
    azure-huggingface/qwen--qwen3-30b-a3b-instruct-2507@2
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    license
    apache-2.0
    context length
    262144
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

      source model id
      HuggingFace/qwen-qwen3-30b-a3b-instruct-2507@1
      hosting
      foundry-managed-compute
      platforms
      • Microsoft Foundry
      license
      apache-2.0
      architecture
      input_modalities
      • text
      output_modalities
      • text
      capabilities

        Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

        source model id
        HuggingFace/qwen-qwen3-30b-a3b-instruct-2507@2
        hosting
        foundry-managed-compute
        platforms
        • Microsoft Foundry
        license
        apache-2.0
        architecture
        input_modalities
        • text
        output_modalities
        • text
        capabilities

          Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

          source model id
          HuggingFace/qwen-qwen3-30b-a3b-instruct-2507@3
          hosting
          foundry-managed-compute
          platforms
          • Microsoft Foundry
          license
          apache-2.0
          architecture
          input_modalities
          • text
          output_modalities
          • text
          capabilities

            Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

            source model id
            HuggingFace/qwen-qwen3-30b-a3b-instruct-2507@4
            hosting
            foundry-managed-compute
            platforms
            • Microsoft Foundry
            license
            apache-2.0
            architecture
            input_modalities
            • text
            output_modalities
            • text
            capabilities

              Qwen/Qwen3-30B-A3B-Instruct-2507 ↗

              source model id
              publishers/hf-qwen/models/qwen3-30b-a3b-instruct-2507@001
              hosting
              vertex-self-deployed
              platforms
              • Google Cloud Vertex AI
              architecture
              input_modalities
              • text
              output_modalities
              • text
              capabilities

                Qwen3-30B-A3B-Instruct-2507 ↗

                source model id
                publishers/qwen/models/qwen3@qwen3-30b-a3b-instruct-2507
                hosting
                vertex-self-deployed
                platforms
                • Google Cloud Vertex AI
                architecture
                input_modalities
                • text
                output_modalities
                • text
                capabilities

                  Qwen3-30B-A3B-Instruct-2507 ↗

                  source model id
                  Qwen/Qwen3-30B-A3B-Instruct-2507
                  hosting
                  siliconflow-hosted
                  platforms
                  • SiliconFlow
                  max output tokens
                  262144
                  architecture
                  input_modalities
                  • text
                  output_modalities
                  • text
                  capabilities
                  • structuredOutputs
                  • toolCalling

                  Safety assessments

                  1–1 / 1

                  Foundation Model Transparency Index 2025 ↗

                  26 / 100

                  source id
                  stanfordFMTI
                  canonical model id
                  qwen/qwen3-30b-a3b-instruct-2507
                  taxonomy
                  transparency
                  hazard category
                  transparency
                  hazard name
                  Organization transparency
                  policy area
                  Alibaba
                  assessment type
                  organization-level disclosure
                  metric
                  disclosure_score
                  result label
                  26 / 100
                  sample size
                  100
                  updated at
                  2026-11-09T18:08:05.227Z

                  Source specification observations

                  1–7 / 7

                  azureFoundry ↗

                  field
                  context_length
                  value
                  262144
                  normalized value
                  262144
                  scope
                  source-model
                  source id
                  azureFoundry
                  confidence
                  1

                  azureFoundry ↗

                  field
                  context_length
                  value
                  262144
                  normalized value
                  262144
                  scope
                  source-model
                  source id
                  azureFoundry
                  confidence
                  1

                  openRouter ↗

                  field
                  context_length
                  value
                  262144
                  normalized value
                  262144
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  siliconFlow ↗

                  field
                  context_length
                  value
                  262K
                  scope
                  source-model
                  source id
                  siliconFlow
                  confidence
                  1

                  openRouter ↗

                  field
                  knowledge_cutoff
                  reported field
                  knowledge_cutoff
                  value
                  2025-06-30
                  normalized value
                  2025-06-30
                  precision
                  day
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  openRouter ↗

                  field
                  max_output_tokens
                  value
                  235929
                  normalized value
                  235929
                  scope
                  source-model
                  source id
                  openRouter
                  confidence
                  1

                  siliconFlow ↗

                  field
                  max_output_tokens
                  value
                  262K
                  scope
                  source-model
                  source id
                  siliconFlow
                  confidence
                  1

                  Original sources

                  21–30 / 30

                  GitHub · Catalog ↗

                  Google Cloud · Catalog ↗

                  Hugging Face · Catalog ↗

                  Hugging Face · Catalog ↗

                  LMArena · Catalog ↗

                  Microsoft Foundry · Catalog ↗

                  OpenRouter · Catalog ↗

                  OpenRouter · Catalog ↗

                  SiliconFlow · Catalog ↗

                  gorilla.cs.berkeley.edu · Catalog ↗

                  Source links

                  1–11 / 11

                  Hugging Face model card

                  1–1 / 1

                  Explore more models

                  Models

                  Loading
                  EVIDENCE / COMPARISON

                  Benchmarks

                  Explore every benchmark

                  Loading

                  These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                  Install TensorTrace