Qwen: Qwen3 235B A22B Instruct 2507
Qwen3-235B-A22B-Instruct-2507 is a multilingual, instruction-tuned mixture-of-experts language model based on the Qwen3-235B architecture, with 22B active parameters per forward pass. It is optimized…
qwen/qwen3-235b-a22b-07-25
Qwen3-235B-A22B-Instruct-2507 is a multilingual, instruction-tuned mixture-of-experts language model based on the Qwen3-235B architecture, with 22B active parameters per forward pass. It is optimized for general-purpose text generation, including instruction following, logical reasoning, math, code, and tool usage. The model supports a native 262K context length and does not implement "thinking mode" (<think> blocks).
Compared to its base variant, this version delivers significant gains in knowledge coverage, long-context reasoning, coding benchmarks, and alignment with open-ended tasks. It is particularly strong on multilingual understanding, math reasoning (e.g., AIME, HMMT), and alignment evaluations like Arena-Hard and WritingBench.
Specifications & capabilities
- Input
- text
- Output
- text
- Context window · tokens
- 262144
- Knowledge cutoff
- 2025-06-30
- Published model ID
- Qwen3-235B-A22B-Instruct-2507
- Capabilities
- promptCaching, structuredOutputs, toolCalling
Description source ↗Evaluations
61–80 / 160
73.204%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Venice
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 30
- sample size
- 1188
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Venice
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:08:56.903Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1188
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.7320426666666667
- accuracy_std_dev
- 0.006532295121590524
- cost_per_task_usd
- 0.006095060606060606
- output_tokens_per_task
- 7966.672558922559
- latency_seconds_per_task
- 411.9418905723906
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 39
- benchmark_evaluation
- true
- openrouter_permaslug
- qwen/qwen3-235b-a22b-07-25
- source model id
- qwen/qwen3-235b-a22b-07-25
Original result ↗88.3%
- source id
- openCompass
- leaderboard name
- IFEval
- dimension
- general
- metric
- IFEval
- rank
- 31
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- IFEval
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Qwen3-235B-A22B-Instruct-2507
Original result ↗73.694%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Google
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 31
- sample size
- 1978
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 10
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Google
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-17T16:03:44.090Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1978
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.7369396718907987
- accuracy_std_dev
- 0.01294608625956121
- cost_per_task_usd
- 0.006695506572295248
- output_tokens_per_task
- 6995.519716885743
- latency_seconds_per_task
- 136.02757836198182
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 40
- benchmark_evaluation
- true
- openrouter_permaslug
- qwen/qwen3-235b-a22b-07-25
- source model id
- qwen/qwen3-235b-a22b-07-25
Original result ↗79.2%
- source id
- openCompass
- leaderboard name
- MMLU-Pro
- dimension
- general
- metric
- MMLU-Pro
- rank
- 34
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- MMLU-Pro
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Qwen3-235B-A22B-Instruct-2507
Original result ↗61.3%
- source id
- openCompass
- leaderboard name
- Average
- dimension
- general
- metric
- Average
- rank
- 34
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- Average
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Qwen3-235B-A22B-Instruct-2507
Original result ↗69.5%
- source id
- openCompass
- leaderboard name
- AIME2025
- dimension
- general
- metric
- AIME2025
- rank
- 35
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- AIME2025
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Qwen3-235B-A22B-Instruct-2507
Original result ↗43.95%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Google
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 35
- sample size
- 146
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Google
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T16:07:05.221Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 146
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.43949773972602735
- accuracy_std_dev
- 0.06335403547437912
- cost_per_task_usd
- 0.053683828767123284
- output_tokens_per_task
- 3261.6164383561645
- latency_seconds_per_task
- 81.78506849315069
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 36
- benchmark_evaluation
- true
- openrouter_permaslug
- qwen/qwen3-235b-a22b-07-25
- source model id
- qwen/qwen3-235b-a22b-07-25
Original result ↗43.278%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Venice
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 35
- sample size
- 300
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 6
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Venice
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T19:56:10.591Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 300
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.43277766666666667
- accuracy_std_dev
- 0.021638103680518977
- cost_per_task_usd
- 0.03916930333333333
- output_tokens_per_task
- 3498.32
- latency_seconds_per_task
- 287.8242466666666
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 38
- benchmark_evaluation
- true
- openrouter_permaslug
- qwen/qwen3-235b-a22b-07-25
- source model id
- qwen/qwen3-235b-a22b-07-25
Original result ↗73.288%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · Novita
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 41
- sample size
- 594
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- Novita
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T16:57:10.991Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 594
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.7328843333333334
- accuracy_std_dev
- 0.0007938452130121063
- cost_per_task_usd
- 0.0046818838383838384
- output_tokens_per_task
- 7946.1447811447815
- latency_seconds_per_task
- 164.38580303030304
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 59
- benchmark_evaluation
- true
- openrouter_permaslug
- qwen/qwen3-235b-a22b-07-25
- source model id
- qwen/qwen3-235b-a22b-07-25
Original result ↗43%
- source id
- openCompass
- leaderboard name
- LiveCodeBenchV6
- dimension
- general
- metric
- LiveCodeBenchV6
- rank
- 42
- aggregation scope
- status
- published
- direction
- higherIsBetter
- conditions
- suite
- OverallTable
- catalog
- open
- snapshot
- month
- REALTIME
- fileName
- assets/research-rank/research-data.REALTIME.20260309.json
- updateTime
- 2026-03-09
- columnFileName
- assets/research-rank/research-column.REALTIME.20260309.json
- metric_kind
- LiveCodeBenchV6
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- 0-data.json
- source_revision
- 117c5effa3066abf39f3e8f61109707b09f0cd95223246c07acd6289f4f7fcd2
- metric_population
- 48
- benchmark_evaluation
- true
- source model id
- Qwen3-235B-A22B-Instruct-2507
Original result ↗42.889%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · Novita
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 45
- sample size
- 150
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 3
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- Novita
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-03T16:13:22.758Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 150
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.42888899999999996
- accuracy_std_dev
- 0.01498961780700229
- cost_per_task_usd
- 0.025741633333333337
- output_tokens_per_task
- 3503.96
- latency_seconds_per_task
- 239.67891333333333
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 52
- benchmark_evaluation
- true
- openrouter_permaslug
- qwen/qwen3-235b-a22b-07-25
- source model id
- qwen/qwen3-235b-a22b-07-25
Original result ↗72.106%
- source id
- openRouterBenchmarks
- leaderboard name
- GPQA Diamond · Accuracy · DeepInfra
- dimension
- knowledge
- metric
- GPQA Diamond · Accuracy
- rank
- 46
- sample size
- 1584
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 8
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- gpqa_diamond
- name
- GPQA Diamond
- record_kind
- aggregate
- provider_name
- DeepInfra
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T16:02:18.363Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- GPQA Diamond · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 1584
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/gpqa-diamond
- measurements
- accuracy
- 0.72106475
- accuracy_std_dev
- 0.011038884066675383
- cost_per_task_usd
- 0.004405270833333332
- output_tokens_per_task
- 7885.837121212121
- latency_seconds_per_task
- 708.0025321969697
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 71
- benchmark_evaluation
- true
- openrouter_permaslug
- qwen/qwen3-235b-a22b-07-25
- source model id
- qwen/qwen3-235b-a22b-07-25
Original result ↗1,441.202
- source id
- lmarena
- leaderboard name
- LMArena Text Factuality Spanish
- dimension
- factuality
- metric
- Arena rating
- rank
- 53
- sample size
- 1252
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Qwen3-235B-A22B-Instruct (2507)
- confidence lower
- 1424.1464775687082
- confidence upper
- 1458.2577466313578
Original result ↗42.375%
- source id
- openRouterBenchmarks
- leaderboard name
- tau2-bench Verified Airline · Accuracy · DeepInfra
- dimension
- agentic
- metric
- tau2-bench Verified Airline · Accuracy
- rank
- 57
- sample size
- 400
- aggregation scope
- status
- published
- harness
- url
- https://github.com/OpenRouterTeam/benchmark-harness
- name
- OpenRouter benchmark harness
- direction
- higherIsBetter
- run_count
- 8
- conditions
- window
- newest 90 days; successful visible nonzero runs
- dataset
- benchmark-leaderboard
- benchmark
- id
- tau_bench_verified_airline
- name
- tau2-bench Verified Airline
- record_kind
- aggregate
- provider_name
- DeepInfra
- attribution
- Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
- last_run_at
- 2026-10-31T19:22:37.207Z
- license_url
- https://creativecommons.org/licenses/by/4.0/
- methodology
- Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
- metric_kind
- tau2-bench Verified Airline · Accuracy
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- sample_size
- 400
- source_file
- benchmark-leaderboard.json
- source_page
- https://openrouter.ai/benchmarks/tau2-bench-airline
- measurements
- accuracy
- 0.423750125
- accuracy_std_dev
- 0.03212117519424492
- cost_per_task_usd
- 0.023984480000000002
- output_tokens_per_task
- 3544.9625
- latency_seconds_per_task
- 452.77921000000003
- published_at
- 2026-11-16T01:46:02.653Z
- snapshot_date
- 2026-11-16
- source_revision
- 7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
- transformations
- TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
- metric_population
- 63
- benchmark_evaluation
- true
- openrouter_permaslug
- qwen/qwen3-235b-a22b-07-25
- source model id
- qwen/qwen3-235b-a22b-07-25
Original result ↗1,395.655
- source id
- lmarena
- leaderboard name
- LMArena Text Factuality Polish
- dimension
- factuality
- metric
- Arena rating
- rank
- 60
- sample size
- 1085
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Qwen3-235B-A22B-Instruct (2507)
- confidence lower
- 1378.708143562488
- confidence upper
- 1412.6012408651156
Original result ↗1,487.665
- source id
- lmarena
- leaderboard name
- LMArena Text Factuality Chinese
- dimension
- factuality
- metric
- Arena rating
- rank
- 65
- sample size
- 2600
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Qwen3-235B-A22B-Instruct (2507)
- confidence lower
- 1475.5251927095821
- confidence upper
- 1499.8044315524776
Original result ↗1,447.027
- source id
- lmarena
- leaderboard name
- LMArena Text Factuality Industry Mathematical
- dimension
- factuality
- metric
- Arena rating
- rank
- 66
- sample size
- 2158
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Qwen3-235B-A22B-Instruct (2507)
- confidence lower
- 1434.7915278135035
- confidence upper
- 1459.2618176163992
Original result ↗1,471.269
- source id
- lmarena
- leaderboard name
- LMArena Text Factuality Industry Medicine And Healthcare
- dimension
- factuality
- metric
- Arena rating
- rank
- 72
- sample size
- 2450
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Qwen3-235B-A22B-Instruct (2507)
- confidence lower
- 1459.5139738840273
- confidence upper
- 1483.0240240162075
Original result ↗1,473.644
- source id
- lmarena
- leaderboard name
- LMArena Text Factuality Expert
- dimension
- factuality
- metric
- Arena rating
- rank
- 77
- sample size
- 2918
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Qwen3-235B-A22B-Instruct (2507)
- confidence lower
- 1462.8364233828888
- confidence upper
- 1484.4514492306264
Original result ↗1,448.593
- source id
- lmarena
- leaderboard name
- LMArena Text Factuality French
- dimension
- factuality
- metric
- Arena rating
- rank
- 80
- sample size
- 1055
- evaluated at
- 2026-11-09T00:00:00.000Z
- source model id
- Qwen3-235B-A22B-Instruct (2507)
- confidence lower
- 1431.175313935033
- confidence upper
- 1466.0116615446204
Original result ↗Published prices
1–3 / 3
$0.05
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.14
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$0.80
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
Provider specifications & access
1–8 / 8
- source model id
- azure-huggingface/qwen--qwen3-235b-a22b-instruct-2507@1
- hosting
- foundry-managed-compute
- license
- apache-2.0
- context length
- 262144
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@1
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@2
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@3
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@4
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@5
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- publishers/qwen/models/qwen3@qwen3-235b-a22b-instruct-2507
- hosting
- vertex-self-deployed
- source model id
- Qwen/Qwen3-235B-A22B-Instruct-2507
- hosting
- siliconflow-hosted
- max output tokens
- 262144
- capabilities
- structuredOutputs
- toolCalling
Safety assessments
1–1 / 1
26 / 100
- source id
- stanfordFMTI
- canonical model id
- qwen/qwen3-235b-a22b-07-25
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Alibaba
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 26 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–6 / 6
- field
- context_length
- value
- 262144
- normalized value
- 262144
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 262144
- normalized value
- 262144
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 262K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-06-30
- normalized value
- 2025-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 117964
- normalized value
- 117964
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 262K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
Explore more models