Qwen: Qwen3 235B A22B Instruct 2507
Qwen3-235B-A22B-Instruct-2507 is a multilingual, instruction-tuned mixture-of-experts language model based on the Qwen3-235B architecture, with 22B active parameters per forward pass. It is optimized…
qwen/qwen3-235b-a22b-07-25
Qwen3-235B-A22B-Instruct-2507 is a multilingual, instruction-tuned mixture-of-experts language model based on the Qwen3-235B architecture, with 22B active parameters per forward pass. It is optimized for general-purpose text generation, including instruction following, logical reasoning, math, code, and tool usage. The model supports a native 262K context length and does not implement "thinking mode" (<think> blocks).
Compared to its base variant, this version delivers significant gains in knowledge coverage, long-context reasoning, coding benchmarks, and alignment with open-ended tasks. It is particularly strong on multilingual understanding, math reasoning (e.g., AIME, HMMT), and alignment evaluations like Arena-Hard and WritingBench.
Specifications & capabilities
- Input
- text
- Output
- text
- Context window · tokens
- 262144
- Knowledge cutoff
- 2025-06-30
- Published model ID
- Qwen3-235B-A22B-Instruct-2507
- Capabilities
- promptCaching, structuredOutputs, toolCalling
Description source ↗Evaluations
1–20 / 160
0.95
- source id
- llmStatsArchive
- leaderboard name
- ZebraLogic · Raw
- dimension
- general
- metric
- ZebraLogic · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- ZebraLogic
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2502.01100
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- ZebraLogic is an evaluation framework for assessing large language models' logical reasoning capabilities through logic grid puzzles derived from constraint satisfaction problems (CSPs). The benchmark consists of 1,000 programmatically generated puzzles with controllable and quantifiable complexity, revealing a 'curse of complexity' where model accuracy declines significantly as problem complexity grows.
- benchmark_id
- zebralogic
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- ZebraLogic · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.95
- source id
- llmStatsArchive
- leaderboard name
- ZebraLogic · Normalized (reported)
- dimension
- general
- metric
- ZebraLogic · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- ZebraLogic
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2502.01100
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- ZebraLogic is an evaluation framework for assessing large language models' logical reasoning capabilities through logic grid puzzles derived from constraint satisfaction problems (CSPs). The benchmark consists of 1,000 programmatically generated puzzles with controllable and quantifiable complexity, revealing a 'curse of complexity' where model accuracy declines significantly as problem complexity grows.
- benchmark_id
- zebralogic
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- ZebraLogic · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.931
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Redux · Normalized (reported)
- dimension
- general
- metric
- MMLU-Redux · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Redux
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2406.04127
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
- benchmark_id
- mmlu-redux
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Redux · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.931
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Redux · Raw
- dimension
- general
- metric
- MMLU-Redux · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Redux
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2406.04127
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
- benchmark_id
- mmlu-redux
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Redux · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.887
- source id
- llmStatsArchive
- leaderboard name
- IFEval · Normalized (reported)
- dimension
- general
- metric
- IFEval · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- IFEval
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.241350+00:00
- paper_link
- https://arxiv.org/abs/2311.07911
- updated_at
- 2025-07-19T19:56:12.241350+00:00
- description
- Instruction-Following Evaluation (IFEval) benchmark for large language models, focusing on verifiable instructions with 25 types of instructions and around 500 prompts containing one or more verifiable constraints
- benchmark_id
- ifeval
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- IFEval · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.887
- source id
- llmStatsArchive
- leaderboard name
- IFEval · Raw
- dimension
- general
- metric
- IFEval · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- IFEval
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.241350+00:00
- paper_link
- https://arxiv.org/abs/2311.07911
- updated_at
- 2025-07-19T19:56:12.241350+00:00
- description
- Instruction-Following Evaluation (IFEval) benchmark for large language models, focusing on verifiable instructions with 25 types of instructions and around 500 prompts containing one or more verifiable constraints
- benchmark_id
- ifeval
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- IFEval · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.852
- source id
- llmStatsArchive
- leaderboard name
- WritingBench · Normalized (reported)
- dimension
- general
- metric
- WritingBench · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- WritingBench
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- writing
- creativity
- communication
- created_at
- 2025-08-03T22:06:11.074130+00:00
- paper_link
- https://arxiv.org/abs/2503.05244
- updated_at
- 2025-08-03T22:06:11.074130+00:00
- description
- A comprehensive benchmark for evaluating large language models' generative writing capabilities across 6 core writing domains (Academic & Engineering, Finance & Business, Politics & Law, Literature & Art, Education, Advertising & Marketing) and 100 subdomains. Contains 1,239 queries with a query-dependent evaluation framework that dynamically generates 5 instance-specific assessment criteria for each writing task, using a fine-tuned critic model to score responses on style, format, and length dimensions.
- benchmark_id
- writingbench
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- WritingBench · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.852
- source id
- llmStatsArchive
- leaderboard name
- WritingBench · Raw
- dimension
- general
- metric
- WritingBench · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- WritingBench
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- writing
- creativity
- communication
- created_at
- 2025-08-03T22:06:11.074130+00:00
- paper_link
- https://arxiv.org/abs/2503.05244
- updated_at
- 2025-08-03T22:06:11.074130+00:00
- description
- A comprehensive benchmark for evaluating large language models' generative writing capabilities across 6 core writing domains (Academic & Engineering, Finance & Business, Politics & Law, Literature & Art, Education, Advertising & Marketing) and 100 subdomains. Contains 1,239 queries with a query-dependent evaluation framework that dynamically generates 5 instance-specific assessment criteria for each writing task, using a fine-tuned critic model to score responses on style, format, and length dimensions.
- benchmark_id
- writingbench
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- WritingBench · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.843
- source id
- llmStatsArchive
- leaderboard name
- CSimpleQA · Normalized (reported)
- dimension
- general
- metric
- CSimpleQA · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- CSimpleQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.931358+00:00
- paper_link
- https://arxiv.org/abs/2411.07140
- updated_at
- 2025-07-19T19:56:11.931358+00:00
- description
- Chinese SimpleQA is the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions. It contains 3,000 high-quality questions spanning 6 major topics with 99 diverse subtopics, designed to assess Chinese factual knowledge across humanities, science, engineering, culture, and society.
- benchmark_id
- csimpleqa
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- CSimpleQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.843
- source id
- llmStatsArchive
- leaderboard name
- CSimpleQA · Raw
- dimension
- general
- metric
- CSimpleQA · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- CSimpleQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.931358+00:00
- paper_link
- https://arxiv.org/abs/2411.07140
- updated_at
- 2025-07-19T19:56:11.931358+00:00
- description
- Chinese SimpleQA is the first comprehensive Chinese benchmark to evaluate the factuality ability of language models to answer short questions. It contains 3,000 high-quality questions spanning 6 major topics with 99 diverse subtopics, designed to assess Chinese factual knowledge across humanities, science, engineering, culture, and society.
- benchmark_id
- csimpleqa
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- CSimpleQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.83
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Pro · Raw
- dimension
- general
- metric
- MMLU-Pro · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Pro
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:11.408351+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:11.408351+00:00
- description
- A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
- benchmark_id
- mmlu-pro
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Pro · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.83
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Pro · Normalized (reported)
- dimension
- general
- metric
- MMLU-Pro · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Pro
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:11.408351+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:11.408351+00:00
- description
- A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
- benchmark_id
- mmlu-pro
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Pro · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.794
- source id
- llmStatsArchive
- leaderboard name
- MMLU-ProX · Normalized (reported)
- dimension
- general
- metric
- MMLU-ProX · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-ProX
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:13.738623+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:13.738623+00:00
- description
- Extended version of MMLU-Pro providing additional challenging multiple-choice questions for evaluating language models across diverse academic and professional domains. Built on the foundation of the Massive Multitask Language Understanding benchmark framework.
- benchmark_id
- mmlu-prox
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-ProX · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.794
- source id
- llmStatsArchive
- leaderboard name
- MMLU-ProX · Raw
- dimension
- general
- metric
- MMLU-ProX · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-ProX
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:13.738623+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:13.738623+00:00
- description
- Extended version of MMLU-Pro providing additional challenging multiple-choice questions for evaluating language models across diverse academic and professional domains. Built on the foundation of the Massive Multitask Language Understanding benchmark framework.
- benchmark_id
- mmlu-prox
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-ProX · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.775
- source id
- llmStatsArchive
- leaderboard name
- Multi-IF · Normalized (reported)
- dimension
- general
- metric
- Multi-IF · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Multi-IF
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- reasoning
- communication
- language
- created_at
- 2025-07-19T19:56:14.638787+00:00
- paper_link
- https://arxiv.org/abs/2410.15553
- updated_at
- 2025-07-19T19:56:14.638787+00:00
- description
- Multi-IF benchmarks LLMs on multi-turn and multilingual instruction following. It expands upon IFEval by incorporating multi-turn sequences and translating English prompts into 7 other languages, resulting in 4,501 multilingual conversations with three turns each. The benchmark reveals that current leading LLMs struggle with maintaining accuracy in multi-turn instructions and shows higher error rates for non-Latin script languages.
- benchmark_id
- multi-if
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Multi-IF · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.775
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Normalized (reported)
- dimension
- general
- metric
- GPQA · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 11
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.775
- source id
- llmStatsArchive
- leaderboard name
- Multi-IF · Raw
- dimension
- general
- metric
- Multi-IF · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- Multi-IF
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- reasoning
- communication
- language
- created_at
- 2025-07-19T19:56:14.638787+00:00
- paper_link
- https://arxiv.org/abs/2410.15553
- updated_at
- 2025-07-19T19:56:14.638787+00:00
- description
- Multi-IF benchmarks LLMs on multi-turn and multilingual instruction following. It expands upon IFEval by incorporating multi-turn sequences and translating English prompts into 7 other languages, resulting in 4,501 multilingual conversations with three turns each. The benchmark reveals that current leading LLMs struggle with maintaining accuracy in multi-turn instructions and shows higher error rates for non-Latin script languages.
- benchmark_id
- multi-if
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- Multi-IF · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.775
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Raw
- dimension
- general
- metric
- GPQA · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 11
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.754
- source id
- llmStatsArchive
- leaderboard name
- LiveBench 20241125 · Raw
- dimension
- general
- metric
- LiveBench 20241125 · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- LiveBench 20241125
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-08-03T22:06:11.046321+00:00
- paper_link
- https://arxiv.org/abs/2406.19314
- updated_at
- 2025-08-03T22:06:11.046321+00:00
- description
- LiveBench is a challenging, contamination-limited LLM benchmark that addresses test set contamination by releasing new questions monthly based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. It comprises tasks across math, coding, reasoning, language, instruction following, and data analysis with verifiable, objective ground-truth answers.
- benchmark_id
- livebench-20241125
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- LiveBench 20241125 · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗0.754
- source id
- llmStatsArchive
- leaderboard name
- LiveBench 20241125 · Normalized (reported)
- dimension
- general
- metric
- LiveBench 20241125 · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- LiveBench 20241125
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-08-03T22:06:11.046321+00:00
- paper_link
- https://arxiv.org/abs/2406.19314
- updated_at
- 2025-08-03T22:06:11.046321+00:00
- description
- LiveBench is a challenging, contamination-limited LLM benchmark that addresses test set contamination by releasing new questions monthly based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. It comprises tasks across math, coding, reasoning, language, instruction following, and data analysis with verifiable, objective ground-truth answers.
- benchmark_id
- livebench-20241125
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- LiveBench 20241125 · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b-instruct-2507/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b-instruct-2507
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b-instruct-2507
Original result ↗Published prices
1–3 / 3
$0.05
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- cache_read_tokens_per_million
- Currency
- USD
$0.14
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$0.80
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
Provider specifications & access
1–8 / 8
- source model id
- azure-huggingface/qwen--qwen3-235b-a22b-instruct-2507@1
- hosting
- foundry-managed-compute
- license
- apache-2.0
- context length
- 262144
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@1
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@2
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@3
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@4
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- HuggingFace/qwen-qwen3-235b-a22b-instruct-2507@5
- hosting
- foundry-managed-compute
- license
- apache-2.0
- source model id
- publishers/qwen/models/qwen3@qwen3-235b-a22b-instruct-2507
- hosting
- vertex-self-deployed
- source model id
- Qwen/Qwen3-235B-A22B-Instruct-2507
- hosting
- siliconflow-hosted
- max output tokens
- 262144
- capabilities
- structuredOutputs
- toolCalling
Safety assessments
1–1 / 1
26 / 100
- source id
- stanfordFMTI
- canonical model id
- qwen/qwen3-235b-a22b-07-25
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Alibaba
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 26 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–6 / 6
- field
- context_length
- value
- 262144
- normalized value
- 262144
- scope
- source-model
- source id
- azureFoundry
- confidence
- 1
- field
- context_length
- value
- 262144
- normalized value
- 262144
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 262K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2025-06-30
- normalized value
- 2025-06-30
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 117964
- normalized value
- 117964
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 262K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
Explore more models