Qwen: Qwen3 235B A22B
Qwen3-235B-A22B is a 235B parameter mixture-of-experts (MoE) model developed by Qwen, activating 22B parameters per forward pass. It supports seamless switching between a "thinking" mode for complex …
qwen/qwen3-235b-a22b-04-28
Qwen3-235B-A22B is a 235B parameter mixture-of-experts (MoE) model developed by Qwen, activating 22B parameters per forward pass. It supports seamless switching between a "thinking" mode for complex reasoning, math, and...
Specifications & capabilities
- Input
- text
- Output
- text
- Context window · tokens
- 131072
- Published model ID
- Qwen3-235B-A22B
- Capabilities
- structuredOutputs, toolCalling
Evaluations
21–35 / 35
0.874
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Redux · Normalized (reported)
- dimension
- general
- metric
- MMLU-Redux · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Redux
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2406.04127
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
- benchmark_id
- mmlu-redux
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Redux · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.874
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Redux · Raw
- dimension
- general
- metric
- MMLU-Redux · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Redux
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2406.04127
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
- benchmark_id
- mmlu-redux
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Redux · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.719
- source id
- llmStatsArchive
- leaderboard name
- MultiLF · Raw
- dimension
- general
- metric
- MultiLF · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MultiLF
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:14.628191+00:00
- updated_at
- 2025-07-19T19:56:14.628191+00:00
- description
- MultiLF benchmark
- benchmark_id
- multilf
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MultiLF · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.719
- source id
- llmStatsArchive
- leaderboard name
- MultiLF · Normalized (reported)
- dimension
- general
- metric
- MultiLF · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MultiLF
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:14.628191+00:00
- updated_at
- 2025-07-19T19:56:14.628191+00:00
- description
- MultiLF benchmark
- benchmark_id
- multilf
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MultiLF · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.682
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Pro · Raw
- dimension
- general
- metric
- MMLU-Pro · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Pro
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:11.408351+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:11.408351+00:00
- description
- A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
- benchmark_id
- mmlu-pro
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Pro · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.682
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Pro · Normalized (reported)
- dimension
- general
- metric
- MMLU-Pro · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Pro
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:11.408351+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:11.408351+00:00
- description
- A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
- benchmark_id
- mmlu-pro
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Pro · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 5
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.835
- source id
- llmStatsArchive
- leaderboard name
- MGSM · Normalized (reported)
- dimension
- general
- metric
- MGSM · Normalized (reported)
- rank
- 3
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MGSM
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.669061+00:00
- paper_link
- https://arxiv.org/abs/2210.03057
- updated_at
- 2025-07-19T19:56:13.669061+00:00
- description
- MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
- benchmark_id
- mgsm
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MGSM · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.835
- source id
- llmStatsArchive
- leaderboard name
- MGSM · Raw
- dimension
- general
- metric
- MGSM · Raw
- rank
- 3
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MGSM
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:13.669061+00:00
- paper_link
- https://arxiv.org/abs/2210.03057
- updated_at
- 2025-07-19T19:56:13.669061+00:00
- description
- MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
- benchmark_id
- mgsm
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MGSM · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.718
- source id
- llmStatsArchive
- leaderboard name
- MATH · Raw
- dimension
- general
- metric
- MATH · Raw
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MATH
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.804258+00:00
- paper_link
- https://arxiv.org/abs/2103.03874
- updated_at
- 2025-07-19T19:56:11.804258+00:00
- description
- MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
- benchmark_id
- math
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MATH · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 7
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.718
- source id
- llmStatsArchive
- leaderboard name
- MATH · Normalized (reported)
- dimension
- general
- metric
- MATH · Normalized (reported)
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MATH
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.804258+00:00
- paper_link
- https://arxiv.org/abs/2103.03874
- updated_at
- 2025-07-19T19:56:11.804258+00:00
- description
- MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
- benchmark_id
- math
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MATH · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 7
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.441
- source id
- llmStatsArchive
- leaderboard name
- SuperGPQA · Normalized (reported)
- dimension
- general
- metric
- SuperGPQA · Normalized (reported)
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- SuperGPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- reasoning
- general
- math
- legal
- healthcare
- finance
- chemistry
- economics
- physics
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2502.14739
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- SuperGPQA is a comprehensive benchmark that evaluates large language models across 285 graduate-level academic disciplines. The benchmark contains 25,957 questions covering 13 broad disciplinary areas including Engineering, Medicine, Science, and Law, with specialized fields in light industry, agriculture, and service-oriented domains. It employs a Human-LLM collaborative filtering mechanism with over 80 expert annotators to create challenging questions that assess graduate-level knowledge and reasoning capabilities.
- benchmark_id
- supergpqa
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- SuperGPQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.441
- source id
- llmStatsArchive
- leaderboard name
- SuperGPQA · Raw
- dimension
- general
- metric
- SuperGPQA · Raw
- rank
- 4
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- SuperGPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- reasoning
- general
- math
- legal
- healthcare
- finance
- chemistry
- economics
- physics
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2502.14739
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- SuperGPQA is a comprehensive benchmark that evaluates large language models across 285 graduate-level academic disciplines. The benchmark contains 25,957 questions covering 13 broad disciplinary areas including Engineering, Medicine, Science, and Law, with specialized fields in light industry, agriculture, and service-oriented domains. It employs a Human-LLM collaborative filtering mechanism with over 80 expert annotators to create challenging questions that assess graduate-level knowledge and reasoning capabilities.
- benchmark_id
- supergpqa
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- SuperGPQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.475
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Normalized (reported)
- dimension
- general
- metric
- GPQA · Normalized (reported)
- rank
- 6
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 11
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗0.475
- source id
- llmStatsArchive
- leaderboard name
- GPQA · Raw
- dimension
- general
- metric
- GPQA · Raw
- rank
- 6
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GPQA
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.588605+00:00
- paper_link
- https://arxiv.org/abs/2311.12022
- updated_at
- 2025-07-19T19:56:11.588605+00:00
- description
- A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
- benchmark_id
- gpqa
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GPQA · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://qwenlm.github.io/blog/qwen3/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- qwen/qwen3-235b-a22b
- metric_population
- 11
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- qwen3-235b-a22b
Original result ↗9.5
- source id
- artificialAnalysis
- dimension
- intelligence
- metric
- Intelligence Index
Original result ↗Provider specifications & access
1–3 / 3
- source model id
- publishers/hf-qwen/models/qwen3-235b-a22b@001
- hosting
- vertex-self-deployed
- source model id
- publishers/qwen/models/qwen3@qwen3-235b-a22b
- hosting
- vertex-self-deployed
- source model id
- Qwen/Qwen3-235B-A22B
- hosting
- siliconflow-hosted
- capabilities
- structuredOutputs
- toolCalling
Safety assessments
1–1 / 1
26 / 100
- source id
- stanfordFMTI
- canonical model id
- qwen/qwen3-235b-a22b-04-28
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Alibaba
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 26 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Source specification observations
1–1 / 1
- field
- context_length
- value
- 131K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
Explore more models