Qwen: Qwen3 235B A22B

Qwen3-235B-A22B is a 235B parameter mixture-of-experts (MoE) model developed by Qwen, activating 22B parameters per forward pass. It supports seamless switching between a "thinking" mode for complex …

qwen/qwen3-235b-a22b-04-28

Qwen3-235B-A22B is a 235B parameter mixture-of-experts (MoE) model developed by Qwen, activating 22B parameters per forward pass. It supports seamless switching between a "thinking" mode for complex reasoning, math, and...

Specifications & capabilities

Input
text
Output
text
Context window · tokens
131072
Published model ID
Qwen3-235B-A22B
Capabilities
structuredOutputs, toolCalling

Evaluations

21–35 / 35

LLM Stats · GitHub archive

0.874

source id
llmStatsArchive
leaderboard name
MMLU-Redux · Normalized (reported)
dimension
general
metric
MMLU-Redux · Normalized (reported)
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU-Redux
language
en
modality
text
verified
false
max_score
1
categories
  • language
  • reasoning
  • math
  • general
created_at
2025-09-05T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2406.04127
updated_at
2025-09-05T00:00:00.000000+00:00
description
An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
benchmark_id
mmlu-redux
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU-Redux · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.874

source id
llmStatsArchive
leaderboard name
MMLU-Redux · Raw
dimension
general
metric
MMLU-Redux · Raw
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU-Redux
language
en
modality
text
verified
false
max_score
1
categories
  • language
  • reasoning
  • math
  • general
created_at
2025-09-05T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2406.04127
updated_at
2025-09-05T00:00:00.000000+00:00
description
An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
benchmark_id
mmlu-redux
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU-Redux · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.719

source id
llmStatsArchive
leaderboard name
MultiLF · Raw
dimension
general
metric
MultiLF · Raw
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MultiLF
language
en
modality
text
verified
false
max_score
1
categories
  • general
created_at
2025-07-19T19:56:14.628191+00:00
updated_at
2025-07-19T19:56:14.628191+00:00
description
MultiLF benchmark
benchmark_id
multilf
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MultiLF · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.719

source id
llmStatsArchive
leaderboard name
MultiLF · Normalized (reported)
dimension
general
metric
MultiLF · Normalized (reported)
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MultiLF
language
en
modality
text
verified
false
max_score
1
categories
  • general
created_at
2025-07-19T19:56:14.628191+00:00
updated_at
2025-07-19T19:56:14.628191+00:00
description
MultiLF benchmark
benchmark_id
multilf
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MultiLF · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.682

source id
llmStatsArchive
leaderboard name
MMLU-Pro · Raw
dimension
general
metric
MMLU-Pro · Raw
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU-Pro
language
en
modality
text
verified
false
max_score
1
categories
  • language
  • reasoning
  • math
  • general
created_at
2025-07-19T19:56:11.408351+00:00
paper_link
https://arxiv.org/abs/2406.01574
updated_at
2025-07-19T19:56:11.408351+00:00
description
A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
benchmark_id
mmlu-pro
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU-Pro · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
5
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.682

source id
llmStatsArchive
leaderboard name
MMLU-Pro · Normalized (reported)
dimension
general
metric
MMLU-Pro · Normalized (reported)
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMLU-Pro
language
en
modality
text
verified
false
max_score
1
categories
  • language
  • reasoning
  • math
  • general
created_at
2025-07-19T19:56:11.408351+00:00
paper_link
https://arxiv.org/abs/2406.01574
updated_at
2025-07-19T19:56:11.408351+00:00
description
A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
benchmark_id
mmlu-pro
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMLU-Pro · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
5
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.835

source id
llmStatsArchive
leaderboard name
MGSM · Normalized (reported)
dimension
general
metric
MGSM · Normalized (reported)
rank
3
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MGSM
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:13.669061+00:00
paper_link
https://arxiv.org/abs/2210.03057
updated_at
2025-07-19T19:56:13.669061+00:00
description
MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
benchmark_id
mgsm
multilingual
true
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MGSM · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.835

source id
llmStatsArchive
leaderboard name
MGSM · Raw
dimension
general
metric
MGSM · Raw
rank
3
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MGSM
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:13.669061+00:00
paper_link
https://arxiv.org/abs/2210.03057
updated_at
2025-07-19T19:56:13.669061+00:00
description
MGSM (Multilingual Grade School Math) is a benchmark of grade-school math problems. Contains 250 grade-school math problems manually translated from the GSM8K dataset into ten typologically diverse languages: Spanish, French, German, Russian, Chinese, Japanese, Thai, Swahili, Bengali, and Telugu. Evaluates multilingual mathematical reasoning capabilities.
benchmark_id
mgsm
multilingual
true
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MGSM · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.718

source id
llmStatsArchive
leaderboard name
MATH · Raw
dimension
general
metric
MATH · Raw
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MATH
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.804258+00:00
paper_link
https://arxiv.org/abs/2103.03874
updated_at
2025-07-19T19:56:11.804258+00:00
description
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
benchmark_id
math
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MATH · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
7
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.718

source id
llmStatsArchive
leaderboard name
MATH · Normalized (reported)
dimension
general
metric
MATH · Normalized (reported)
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MATH
language
en
modality
text
verified
false
max_score
1
categories
  • math
  • reasoning
created_at
2025-07-19T19:56:11.804258+00:00
paper_link
https://arxiv.org/abs/2103.03874
updated_at
2025-07-19T19:56:11.804258+00:00
description
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
benchmark_id
math
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MATH · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
7
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.441

source id
llmStatsArchive
leaderboard name
SuperGPQA · Normalized (reported)
dimension
general
metric
SuperGPQA · Normalized (reported)
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
SuperGPQA
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
  • math
  • legal
  • healthcare
  • finance
  • chemistry
  • economics
  • physics
created_at
2025-09-05T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2502.14739
updated_at
2025-09-05T00:00:00.000000+00:00
description
SuperGPQA is a comprehensive benchmark that evaluates large language models across 285 graduate-level academic disciplines. The benchmark contains 25,957 questions covering 13 broad disciplinary areas including Engineering, Medicine, Science, and Law, with specialized fields in light industry, agriculture, and service-oriented domains. It employs a Human-LLM collaborative filtering mechanism with over 80 expert annotators to create challenging questions that assess graduate-level knowledge and reasoning capabilities.
benchmark_id
supergpqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
SuperGPQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
4
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.441

source id
llmStatsArchive
leaderboard name
SuperGPQA · Raw
dimension
general
metric
SuperGPQA · Raw
rank
4
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
SuperGPQA
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
  • math
  • legal
  • healthcare
  • finance
  • chemistry
  • economics
  • physics
created_at
2025-09-05T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2502.14739
updated_at
2025-09-05T00:00:00.000000+00:00
description
SuperGPQA is a comprehensive benchmark that evaluates large language models across 285 graduate-level academic disciplines. The benchmark contains 25,957 questions covering 13 broad disciplinary areas including Engineering, Medicine, Science, and Law, with specialized fields in light industry, agriculture, and service-oriented domains. It employs a Human-LLM collaborative filtering mechanism with over 80 expert annotators to create challenging questions that assess graduate-level knowledge and reasoning capabilities.
benchmark_id
supergpqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
SuperGPQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
4
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.475

source id
llmStatsArchive
leaderboard name
GPQA · Normalized (reported)
dimension
general
metric
GPQA · Normalized (reported)
rank
6
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
created_at
2025-07-19T19:56:11.588605+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:11.588605+00:00
description
A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
benchmark_id
gpqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
11
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

LLM Stats · GitHub archive

0.475

source id
llmStatsArchive
leaderboard name
GPQA · Raw
dimension
general
metric
GPQA · Raw
rank
6
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
GPQA
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • general
created_at
2025-07-19T19:56:11.588605+00:00
paper_link
https://arxiv.org/abs/2311.12022
updated_at
2025-07-19T19:56:11.588605+00:00
description
A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Questions are Google-proof and extremely difficult, with PhD experts reaching 65% accuracy.
benchmark_id
gpqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
GPQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/qwen/models/qwen3-235b-a22b/benchmarks.json
archive_kind
llmStats
evidence_url
https://qwenlm.github.io/blog/qwen3/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
qwen/qwen3-235b-a22b
metric_population
11
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
qwen3-235b-a22b
Original result ↗

Provider specifications & access

1–3 / 3

Qwen/Qwen3-235B-A22B ↗

source model id
publishers/hf-qwen/models/qwen3-235b-a22b@001
hosting
vertex-self-deployed
platforms
  • Google Cloud Vertex AI
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    Qwen3-235B-A22B ↗

    source model id
    publishers/qwen/models/qwen3@qwen3-235b-a22b
    hosting
    vertex-self-deployed
    platforms
    • Google Cloud Vertex AI
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      Qwen3-235B-A22B ↗

      source model id
      Qwen/Qwen3-235B-A22B
      hosting
      siliconflow-hosted
      platforms
      • SiliconFlow
      architecture
      input_modalities
      • text
      output_modalities
      • text
      capabilities
      • structuredOutputs
      • toolCalling

      Safety assessments

      1–1 / 1

      Foundation Model Transparency Index 2025 ↗

      26 / 100

      source id
      stanfordFMTI
      canonical model id
      qwen/qwen3-235b-a22b-04-28
      taxonomy
      transparency
      hazard category
      transparency
      hazard name
      Organization transparency
      policy area
      Alibaba
      assessment type
      organization-level disclosure
      metric
      disclosure_score
      result label
      26 / 100
      sample size
      100
      updated at
      2026-11-09T18:08:05.227Z

      Source specification observations

      1–1 / 1

      siliconFlow ↗

      field
      context_length
      value
      131K
      scope
      source-model
      source id
      siliconFlow
      confidence
      1

      Original sources

      1–12 / 12

      SiliconFlow · Model page ↗

      Hugging Face · Model repository ↗

      Google Cloud · Model page ↗

      Google Cloud · Model page ↗

      SiliconFlow · Playground ↗

      Google Cloud · Model documentation ↗

      SiliconFlow · API documentation ↗

      GitHub · Catalog ↗

      GitHub · Catalog ↗

      Google Cloud · Catalog ↗

      Hugging Face · Catalog ↗

      SiliconFlow · Catalog ↗

      Source links

      1–7 / 7

      Explore more models

      Models

      Loading
      EVIDENCE / COMPARISON

      Benchmarks

      Explore every benchmark

      Loading

      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

      Install TensorTrace