MoonshotAI: Kimi K2 0711
Kimi K2 Instruct is a large-scale Mixture-of-Experts (MoE) language model developed by Moonshot AI, featuring 1 trillion total parameters with 32 billion active per forward pass. It is optimized for …
moonshotai/kimi-k2
Kimi K2 Instruct is a large-scale Mixture-of-Experts (MoE) language model developed by Moonshot AI, featuring 1 trillion total parameters with 32 billion active per forward pass. It is optimized for agentic capabilities, including advanced tool use, reasoning, and code synthesis. Kimi K2 excels across a broad range of benchmarks, particularly in coding (LiveCodeBench, SWE-bench), reasoning (ZebraLogic, GPQA), and tool-use (Tau2, AceBench) tasks. It supports long-context inference up to 128K tokens and is designed with a novel training stack that includes the MuonClip optimizer for stable large-scale MoE training.
Specifications & capabilities
- Input
- text
- Output
- text
- Context window · tokens
- 131072
- Knowledge cutoff
- 2024-12-31
- Published model ID
- Kimi-K2-Instruct
- Capabilities
- structuredOutputs, toolCalling
Description source ↗Evaluations
1–20 / 60
0.974
- source id
- llmStatsArchive
- leaderboard name
- MATH-500 · Normalized (reported)
- dimension
- general
- metric
- MATH-500 · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MATH-500
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.027850+00:00
- paper_link
- https://arxiv.org/abs/2103.03874
- updated_at
- 2025-07-19T19:56:12.027850+00:00
- description
- MATH-500 is a subset of the MATH dataset containing 500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
- benchmark_id
- math-500
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MATH-500 · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.974
- source id
- llmStatsArchive
- leaderboard name
- MATH-500 · Raw
- dimension
- general
- metric
- MATH-500 · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MATH-500
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.027850+00:00
- paper_link
- https://arxiv.org/abs/2103.03874
- updated_at
- 2025-07-19T19:56:12.027850+00:00
- description
- MATH-500 is a subset of the MATH dataset containing 500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
- benchmark_id
- math-500
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MATH-500 · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.973
- source id
- llmStatsArchive
- leaderboard name
- GSM8k · Raw
- dimension
- general
- metric
- GSM8k · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GSM8k
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.397385+00:00
- paper_link
- https://arxiv.org/abs/2110.14168
- updated_at
- 2025-07-19T19:56:11.397385+00:00
- description
- Grade School Math 8K, a dataset of 8.5K high-quality linguistically diverse grade school math word problems requiring multi-step reasoning and elementary arithmetic operations.
- benchmark_id
- gsm8k
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GSM8k · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 7
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.973
- source id
- llmStatsArchive
- leaderboard name
- GSM8k · Normalized (reported)
- dimension
- general
- metric
- GSM8k · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- GSM8k
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:11.397385+00:00
- paper_link
- https://arxiv.org/abs/2110.14168
- updated_at
- 2025-07-19T19:56:11.397385+00:00
- description
- Grade School Math 8K, a dataset of 8.5K high-quality linguistically diverse grade school math word problems requiring multi-step reasoning and elementary arithmetic operations.
- benchmark_id
- gsm8k
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- GSM8k · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 7
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.956
- source id
- llmStatsArchive
- leaderboard name
- CBNSL · Normalized (reported)
- dimension
- general
- metric
- CBNSL · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- CBNSL
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.590999+00:00
- paper_link
- http://proceedings.mlr.press/v45/Zhao15a.pdf
- updated_at
- 2025-07-19T19:56:12.590999+00:00
- description
- Curriculum Learning of Bayesian Network Structures (CBNSL) benchmark for evaluating algorithms that learn Bayesian network structures from data using curriculum learning techniques. The benchmark uses networks from the bnlearn repository and evaluates structure learning performance using BDeu scoring metrics.
- benchmark_id
- cbnsl
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- CBNSL · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.956
- source id
- llmStatsArchive
- leaderboard name
- CBNSL · Raw
- dimension
- general
- metric
- CBNSL · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- CBNSL
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.590999+00:00
- paper_link
- http://proceedings.mlr.press/v45/Zhao15a.pdf
- updated_at
- 2025-07-19T19:56:12.590999+00:00
- description
- Curriculum Learning of Bayesian Network Structures (CBNSL) benchmark for evaluating algorithms that learn Bayesian network structures from data using curriculum learning techniques. The benchmark uses networks from the bnlearn repository and evaluates structure learning performance using BDeu scoring metrics.
- benchmark_id
- cbnsl
- multilingual
- false
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- CBNSL · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.933
- source id
- llmStatsArchive
- leaderboard name
- HumanEval · Raw
- dimension
- general
- metric
- HumanEval · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- HumanEval
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.595263+00:00
- paper_link
- https://arxiv.org/abs/2107.03374
- updated_at
- 2025-07-19T19:56:12.595263+00:00
- description
- A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
- benchmark_id
- humaneval
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- HumanEval · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 9
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.933
- source id
- llmStatsArchive
- leaderboard name
- HumanEval · Normalized (reported)
- dimension
- general
- metric
- HumanEval · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- HumanEval
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.595263+00:00
- paper_link
- https://arxiv.org/abs/2107.03374
- updated_at
- 2025-07-19T19:56:12.595263+00:00
- description
- A benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
- benchmark_id
- humaneval
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- HumanEval · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 9
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.927
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Redux · Raw
- dimension
- general
- metric
- MMLU-Redux · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Redux
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2406.04127
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
- benchmark_id
- mmlu-redux
- multilingual
- false
- analysis_method
- EM
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Redux · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.927
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Redux · Normalized (reported)
- dimension
- general
- metric
- MMLU-Redux · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Redux
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2406.04127
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- An improved version of the MMLU benchmark featuring manually re-annotated questions to identify and correct errors in the original dataset. Provides more reliable evaluation metrics for language models by addressing dataset quality issues found in the original MMLU.
- benchmark_id
- mmlu-redux
- multilingual
- false
- analysis_method
- EM
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Redux · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.895
- source id
- llmStatsArchive
- leaderboard name
- AutoLogi · Normalized (reported)
- dimension
- general
- metric
- AutoLogi · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AutoLogi
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2502.16906
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- AutoLogi is an automated method for synthesizing open-ended logic puzzles to evaluate reasoning abilities of Large Language Models. The benchmark addresses limitations of existing multiple-choice reasoning evaluations by featuring program-based verification and controllable difficulty levels. It includes 1,575 English and 883 Chinese puzzles, enabling more reliable evaluation that better distinguishes models' reasoning capabilities across languages.
- benchmark_id
- autologi
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AutoLogi · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.895
- source id
- llmStatsArchive
- leaderboard name
- AutoLogi · Raw
- dimension
- general
- metric
- AutoLogi · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- AutoLogi
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-09-05T00:00:00.000000+00:00
- paper_link
- https://arxiv.org/abs/2502.16906
- updated_at
- 2025-09-05T00:00:00.000000+00:00
- description
- AutoLogi is an automated method for synthesizing open-ended logic puzzles to evaluate reasoning abilities of Large Language Models. The benchmark addresses limitations of existing multiple-choice reasoning evaluations by featuring program-based verification and controllable difficulty levels. It includes 1,575 English and 883 Chinese puzzles, enabling more reliable evaluation that better distinguishes models' reasoning capabilities across languages.
- benchmark_id
- autologi
- multilingual
- true
- analysis_method
- Accuracy
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- AutoLogi · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 2
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.895
- source id
- llmStatsArchive
- leaderboard name
- MMLU · Normalized (reported)
- dimension
- general
- metric
- MMLU · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- general
- reasoning
- language
- math
- created_at
- 2025-07-19T19:56:11.200416+00:00
- paper_link
- https://arxiv.org/abs/2009.03300
- updated_at
- 2025-07-19T19:56:11.200416+00:00
- description
- Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
- benchmark_id
- mmlu
- multilingual
- false
- analysis_method
- EM
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.895
- source id
- llmStatsArchive
- leaderboard name
- MMLU · Raw
- dimension
- general
- metric
- MMLU · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- general
- reasoning
- language
- math
- created_at
- 2025-07-19T19:56:11.200416+00:00
- paper_link
- https://arxiv.org/abs/2009.03300
- updated_at
- 2025-07-19T19:56:11.200416+00:00
- description
- Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
- benchmark_id
- mmlu
- multilingual
- false
- analysis_method
- EM
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.857
- source id
- llmStatsArchive
- leaderboard name
- MultiPL-E · Raw
- dimension
- general
- metric
- MultiPL-E · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MultiPL-E
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.311919+00:00
- paper_link
- https://arxiv.org/abs/2208.08227
- updated_at
- 2025-07-19T19:56:12.311919+00:00
- description
- MultiPL-E is a scalable and extensible system for translating unit test-driven code generation benchmarks to multiple programming languages. It extends HumanEval and MBPP Python benchmarks to 18 additional programming languages, enabling evaluation of neural code generation models across diverse programming paradigms and language features.
- benchmark_id
- multipl-e
- multilingual
- true
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MultiPL-E · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.857
- source id
- llmStatsArchive
- leaderboard name
- MultiPL-E · Normalized (reported)
- dimension
- general
- metric
- MultiPL-E · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MultiPL-E
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.311919+00:00
- paper_link
- https://arxiv.org/abs/2208.08227
- updated_at
- 2025-07-19T19:56:12.311919+00:00
- description
- MultiPL-E is a scalable and extensible system for translating unit test-driven code generation benchmarks to multiple programming languages. It extends HumanEval and MBPP Python benchmarks to 18 additional programming languages, enabling evaluation of neural code generation models across diverse programming paradigms and language features.
- benchmark_id
- multipl-e
- multilingual
- true
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MultiPL-E · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 4
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.811
- source id
- llmStatsArchive
- leaderboard name
- HumanEval-ER · Raw
- dimension
- general
- metric
- HumanEval-ER · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- HumanEval-ER
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.704744+00:00
- paper_link
- https://arxiv.org/abs/2107.03374
- updated_at
- 2025-07-19T19:56:12.704744+00:00
- description
- A variant of the HumanEval benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
- benchmark_id
- humaneval-er
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- HumanEval-ER · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.811
- source id
- llmStatsArchive
- leaderboard name
- HumanEval-ER · Normalized (reported)
- dimension
- general
- metric
- HumanEval-ER · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- HumanEval-ER
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- created_at
- 2025-07-19T19:56:12.704744+00:00
- paper_link
- https://arxiv.org/abs/2107.03374
- updated_at
- 2025-07-19T19:56:12.704744+00:00
- description
- A variant of the HumanEval benchmark that measures functional correctness for synthesizing programs from docstrings, consisting of 164 original programming problems assessing language comprehension, algorithms, and simple mathematics
- benchmark_id
- humaneval-er
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- HumanEval-ER · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 1
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.811
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Pro · Raw
- dimension
- general
- metric
- MMLU-Pro · Raw
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Pro
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:11.408351+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:11.408351+00:00
- description
- A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
- benchmark_id
- mmlu-pro
- multilingual
- false
- analysis_method
- EM
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Pro · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗0.811
- source id
- llmStatsArchive
- leaderboard name
- MMLU-Pro · Normalized (reported)
- dimension
- general
- metric
- MMLU-Pro · Normalized (reported)
- rank
- 1
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MMLU-Pro
- language
- en
- modality
- text
- verified
- false
- max_score
- 1
- categories
- language
- reasoning
- math
- general
- created_at
- 2025-07-19T19:56:11.408351+00:00
- paper_link
- https://arxiv.org/abs/2406.01574
- updated_at
- 2025-07-19T19:56:11.408351+00:00
- description
- A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
- benchmark_id
- mmlu-pro
- multilingual
- false
- analysis_method
- EM
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MMLU-Pro · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/moonshotai/models/kimi-k2-instruct/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://moonshotai.github.io/Kimi-K2/
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- moonshotai/kimi-k2-instruct
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- kimi-k2-instruct
Original result ↗Published prices
1–6 / 6
$0.57
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- input_tokens_per_million
- Currency
- USD
$2.30
- Inference provider
- openRouter
- Platform
- OpenRouter
- Unit
- output_tokens_per_million
- Currency
- USD
$0.57
- Inference provider
- vercelAIGatewayCatalog
- Platform
- novita
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- novita | moonshotai/kimi-k2
- model_type
- language
- quantization
- fp8
- billing_scope
- via Vercel AI Gateway
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$2.30
- Inference provider
- vercelAIGatewayCatalog
- Platform
- novita
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- endpoint
- novita | moonshotai/kimi-k2
- model_type
- language
- quantization
- fp8
- billing_scope
- via Vercel AI Gateway
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
$0.57
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- input_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Input tokens / 1M
- Pricing via
- Vercel AI Gateway
$2.30
- Inference provider
- vercelAIGatewayCatalog
- Platform
- Vercel AI Gateway
- Unit
- output_tokens_per_million
- Currency
- USD
- Billing conditions
- model_type
- language
- quote_scope
- gateway_catalog
- billing_scope
- Vercel public catalog; endpoint prices may differ
- Billing label
- Output tokens / 1M
- Pricing via
- Vercel AI Gateway
Provider specifications & access
1–4 / 4
- source model id
- moonshotai/kimi-k2
- hosting
- vercel-gateway
- context length
- 131072
- max output tokens
- 131072
- source model id
- moonshotai/Kimi-K2-Instruct
- hosting
- siliconflow-hosted
- max output tokens
- 131072
- capabilities
- structuredOutputs
- toolCalling
- source model id
- publishers/hf-moonshotai/models/kimi-k2-instruct@001
- hosting
- vertex-self-deployed
- source model id
- publishers/moonshotai/models/kimi-k2@kimi-k2-instruct
- hosting
- vertex-self-deployed
Source specification observations
1–7 / 7
- field
- context_length
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- context_length
- value
- 131K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- context_length
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
- field
- knowledge_cutoff
- reported field
- knowledge_cutoff
- value
- 2024-12-31
- normalized value
- 2024-12-31
- precision
- day
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 98304
- normalized value
- 98304
- scope
- source-model
- source id
- openRouter
- confidence
- 1
- field
- max_output_tokens
- value
- 131K
- scope
- source-model
- source id
- siliconFlow
- confidence
- 1
- field
- max_output_tokens
- value
- 131072
- normalized value
- 131072
- scope
- source-model
- source id
- vercelAIGatewayCatalog
- confidence
- 1
Explore more models