mistralai/Mistral-Small-24B-Base-2501
Explore mistralai/Mistral-Small-24B-Base-2501: model specifications, capabilities, benchmark results, platform pricing and original sources.
huggingface/mistralai/Mistral-Small-24B-Base-2501
Specifications & capabilities
- Input
- text
- Output
- text
- Published model ID
- Mistral-Small-24B-Base-2501
Evaluations
1–15 / 15
0.696
- source id
- llmStatsArchive
- leaderboard name
- MBPP · Normalized (reported)
- dimension
- general
- metric
- MBPP · Normalized (reported)
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MBPP
- language
- en
- modality
- text
- verified
- false
- max_score
- 100
- created_at
- 2025-07-19T19:56:13.453370+00:00
- paper_link
- https://arxiv.org/abs/2108.07732
- updated_at
- 2025-07-19T19:56:13.453370+00:00
- description
- MBPP (Mostly Basic Python Problems) is a benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers. Each problem consists of a task description, code solution, and 3 automated test cases covering programming fundamentals and standard library functionality.
- benchmark_id
- mbpp
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MBPP · Normalized (reported)
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/mistral/models/mistral-small-24b-base-2501/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/mistralai/Mistral-Small-24B-Base-2501
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- mistral/mistral-small-24b-base-2501
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- mistral-small-24b-base-2501
Original result ↗0.696
- source id
- llmStatsArchive
- leaderboard name
- MBPP · Raw
- dimension
- general
- metric
- MBPP · Raw
- rank
- 2
- aggregation scope
- status
- archived
- license
- CC-BY-4.0
- direction
- higherIsBetter
- conditions
- language
- en
- benchmark
- name
- MBPP
- language
- en
- modality
- text
- verified
- false
- max_score
- 100
- created_at
- 2025-07-19T19:56:13.453370+00:00
- paper_link
- https://arxiv.org/abs/2108.07732
- updated_at
- 2025-07-19T19:56:13.453370+00:00
- description
- MBPP (Mostly Basic Python Problems) is a benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers. Each problem consists of a task description, code solution, and 3 automated test cases covering programming fundamentals and standard library functionality.
- benchmark_id
- mbpp
- multilingual
- false
- analysis_method
- Pass@1
- is_self_reported
- true
- verified_by_llmstats
- false
- attribution
- Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
- license_url
- https://creativecommons.org/licenses/by/4.0/
- metric_kind
- MBPP · Raw
- rank_method
- competition rank among reported evaluations in identical conditions, including unlinked observations
- source_file
- data/organizations/mistral/models/mistral-small-24b-base-2501/benchmarks.json
- archive_kind
- llmStats
- evidence_url
- https://huggingface.co/mistralai/Mistral-Small-24B-Base-2501
- git_revision
- 872b75f63b8d853c46ed17856424029a0c47b1d2
- published_at
- 2025-10-24T17:47:59Z
- snapshot_date
- 2025-10-24T17:47:59Z
- source_revision
- 6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
- publisher_anchor
- mistral/mistral-small-24b-base-2501
- metric_population
- 3
- current_catalog_url
- https://llm-stats.com/
- benchmark_evaluation
- true
- source model id
- mistral-small-24b-base-2501
Original result ↗38.758%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 GPQA · Raw
- dimension
- knowledge
- metric
- Raw accuracy
- rank
- 139
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Raw accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗18.345%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 GPQA · Normalized
- dimension
- knowledge
- metric
- Normalized score
- rank
- 139
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗54.064%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 MMLU-PRO · Raw
- dimension
- knowledge
- metric
- Raw accuracy
- rank
- 155
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Raw accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗48.96%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 MMLU-PRO · Normalized
- dimension
- knowledge
- metric
- Normalized score
- rank
- 155
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗64.419%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 BBH · Raw
- dimension
- reasoning
- metric
- Raw accuracy
- rank
- 483
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Raw accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗48.538%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 BBH · Normalized
- dimension
- reasoning
- metric
- Normalized score
- rank
- 513
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗19.713%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 MATH Lvl 5 · Raw
- dimension
- math
- metric
- Raw accuracy
- rank
- 1339
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Raw accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗19.713%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 MATH Lvl 5 · Normalized
- dimension
- math
- metric
- Normalized score
- rank
- 1339
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗27.195%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 Average
- dimension
- general
- metric
- Normalized six-task average
- rank
- 1370
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized six-task average
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗42.367%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 MUSR · Raw
- dimension
- reasoning
- metric
- Raw accuracy
- rank
- 1630
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Raw accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗10.892%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 MUSR · Normalized
- dimension
- reasoning
- metric
- Normalized score
- rank
- 2017
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗16.724%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 IFEval · Raw
- dimension
- instructionFollowing
- metric
- Raw accuracy
- rank
- 4222
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Raw accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗16.724%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 IFEval · Normalized
- dimension
- instructionFollowing
- metric
- Normalized score
- rank
- 4222
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 4.275624553104299
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- false
- suite_version
- 2
- model_revision
- a8f19b61efa3c0a8d8a5f901ed48b30ff6b8c70e
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- mistralai/Mistral-Small-24B-Base-2501
Original result ↗Safety assessments
1–1 / 1
18 / 100
- source id
- stanfordFMTI
- canonical model id
- huggingface/mistralai/Mistral-Small-24B-Base-2501
- taxonomy
- transparency
- hazard category
- transparency
- hazard name
- Organization transparency
- policy area
- Mistral
- assessment type
- organization-level disclosure
- metric
- disclosure_score
- result label
- 18 / 100
- sample size
- 100
- updated at
- 2026-11-09T18:08:05.227Z
Explore more models