Mistral: Mistral Small 3.1 24B

Mistral Small 3.1 24B Instruct is an upgraded variant of Mistral Small 3 (2501), featuring 24 billion parameters with advanced multimodal capabilities. It provides state-of-the-art performance in tex…

mistralai/mistral-small-3.1-24b-instruct-2503

Mistral Small 3.1 24B Instruct is an upgraded variant of Mistral Small 3 (2501), featuring 24 billion parameters with advanced multimodal capabilities. It provides state-of-the-art performance in text-based reasoning and vision tasks, including image analysis, programming, mathematical reasoning, and multilingual support across dozens of languages. Equipped with an extensive 128k token context window and optimized for efficient local inference, it supports use cases such as conversational agents, function calling, long-document comprehension, and privacy-sensitive deployments. The updated version is [Mistral Small 3.2](mistralai/mistral-small-3.2-24b-instruct)

Specifications & capabilities

Input
text, image
Output
text
Context window · tokens
128000
Knowledge cutoff
2023-10-31
Published model ID
Mistral-Small-3.1-24B-Instruct-2503
Capabilities
multimodal, structuredOutputs, toolCalling, visionInput
Description source ↗

Evaluations

1–2 / 2

LLM Stats · GitHub archive

0.593

source id
llmStatsArchive
leaderboard name
MMMU · Raw
dimension
general
metric
MMMU · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMU
language
en
modality
multimodal
verified
false
max_score
1
categories
  • multimodal
  • reasoning
  • general
created_at
2025-07-19T19:56:12.130105+00:00
paper_link
https://arxiv.org/abs/2311.16502
updated_at
2025-07-19T19:56:12.130105+00:00
description
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
benchmark_id
mmmu
multilingual
false
analysis_method
CoT accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMU · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/mistral/models/mistral-small-3.1-24b-instruct-2503/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
mistral/mistral-small-3.1-24b-instruct-2503
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
mistral-small-3.1-24b-instruct-2503
Original result ↗

LLM Stats · GitHub archive

0.593

source id
llmStatsArchive
leaderboard name
MMMU · Normalized (reported)
dimension
general
metric
MMMU · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MMMU
language
en
modality
multimodal
verified
false
max_score
1
categories
  • multimodal
  • reasoning
  • general
created_at
2025-07-19T19:56:12.130105+00:00
paper_link
https://arxiv.org/abs/2311.16502
updated_at
2025-07-19T19:56:12.130105+00:00
description
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.
benchmark_id
mmmu
multilingual
false
analysis_method
CoT accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MMMU · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/mistral/models/mistral-small-3.1-24b-instruct-2503/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
mistral/mistral-small-3.1-24b-instruct-2503
metric_population
3
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
mistral-small-3.1-24b-instruct-2503
Original result ↗

Published prices

1–6 / 6

Provider specifications & access

1–4 / 4

mistralai/Mistral-Small-3.1-24B-Instruct-2503 ↗

source model id
azure-huggingface/mistralai--mistral-small-3.1-24b-instruct-2503@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
context length
131072
architecture
input_modalities
  • text
  • image
output_modalities
  • text
capabilities

    mistralai/Mistral-Small-3.1-24B-Instruct-2503 ↗

    source model id
    azure-huggingface/mistralai--mistral-small-3.1-24b-instruct-2503@2
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    license
    apache-2.0
    context length
    131072
    architecture
    input_modalities
    • text
    • image
    output_modalities
    • text
    capabilities

      mistralai/Mistral-Small-3.1-24B-Instruct-2503 ↗

      source model id
      azure-huggingface/mistralai--mistral-small-3.1-24b-instruct-2503@3
      hosting
      foundry-managed-compute
      platforms
      • Microsoft Foundry
      license
      apache-2.0
      context length
      131072
      architecture
      input_modalities
      • text
      • image
      output_modalities
      • text
      capabilities

        mistral-small-3.1-24b-instruct ↗

        source model id
        @cf/mistralai/mistral-small-3.1-24b-instruct
        hosting
        cloudflare-hosted
        platforms
        • Cloudflare Workers AI
        context length
        128000
        architecture
        input_modalities
        • text
        output_modalities
        • text
        capabilities
        • toolCalling

        Safety assessments

        1–1 / 1

        Foundation Model Transparency Index 2025 ↗

        18 / 100

        source id
        stanfordFMTI
        canonical model id
        mistralai/mistral-small-3.1-24b-instruct-2503
        taxonomy
        transparency
        hazard category
        transparency
        hazard name
        Organization transparency
        policy area
        Mistral
        assessment type
        organization-level disclosure
        metric
        disclosure_score
        result label
        18 / 100
        sample size
        100
        updated at
        2026-11-09T18:08:05.227Z

        Source specification observations

        1–6 / 6

        azureFoundry ↗

        field
        context_length
        value
        131072
        normalized value
        131072
        scope
        source-model
        source id
        azureFoundry
        confidence
        1

        azureFoundry ↗

        field
        context_length
        value
        131072
        normalized value
        131072
        scope
        source-model
        source id
        azureFoundry
        confidence
        1

        azureFoundry ↗

        field
        context_length
        value
        131072
        normalized value
        131072
        scope
        source-model
        source id
        azureFoundry
        confidence
        1

        openRouter ↗

        field
        context_length
        value
        128000
        normalized value
        128000
        scope
        source-model
        source id
        openRouter
        confidence
        1

        openRouter ↗

        field
        knowledge_cutoff
        reported field
        knowledge_cutoff
        value
        2023-10-31
        normalized value
        2023-10-31
        precision
        day
        scope
        source-model
        source id
        openRouter
        confidence
        1

        openRouter ↗

        field
        max_output_tokens
        value
        102400
        normalized value
        102400
        scope
        source-model
        source id
        openRouter
        confidence
        1

        Original sources

        1–16 / 16

        Hugging Face · mistralai/Mistral-Small-3.1-24B-Instruct-2503 ↗

        Cloudflare AI · Model page ↗

        DeepInfra · Model page ↗

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model page ↗

        OpenRouter · Model page ↗

        Microsoft Foundry · Model documentation ↗

        GitHub · Source snapshot ↗

        Cloudflare AI · Catalog ↗

        DeepInfra · Catalog ↗

        GitHub · Catalog ↗

        GitHub · Catalog ↗

        IBM · Catalog ↗

        Microsoft Foundry · Catalog ↗

        OpenRouter · Catalog ↗

        Source links

        1–8 / 8

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace