Google: Gemini 2.5 Flash

Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, e…

google/gemini-2.5-flash

Gemini 2.5 Flash is Google's state-of-the-art workhorse model, specifically designed for advanced reasoning, coding, mathematics, and scientific tasks. It includes built-in "thinking" capabilities, enabling it to provide responses with greater accuracy and nuanced context handling.

Additionally, Gemini 2.5 Flash is configurable through the "max tokens for reasoning" parameter, as described in the documentation (https://openrouter.ai/docs/use-cases/reasoning-tokens#max-tokens-for-reasoning).

Specifications & capabilities

Input
file, image, text, audio, video
Output
text
Context window · tokens
1048576
Knowledge cutoff
2025-01
Published model ID
gemini-2.5-flash@default
Capabilities
audioInput, batch, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗

Evaluations

61–80 / 2,449

Kaggle Benchmarks

0.91

source id
kaggleBenchmarks
leaderboard name
Global MMLU Lite French · French
dimension
reasoning
metric
Global MMLU Lite French · French
rank
16
aggregation scope
kaggle
true
status
published
license
Apache-2.0
direction
higherIsBetter
conditions
task
id
41752
name
French
version
2
sort_order
DESCENDING
display_type
rawValues
benchmark
id
156
name
Global MMLU Lite French
type
INDIVIDUAL
owner
cohere-labs
version
1
categories
  • multilingual and cross-lingual capabilities
  • general knowledge and reasoning
version_id
138
maintainer_tier
Kaggle-maintained
maintenance_level
KAGGLE_MAINTAINED
candidate
candidate_class
modelVersion
record_kind
aggregate
attribution
Kaggle · cohere-labs · Global MMLU Lite French
license_url
https://www.apache.org/licenses/LICENSE-2.0
metric_kind
Global MMLU Lite French · French
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
138/leaderboard.json
source_page
https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-french/versions/1
measurements
  • name
    French
    unit
    score
    scale
    1
    metric
    Global MMLU Lite French · French
    result
    resultCase
    numericResult
    booleanResult
    false
    numericResult
    value
    0.91
    standardDeviation
    0
    confidenceInterval
    0.0280452971732717
    hasStandardDeviation
    false
    hasConfidenceInterval
    true
    confidenceIntervalNullable
    0.0280452971732717
    hasUnevenConfidenceInterval
    false
    taskVersionId
    0
    evaluationDate
    2025-10-30T00:00:00Z
    hasBooleanResult
    false
    hasNumericResult
    true
    hasTaskVersionId
    false
    hasEvaluationDate
    true
    numericResultNullable
    value
    0.91
    standardDeviation
    0
    confidenceInterval
    0.0280452971732717
    hasStandardDeviation
    false
    hasConfidenceInterval
    true
    confidenceIntervalNullable
    0.0280452971732717
    hasUnevenConfidenceInterval
    false
    hasNumericResultPublic
    false
    customAdditionalResults
      hasNumericResultPrivate
      false
      task_id
      41752
      task_url
      /benchmarks/tasks/benchmarkler/global-mmlu-lite-french
      sort_order
      DESCENDING
      source_name
      French
      display_type
      rawValues
      task_version
      2
      task_display_type
      RAW_VALUES
      normalization_status
      normalized
    published_at
    2025-10-17T15:30:43.548947500Z
    candidate_class
    modelVersion
    source_revision
    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
    publisher_anchor
    google/gemini-2.5-flash
    metric_population
    41
    benchmark_evaluation
    true
    task_metadata_source
    benchmarkVersion
    candidate_metadata_source
    benchmarkVersionMappings
    evaluated at
    2025-10-30T00:00:00.000Z
    source model id
    Gemini 2.5 Flash
    confidence lower
    0.8819547028267283
    confidence upper
    0.9380452971732718
    Original result ↗

    Kaggle Benchmarks

    0.908

    source id
    kaggleBenchmarks
    leaderboard name
    Global MMLU Lite Indonesian · Indonesian
    dimension
    reasoning
    metric
    Global MMLU Lite Indonesian · Indonesian
    rank
    16
    aggregation scope
    kaggle
    true
    status
    published
    license
    Apache-2.0
    direction
    higherIsBetter
    conditions
    task
    id
    41740
    name
    Indonesian
    version
    2
    sort_order
    DESCENDING
    display_type
    rawValues
    benchmark
    id
    158
    name
    Global MMLU Lite Indonesian
    type
    INDIVIDUAL
    owner
    cohere-labs
    version
    1
    categories
    • multilingual and cross-lingual capabilities
    • general knowledge and reasoning
    version_id
    140
    maintainer_tier
    Kaggle-maintained
    maintenance_level
    KAGGLE_MAINTAINED
    candidate
    candidate_class
    modelVersion
    record_kind
    aggregate
    attribution
    Kaggle · cohere-labs · Global MMLU Lite Indonesian
    license_url
    https://www.apache.org/licenses/LICENSE-2.0
    metric_kind
    Global MMLU Lite Indonesian · Indonesian
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    140/leaderboard.json
    source_page
    https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-indonesian/versions/1
    measurements
    • name
      Indonesian
      unit
      score
      scale
      1
      metric
      Global MMLU Lite Indonesian · Indonesian
      result
      resultCase
      numericResult
      booleanResult
      false
      numericResult
      value
      0.9075
      standardDeviation
      0
      confidenceInterval
      0.0283930651251164
      hasStandardDeviation
      false
      hasConfidenceInterval
      true
      confidenceIntervalNullable
      0.0283930651251164
      hasUnevenConfidenceInterval
      false
      taskVersionId
      0
      evaluationDate
      2025-10-30T00:00:00Z
      hasBooleanResult
      false
      hasNumericResult
      true
      hasTaskVersionId
      false
      hasEvaluationDate
      true
      numericResultNullable
      value
      0.9075
      standardDeviation
      0
      confidenceInterval
      0.0283930651251164
      hasStandardDeviation
      false
      hasConfidenceInterval
      true
      confidenceIntervalNullable
      0.0283930651251164
      hasUnevenConfidenceInterval
      false
      hasNumericResultPublic
      false
      customAdditionalResults
        hasNumericResultPrivate
        false
        task_id
        41740
        task_url
        /benchmarks/tasks/benchmarkler/global-mmlu-lite-indonesian
        sort_order
        DESCENDING
        source_name
        Indonesian
        display_type
        rawValues
        task_version
        2
        task_display_type
        RAW_VALUES
        normalization_status
        normalized
      published_at
      2025-10-17T15:30:39.240452500Z
      candidate_class
      modelVersion
      source_revision
      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
      publisher_anchor
      google/gemini-2.5-flash
      metric_population
      41
      benchmark_evaluation
      true
      task_metadata_source
      benchmarkVersion
      candidate_metadata_source
      benchmarkVersionMappings
      evaluated at
      2025-10-30T00:00:00.000Z
      source model id
      Gemini 2.5 Flash
      confidence lower
      0.8791069348748836
      confidence upper
      0.9358930651251164
      Original result ↗

      Kaggle Benchmarks

      0.915

      source id
      kaggleBenchmarks
      leaderboard name
      Global MMLU Lite Chinese · Chinese
      dimension
      reasoning
      metric
      Global MMLU Lite Chinese · Chinese
      rank
      17
      aggregation scope
      kaggle
      true
      status
      published
      license
      Apache-2.0
      direction
      higherIsBetter
      conditions
      task
      id
      41735
      name
      Chinese
      version
      2
      sort_order
      DESCENDING
      display_type
      rawValues
      benchmark
      id
      166
      name
      Global MMLU Lite Chinese
      type
      INDIVIDUAL
      owner
      cohere-labs
      version
      1
      categories
      • multilingual and cross-lingual capabilities
      • general knowledge and reasoning
      version_id
      148
      maintainer_tier
      Kaggle-maintained
      maintenance_level
      KAGGLE_MAINTAINED
      candidate
      candidate_class
      modelVersion
      record_kind
      aggregate
      attribution
      Kaggle · cohere-labs · Global MMLU Lite Chinese
      license_url
      https://www.apache.org/licenses/LICENSE-2.0
      metric_kind
      Global MMLU Lite Chinese · Chinese
      rank_method
      competition rank among reported evaluations in identical conditions, including unlinked observations
      source_file
      148/leaderboard.json
      source_page
      https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-chinese/versions/1
      measurements
      • name
        Chinese
        unit
        score
        scale
        1
        metric
        Global MMLU Lite Chinese · Chinese
        result
        resultCase
        numericResult
        booleanResult
        false
        numericResult
        value
        0.915
        standardDeviation
        0
        confidenceInterval
        0.0273299039414468
        hasStandardDeviation
        false
        hasConfidenceInterval
        true
        confidenceIntervalNullable
        0.0273299039414468
        hasUnevenConfidenceInterval
        false
        taskVersionId
        0
        evaluationDate
        2025-10-30T00:00:00Z
        hasBooleanResult
        false
        hasNumericResult
        true
        hasTaskVersionId
        false
        hasEvaluationDate
        true
        numericResultNullable
        value
        0.915
        standardDeviation
        0
        confidenceInterval
        0.0273299039414468
        hasStandardDeviation
        false
        hasConfidenceInterval
        true
        confidenceIntervalNullable
        0.0273299039414468
        hasUnevenConfidenceInterval
        false
        hasNumericResultPublic
        false
        customAdditionalResults
          hasNumericResultPrivate
          false
          task_id
          41735
          task_url
          /benchmarks/tasks/benchmarkler/global-mmlu-lite-chinese
          sort_order
          DESCENDING
          source_name
          Chinese
          display_type
          rawValues
          task_version
          2
          task_display_type
          RAW_VALUES
          normalization_status
          normalized
        published_at
        2025-10-17T15:30:22.953186100Z
        candidate_class
        modelVersion
        source_revision
        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
        publisher_anchor
        google/gemini-2.5-flash
        metric_population
        41
        benchmark_evaluation
        true
        task_metadata_source
        benchmarkVersion
        candidate_metadata_source
        benchmarkVersionMappings
        evaluated at
        2025-10-30T00:00:00.000Z
        source model id
        Gemini 2.5 Flash
        confidence lower
        0.8876700960585532
        confidence upper
        0.9423299039414469
        Original result ↗

        LMArena

        1,251.841

        source id
        lmarena
        leaderboard name
        LMArena Vision Entity Recognition
        dimension
        multimodal
        metric
        Arena rating
        rank
        18
        sample size
        628
        evaluated at
        2026-11-09T00:00:00.000Z
        source model id
        Gemini 2.5 Flash
        confidence lower
        1227.8505859692873
        confidence upper
        1275.8322586786646
        Original result ↗

        Kaggle Benchmarks

        0.913

        source id
        kaggleBenchmarks
        leaderboard name
        Global MMLU Lite Portuguese · Portuguese
        dimension
        reasoning
        metric
        Global MMLU Lite Portuguese · Portuguese
        rank
        18
        aggregation scope
        kaggle
        true
        status
        published
        license
        Apache-2.0
        direction
        higherIsBetter
        conditions
        task
        id
        41746
        name
        Portuguese
        version
        2
        sort_order
        DESCENDING
        display_type
        rawValues
        benchmark
        id
        162
        name
        Global MMLU Lite Portuguese
        type
        INDIVIDUAL
        owner
        cohere-labs
        version
        1
        categories
        • multilingual and cross-lingual capabilities
        • general knowledge and reasoning
        version_id
        144
        maintainer_tier
        Kaggle-maintained
        maintenance_level
        KAGGLE_MAINTAINED
        candidate
        candidate_class
        modelVersion
        record_kind
        aggregate
        attribution
        Kaggle · cohere-labs · Global MMLU Lite Portuguese
        license_url
        https://www.apache.org/licenses/LICENSE-2.0
        metric_kind
        Global MMLU Lite Portuguese · Portuguese
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        source_file
        144/leaderboard.json
        source_page
        https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-portuguese/versions/1
        measurements
        • name
          Portuguese
          unit
          score
          scale
          1
          metric
          Global MMLU Lite Portuguese · Portuguese
          result
          resultCase
          numericResult
          booleanResult
          false
          numericResult
          value
          0.9125
          standardDeviation
          0
          confidenceInterval
          0.0276909948229923
          hasStandardDeviation
          false
          hasConfidenceInterval
          true
          confidenceIntervalNullable
          0.0276909948229923
          hasUnevenConfidenceInterval
          false
          taskVersionId
          0
          evaluationDate
          2025-10-30T00:00:00Z
          hasBooleanResult
          false
          hasNumericResult
          true
          hasTaskVersionId
          false
          hasEvaluationDate
          true
          numericResultNullable
          value
          0.9125
          standardDeviation
          0
          confidenceInterval
          0.0276909948229923
          hasStandardDeviation
          false
          hasConfidenceInterval
          true
          confidenceIntervalNullable
          0.0276909948229923
          hasUnevenConfidenceInterval
          false
          hasNumericResultPublic
          false
          customAdditionalResults
            hasNumericResultPrivate
            false
            task_id
            41746
            task_url
            /benchmarks/tasks/benchmarkler/global-mmlu-lite-portuguese
            sort_order
            DESCENDING
            source_name
            Portuguese
            display_type
            rawValues
            task_version
            2
            task_display_type
            RAW_VALUES
            normalization_status
            normalized
          published_at
          2025-10-17T15:30:30.662890500Z
          candidate_class
          modelVersion
          source_revision
          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
          publisher_anchor
          google/gemini-2.5-flash
          metric_population
          41
          benchmark_evaluation
          true
          task_metadata_source
          benchmarkVersion
          candidate_metadata_source
          benchmarkVersionMappings
          evaluated at
          2025-10-30T00:00:00.000Z
          source model id
          Gemini 2.5 Flash
          confidence lower
          0.8848090051770077
          confidence upper
          0.9401909948229923
          Original result ↗

          Kaggle Benchmarks

          0.903

          source id
          kaggleBenchmarks
          leaderboard name
          Global MMLU Lite German · German
          dimension
          reasoning
          metric
          Global MMLU Lite German · German
          rank
          20
          aggregation scope
          kaggle
          true
          status
          published
          license
          Apache-2.0
          direction
          higherIsBetter
          conditions
          task
          id
          41738
          name
          German
          version
          2
          sort_order
          DESCENDING
          display_type
          rawValues
          benchmark
          id
          155
          name
          Global MMLU Lite German
          type
          INDIVIDUAL
          owner
          cohere-labs
          version
          1
          categories
          • multilingual and cross-lingual capabilities
          • general knowledge and reasoning
          version_id
          137
          maintainer_tier
          Kaggle-maintained
          maintenance_level
          KAGGLE_MAINTAINED
          candidate
          candidate_class
          modelVersion
          record_kind
          aggregate
          attribution
          Kaggle · cohere-labs · Global MMLU Lite German
          license_url
          https://www.apache.org/licenses/LICENSE-2.0
          metric_kind
          Global MMLU Lite German · German
          rank_method
          competition rank among reported evaluations in identical conditions, including unlinked observations
          source_file
          137/leaderboard.json
          source_page
          https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-german/versions/1
          measurements
          • name
            German
            unit
            score
            scale
            1
            metric
            Global MMLU Lite German · German
            result
            resultCase
            numericResult
            booleanResult
            false
            numericResult
            value
            0.9025
            standardDeviation
            0
            confidenceInterval
            0.0290699315059157
            hasStandardDeviation
            false
            hasConfidenceInterval
            true
            confidenceIntervalNullable
            0.0290699315059157
            hasUnevenConfidenceInterval
            false
            taskVersionId
            0
            evaluationDate
            2025-10-30T00:00:00Z
            hasBooleanResult
            false
            hasNumericResult
            true
            hasTaskVersionId
            false
            hasEvaluationDate
            true
            numericResultNullable
            value
            0.9025
            standardDeviation
            0
            confidenceInterval
            0.0290699315059157
            hasStandardDeviation
            false
            hasConfidenceInterval
            true
            confidenceIntervalNullable
            0.0290699315059157
            hasUnevenConfidenceInterval
            false
            hasNumericResultPublic
            false
            customAdditionalResults
              hasNumericResultPrivate
              false
              task_id
              41738
              task_url
              /benchmarks/tasks/benchmarkler/global-mmlu-lite-german
              sort_order
              DESCENDING
              source_name
              German
              display_type
              rawValues
              task_version
              2
              task_display_type
              RAW_VALUES
              normalization_status
              normalized
            published_at
            2025-10-17T15:30:45.767915600Z
            candidate_class
            modelVersion
            source_revision
            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
            publisher_anchor
            google/gemini-2.5-flash
            metric_population
            41
            benchmark_evaluation
            true
            task_metadata_source
            benchmarkVersion
            candidate_metadata_source
            benchmarkVersionMappings
            evaluated at
            2025-10-30T00:00:00.000Z
            source model id
            Gemini 2.5 Flash
            confidence lower
            0.8734300684940842
            confidence upper
            0.9315699315059157
            Original result ↗

            BFCL

            62%

            source id
            bfcl
            leaderboard name
            BFCL Web Search
            dimension
            agentic
            metric
            Web-search accuracy
            rank
            20
            Original result ↗

            LMArena

            1,221.155

            source id
            lmarena
            leaderboard name
            LMArena Vision Style Control Entity Recognition
            dimension
            multimodal
            metric
            Arena rating
            rank
            21
            sample size
            628
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Flash
            confidence lower
            1196.8978619359902
            confidence upper
            1245.411326152644
            Original result ↗

            BFCL

            88.08%

            source id
            bfcl
            leaderboard name
            BFCL Non-Live AST
            dimension
            toolUse
            metric
            Non-live AST accuracy
            rank
            21
            Original result ↗

            BFCL

            59%

            source id
            bfcl
            leaderboard name
            BFCL Web Search
            dimension
            agentic
            metric
            Web-search accuracy
            rank
            21
            Original result ↗

            Kaggle Benchmarks

            33.655%

            source id
            kaggleBenchmarks
            leaderboard name
            SimpleQA · simpleqa_task
            dimension
            knowledge
            metric
            SimpleQA · simpleqa_task
            rank
            27
            aggregation scope
            kaggle
            true
            status
            published
            license
            Apache-2.0
            direction
            higherIsBetter
            conditions
            task
            id
            40103
            name
            simpleqa_task
            version
            7
            sort_order
            DESCENDING
            display_type
            percentages
            benchmark
            id
            40
            name
            SimpleQA
            type
            INDIVIDUAL
            owner
            openai
            version
            1
            categories
            • factuality and grounding
            version_id
            26
            maintainer_tier
            Kaggle-maintained
            maintenance_level
            KAGGLE_MAINTAINED
            candidate
            candidate_class
            modelVersion
            record_kind
            aggregate
            attribution
            Kaggle · openai · SimpleQA
            license_url
            https://www.apache.org/licenses/LICENSE-2.0
            metric_kind
            SimpleQA · simpleqa_task
            rank_method
            competition rank among reported evaluations in identical conditions, including unlinked observations
            source_file
            26/leaderboard.json
            source_page
            https://www.kaggle.com/benchmarks/openai/simpleqa/versions/1
            measurements
            • name
              simpleqa_task
              unit
              percent
              scale
              1
              metric
              SimpleQA · simpleqa_task
              result
              resultCase
              numericResult
              booleanResult
              false
              numericResult
              value
              0.3365462780164905
              standardDeviation
              0
              confidenceInterval
              0.0143611653772581
              hasStandardDeviation
              false
              hasConfidenceInterval
              true
              confidenceIntervalNullable
              0.0143611653772581
              hasUnevenConfidenceInterval
              false
              taskVersionId
              0
              evaluationDate
              2026-03-10T00:00:00Z
              hasBooleanResult
              false
              hasNumericResult
              true
              hasTaskVersionId
              false
              hasEvaluationDate
              true
              numericResultNullable
              value
              0.3365462780164905
              standardDeviation
              0
              confidenceInterval
              0.0143611653772581
              hasStandardDeviation
              false
              hasConfidenceInterval
              true
              confidenceIntervalNullable
              0.0143611653772581
              hasUnevenConfidenceInterval
              false
              hasNumericResultPublic
              false
              customAdditionalResults
                hasNumericResultPrivate
                false
                task_id
                40103
                task_url
                /benchmarks/tasks/benchmarkler/simpleqa-task
                sort_order
                DESCENDING
                source_name
                simpleqa_task
                display_type
                percentages
                task_version
                7
                task_display_type
                PERCENTAGES
                normalization_status
                normalized
              published_at
              2025-07-15T11:49:31.790Z
              candidate_class
              modelVersion
              source_revision
              b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
              publisher_anchor
              google/gemini-2.5-flash
              metric_population
              49
              benchmark_evaluation
              true
              task_metadata_source
              benchmarkVersion
              candidate_metadata_source
              benchmarkVersionMappings
              evaluated at
              2026-03-10T00:00:00.000Z
              source model id
              Gemini 2.5 Flash
              confidence lower
              0.3221851126392324
              confidence upper
              0.35090744339374863
              Original result ↗

              Kaggle Benchmarks

              49.23%

              source id
              kaggleBenchmarks
              leaderboard name
              ParseBench: Doc OCR for AI agents · Score
              dimension
              knowledge
              metric
              ParseBench: Doc OCR for AI agents · Score
              rank
              28
              aggregation scope
              kaggle
              true
              status
              published
              license
              Apache-2.0
              direction
              higherIsBetter
              conditions
              task
              id
              392887
              name
              Score
              version
              4
              sort_order
              DESCENDING
              display_type
              percentages
              benchmark
              id
              4974
              name
              ParseBench: Doc OCR for AI agents
              type
              INDIVIDUAL
              owner
              llamaindex-org
              version
              1
              categories
              • factuality and grounding
              version_id
              4997
              maintainer_tier
              Kaggle-maintained
              maintenance_level
              KAGGLE_MAINTAINED
              candidate
              candidate_class
              modelVersion
              record_kind
              aggregate
              attribution
              Kaggle · llamaindex-org · ParseBench: Doc OCR for AI agents
              license_url
              https://www.apache.org/licenses/LICENSE-2.0
              metric_kind
              ParseBench: Doc OCR for AI agents · Score
              rank_method
              competition rank among reported evaluations in identical conditions, including unlinked observations
              source_file
              4997/leaderboard.json
              source_page
              https://www.kaggle.com/benchmarks/llamaindex-org/parsebench/versions/1
              measurements
              • name
                Score
                unit
                percent
                scale
                1
                metric
                ParseBench: Doc OCR for AI agents · Score
                result
                resultCase
                numericResult
                booleanResult
                false
                numericResult
                value
                0.49229519491630386
                standardDeviation
                0
                confidenceInterval
                0
                hasStandardDeviation
                false
                hasConfidenceInterval
                false
                hasUnevenConfidenceInterval
                false
                taskVersionId
                0
                evaluationDate
                2026-04-16T16:42:55.059072Z
                hasBooleanResult
                false
                hasNumericResult
                true
                hasTaskVersionId
                false
                hasEvaluationDate
                true
                numericResultNullable
                value
                0.49229519491630386
                standardDeviation
                0
                confidenceInterval
                0
                hasStandardDeviation
                false
                hasConfidenceInterval
                false
                hasUnevenConfidenceInterval
                false
                hasNumericResultPublic
                false
                customAdditionalResults
                  hasNumericResultPrivate
                  false
                  task_id
                  392887
                  task_url
                  /benchmarks/tasks/benchmarkler/parsebench
                  sort_order
                  DESCENDING
                  source_name
                  Score
                  display_type
                  percentages
                  task_version
                  4
                  task_display_type
                  PERCENTAGES
                  normalization_status
                  normalized
                published_at
                2026-04-17T23:39:17.199068200Z
                candidate_class
                modelVersion
                source_revision
                b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                publisher_anchor
                google/gemini-2.5-flash
                metric_population
                34
                benchmark_evaluation
                true
                task_metadata_source
                benchmarkVersion
                candidate_metadata_source
                benchmarkVersionMappings
                evaluated at
                2026-04-16T16:42:55.059Z
                source model id
                Gemini 2.5 Flash
                Original result ↗

                OpenRouter Benchmarks

                73.85%

                source id
                openRouterBenchmarks
                leaderboard name
                GPQA Diamond · Accuracy · Google
                dimension
                knowledge
                metric
                GPQA Diamond · Accuracy
                rank
                30
                sample size
                1188
                aggregation scope
                status
                published
                harness
                url
                https://github.com/OpenRouterTeam/benchmark-harness
                name
                OpenRouter benchmark harness
                direction
                higherIsBetter
                run_count
                6
                conditions
                window
                newest 90 days; successful visible nonzero runs
                dataset
                benchmark-leaderboard
                benchmark
                id
                gpqa_diamond
                name
                GPQA Diamond
                record_kind
                aggregate
                provider_name
                Google
                attribution
                Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
                last_run_at
                2026-10-01T19:15:31.968Z
                license_url
                https://creativecommons.org/licenses/by/4.0/
                methodology
                Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
                metric_kind
                GPQA Diamond · Accuracy
                rank_method
                competition rank among reported evaluations in identical conditions, including unlinked observations
                sample_size
                1188
                source_file
                benchmark-leaderboard.json
                source_page
                https://openrouter.ai/benchmarks/gpqa-diamond
                measurements
                accuracy
                0.7384959999999999
                accuracy_std_dev
                0.008170584087648423
                cost_per_task_usd
                0.06071549915824916
                output_tokens_per_task
                24199.07239057239
                latency_seconds_per_task
                125.90270959595959
                published_at
                2026-11-16T01:46:02.653Z
                snapshot_date
                2026-11-16
                source_revision
                7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
                transformations
                TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
                metric_population
                40
                benchmark_evaluation
                true
                openrouter_permaslug
                google/gemini-2.5-flash
                source model id
                google/gemini-2.5-flash
                Original result ↗

                LMArena

                1,411.346

                source id
                lmarena
                leaderboard name
                LMArena Text Factuality German
                dimension
                factuality
                metric
                Arena rating
                rank
                31
                sample size
                570
                evaluated at
                2026-11-09T00:00:00.000Z
                source model id
                Gemini 2.5 Flash
                confidence lower
                1389.1402422977237
                confidence upper
                1433.5524680951612
                Original result ↗

                OpenRouter Benchmarks

                59.111%

                source id
                openRouterBenchmarks
                leaderboard name
                tau2-bench Verified Airline · Accuracy · Google
                dimension
                agentic
                metric
                tau2-bench Verified Airline · Accuracy
                rank
                31
                sample size
                300
                aggregation scope
                status
                published
                harness
                url
                https://github.com/OpenRouterTeam/benchmark-harness
                name
                OpenRouter benchmark harness
                direction
                higherIsBetter
                run_count
                6
                conditions
                window
                newest 90 days; successful visible nonzero runs
                dataset
                benchmark-leaderboard
                benchmark
                id
                tau_bench_verified_airline
                name
                tau2-bench Verified Airline
                record_kind
                aggregate
                provider_name
                Google
                attribution
                Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
                last_run_at
                2026-10-01T19:05:46.198Z
                license_url
                https://creativecommons.org/licenses/by/4.0/
                methodology
                Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
                metric_kind
                tau2-bench Verified Airline · Accuracy
                rank_method
                competition rank among reported evaluations in identical conditions, including unlinked observations
                sample_size
                300
                source_file
                benchmark-leaderboard.json
                source_page
                https://openrouter.ai/benchmarks/tau2-bench-airline
                measurements
                accuracy
                0.591111
                accuracy_std_dev
                0.006285295405415193
                cost_per_task_usd
                0.029060479999999996
                output_tokens_per_task
                7763.98
                latency_seconds_per_task
                77.75587
                published_at
                2026-11-16T01:46:02.653Z
                snapshot_date
                2026-11-16
                source_revision
                7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
                transformations
                TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
                metric_population
                36
                benchmark_evaluation
                true
                openrouter_permaslug
                google/gemini-2.5-flash
                source model id
                google/gemini-2.5-flash
                Original result ↗

                BFCL

                36.25%

                source id
                bfcl
                leaderboard name
                BFCL Multi Turn
                dimension
                agentic
                metric
                Multi-turn accuracy
                rank
                32
                Original result ↗

                Published prices

                1–20 / 45

                gemini-2.5-flash ↗

                $0.02

                Inference provider
                cheaperInference
                Platform
                Cheaper Inference
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                zdr
                false
                protocol
                chat_completions
                service_tier
                standard
                Billing label
                Cached input tokens (per 1M)

                gemini-2.5-flash ↗

                $0.20

                Inference provider
                cheaperInference
                Platform
                Cheaper Inference
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions
                zdr
                false
                protocol
                chat_completions
                service_tier
                standard
                Billing label
                Input tokens (per 1M)

                gemini-2.5-flash ↗

                $1.63

                Inference provider
                cheaperInference
                Platform
                Cheaper Inference
                Unit
                output_tokens_per_million
                Currency
                USD
                Billing conditions
                zdr
                false
                protocol
                chat_completions
                service_tier
                standard
                Billing label
                Output tokens (per 1M)

                google/gemini-2.5-flash ↗

                $0.03

                Inference provider
                cloudflareWorkersAI
                Platform
                Google Vertex AI
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                Billing label
                Cached input tokens (per 1M)
                Pricing via
                Cloudflare AI Gateway

                google/gemini-2.5-flash ↗

                $0.30

                Inference provider
                cloudflareWorkersAI
                Platform
                Google Vertex AI
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions
                Billing label
                Input tokens (per 1M)
                Pricing via
                Cloudflare AI Gateway

                google/gemini-2.5-flash ↗

                $2.50

                Inference provider
                cloudflareWorkersAI
                Platform
                Google Vertex AI
                Unit
                output_tokens_per_million
                Currency
                USD
                Billing conditions
                Billing label
                Output tokens (per 1M)
                Pricing via
                Cloudflare AI Gateway

                google/gemini-2.5-flash ↗

                $0.03

                Inference provider
                openRouter
                Platform
                OpenRouter
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-flash ↗

                $0.08

                Inference provider
                openRouter
                Platform
                OpenRouter
                Unit
                cache_write_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-flash ↗

                $0.30

                Inference provider
                openRouter
                Platform
                OpenRouter
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-flash ↗

                $2.50

                Inference provider
                openRouter
                Platform
                OpenRouter
                Unit
                output_tokens_per_million
                Currency
                USD
                Billing conditions

                google/gemini-2.5-flash ↗

                $0.03

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-flash ↗

                $0.05

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                service_tier
                priority
                billing_scope
                Vercel public catalog; endpoint prices may differ
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-flash ↗

                $0.03

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                region
                eu
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-flash ↗

                $0.03

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                cache_read_tokens_per_million
                Currency
                USD
                Billing conditions
                region
                us
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                Billing label
                Cached input / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-flash ↗

                $0.30

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                billing_scope
                Vercel public catalog; endpoint prices may differ
                Billing label
                Input tokens / 1M
                Pricing via
                Vercel AI Gateway

                google/gemini-2.5-flash ↗

                $0.15

                Inference provider
                vercelAIGatewayCatalog
                Platform
                Vercel AI Gateway
                Unit
                input_tokens_per_million
                Currency
                USD
                Billing conditions
                model_type
                language
                quote_scope
                gateway_catalog
                service_tier
                flex
                billing_scope
                Vercel public catalog; endpoint prices may differ
                Billing label
                Input tokens / 1M
                Pricing via
                Vercel AI Gateway

                Provider specifications & access

                1–4 / 4

                gemini-2.5-flash ↗

                source model id
                gemini-2.5-flash
                hosting
                gateway
                platforms
                • Cheaper Inference
                context length
                1048576
                max output tokens
                65535
                architecture
                input_modalities
                • text
                • image
                output_modalities
                • text
                capabilities
                • structuredOutputs
                • toolCalling
                • visionInput
                • webSearch

                Gemini 2.5 Flash ↗

                source model id
                google/gemini-2.5-flash
                hosting
                third-party
                platforms
                • Google Vertex AI
                context length
                1000000
                max output tokens
                8192
                architecture
                input_modalities
                • text
                output_modalities
                • text
                capabilities
                • reasoning

                Gemini 2.5 Flash ↗

                source model id
                google/gemini-2.5-flash
                hosting
                vercel-gateway
                platforms
                • Vercel AI Gateway
                context length
                1000000
                max output tokens
                65535
                architecture
                input_modalities
                • text
                • image
                • pdf
                output_modalities
                • text
                capabilities
                • promptCaching
                • reasoning
                • structuredOutputs
                • toolCalling
                • visionInput
                • webSearch
                regions
                • eu
                • us

                Gemini 2.5 Flash ↗

                source model id
                publishers/google/models/gemini-2.5-flash@default
                hosting
                vertex-managed
                platforms
                • Google Cloud Vertex AI
                context length
                1048576
                max output tokens
                65536
                architecture
                input_modalities
                • audio
                • image
                • text
                • video
                output_modalities
                • text
                capabilities
                • batch
                • multimodal
                • promptCaching
                • reasoning
                • structuredOutputs
                • toolCalling
                • visionInput

                Safety assessments

                1–1 / 1

                Foundation Model Transparency Index 2025 ↗

                41 / 100

                source id
                stanfordFMTI
                canonical model id
                google/gemini-2.5-flash
                taxonomy
                transparency
                hazard category
                transparency
                hazard name
                Organization transparency
                policy area
                Google
                assessment type
                organization-level disclosure
                metric
                disclosure_score
                result label
                41 / 100
                sample size
                100
                updated at
                2026-11-09T18:08:05.227Z

                Source specification observations

                1–10 / 10

                cloudflareWorkersAI ↗

                field
                context_length
                value
                1000000
                normalized value
                1000000
                scope
                source-model
                source id
                cloudflareWorkersAI
                confidence
                1

                openRouter ↗

                field
                context_length
                value
                1048576
                normalized value
                1048576
                scope
                source-model
                source id
                openRouter
                confidence
                1

                vercelAIGatewayCatalog ↗

                field
                context_length
                value
                1000000
                normalized value
                1000000
                scope
                source-model
                source id
                vercelAIGatewayCatalog
                confidence
                1

                vertexAI ↗

                field
                context_length
                value
                1048576
                normalized value
                1048576
                scope
                source-model
                source id
                vertexAI
                confidence
                1

                openRouter ↗

                field
                knowledge_cutoff
                reported field
                knowledge_cutoff
                value
                2025-01-31
                normalized value
                2025-01-31
                precision
                day
                scope
                source-model
                source id
                openRouter
                confidence
                1

                vercelAIGatewayCatalog ↗

                field
                knowledge_cutoff
                reported field
                knowledge_cutoff
                value
                2025-01
                normalized value
                2025-01
                precision
                month
                scope
                source-model
                source id
                vercelAIGatewayCatalog
                confidence
                1

                cloudflareWorkersAI ↗

                field
                max_output_tokens
                value
                8192
                normalized value
                8192
                scope
                source-model
                source id
                cloudflareWorkersAI
                confidence
                1

                openRouter ↗

                field
                max_output_tokens
                value
                65535
                normalized value
                65535
                scope
                source-model
                source id
                openRouter
                confidence
                1

                vercelAIGatewayCatalog ↗

                field
                max_output_tokens
                value
                65535
                normalized value
                65535
                scope
                source-model
                source id
                vercelAIGatewayCatalog
                confidence
                1

                vertexAI ↗

                field
                max_output_tokens
                value
                65536
                normalized value
                65536
                scope
                source-model
                source id
                vertexAI
                confidence
                1

                Original sources

                1–20 / 22

                Google DeepMind · Model reference ↗

                Google DeepMind · Model reference ↗

                DeepInfra · Model page ↗

                Google Cloud · Model page ↗

                OpenRouter · Model page ↗

                Vercel AI Gateway · Model page ↗

                Google Cloud · Documentation ↗

                GitHub · Source snapshot ↗

                api.cheaperinference.com · Source snapshot ↗

                Cloudflare AI · Catalog ↗

                DeepInfra · Catalog ↗

                Epoch AI · Catalog ↗

                GitHub · Catalog ↗

                GitHub · Catalog ↗

                Google Cloud · Catalog ↗

                Kaggle · Catalog ↗

                LMArena · Catalog ↗

                OpenRouter · Catalog ↗

                OpenRouter · Catalog ↗

                Vercel AI Gateway · Catalog ↗

                Source links

                1–15 / 15

                Explore more models

                Models

                Loading
                EVIDENCE / COMPARISON

                Benchmarks

                Explore every benchmark

                Loading

                These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                Install TensorTrace