Google: Gemini 2.5 Pro

Gemini 2.5 Pro is Google’s state-of-the-art AI model designed for advanced reasoning, coding, mathematics, and scientific tasks. It employs “thinking” capabilities, enabling it to reason through resp…

google/gemini-2.5-pro

Gemini 2.5 Pro is Google’s state-of-the-art AI model designed for advanced reasoning, coding, mathematics, and scientific tasks. It employs “thinking” capabilities, enabling it to reason through responses with enhanced accuracy and nuanced context handling. Gemini 2.5 Pro achieves top-tier performance on multiple benchmarks, including first-place positioning on the LMArena leaderboard, reflecting superior human-preference alignment and complex problem-solving abilities.

Specifications & capabilities

Input
text, image, file, audio, video
Output
text
Context window · tokens
1048576
Knowledge cutoff
2025-01
Published model ID
gemini-2.5-pro@default
Capabilities
audioInput, batch, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗

Evaluations

61–80 / 1,319

Kaggle Benchmarks

0.93

source id
kaggleBenchmarks
leaderboard name
Global MMLU Lite German · German
dimension
reasoning
metric
Global MMLU Lite German · German
rank
14
aggregation scope
kaggle
true
status
published
license
Apache-2.0
direction
higherIsBetter
conditions
task
id
41738
name
German
version
2
sort_order
DESCENDING
display_type
rawValues
benchmark
id
155
name
Global MMLU Lite German
type
INDIVIDUAL
owner
cohere-labs
version
1
categories
  • multilingual and cross-lingual capabilities
  • general knowledge and reasoning
version_id
137
maintainer_tier
Kaggle-maintained
maintenance_level
KAGGLE_MAINTAINED
candidate
candidate_class
modelVersion
record_kind
aggregate
attribution
Kaggle · cohere-labs · Global MMLU Lite German
license_url
https://www.apache.org/licenses/LICENSE-2.0
metric_kind
Global MMLU Lite German · German
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
137/leaderboard.json
source_page
https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-german/versions/1
measurements
  • name
    German
    unit
    score
    scale
    1
    metric
    Global MMLU Lite German · German
    result
    resultCase
    numericResult
    booleanResult
    false
    numericResult
    value
    0.93
    standardDeviation
    0
    confidenceInterval
    0.0250039481496016
    hasStandardDeviation
    false
    hasConfidenceInterval
    true
    confidenceIntervalNullable
    0.0250039481496016
    hasUnevenConfidenceInterval
    false
    taskVersionId
    0
    evaluationDate
    2025-10-30T00:00:00Z
    hasBooleanResult
    false
    hasNumericResult
    true
    hasTaskVersionId
    false
    hasEvaluationDate
    true
    numericResultNullable
    value
    0.93
    standardDeviation
    0
    confidenceInterval
    0.0250039481496016
    hasStandardDeviation
    false
    hasConfidenceInterval
    true
    confidenceIntervalNullable
    0.0250039481496016
    hasUnevenConfidenceInterval
    false
    hasNumericResultPublic
    false
    customAdditionalResults
      hasNumericResultPrivate
      false
      task_id
      41738
      task_url
      /benchmarks/tasks/benchmarkler/global-mmlu-lite-german
      sort_order
      DESCENDING
      source_name
      German
      display_type
      rawValues
      task_version
      2
      task_display_type
      RAW_VALUES
      normalization_status
      normalized
    published_at
    2025-10-17T15:30:45.767915600Z
    candidate_class
    modelVersion
    source_revision
    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
    publisher_anchor
    google/gemini-2.5-pro
    metric_population
    41
    benchmark_evaluation
    true
    task_metadata_source
    benchmarkVersion
    candidate_metadata_source
    benchmarkVersionMappings
    evaluated at
    2025-10-30T00:00:00.000Z
    source model id
    Gemini 2.5 Pro
    confidence lower
    0.9049960518503984
    confidence upper
    0.9550039481496017
    Original result ↗

    Kaggle Benchmarks

    0.924

    source id
    kaggleBenchmarks
    leaderboard name
    Global MMLU Lite Culturally Sensitive · CS Score
    dimension
    reasoning
    metric
    Global MMLU Lite Culturally Sensitive · CS Score
    rank
    14
    aggregation scope
    kaggle
    true
    status
    published
    license
    Apache-2.0
    direction
    higherIsBetter
    conditions
    task
    id
    41747
    name
    CS Score
    version
    2
    sort_order
    DESCENDING
    display_type
    rawValues
    benchmark
    id
    208
    name
    Global MMLU Lite Culturally Sensitive
    type
    INDIVIDUAL
    owner
    cohere-labs
    version
    1
    categories
    • multilingual and cross-lingual capabilities
    • general knowledge and reasoning
    version_id
    187
    maintainer_tier
    Kaggle-maintained
    maintenance_level
    KAGGLE_MAINTAINED
    candidate
    candidate_class
    modelVersion
    record_kind
    aggregate
    attribution
    Kaggle · cohere-labs · Global MMLU Lite Culturally Sensitive
    license_url
    https://www.apache.org/licenses/LICENSE-2.0
    metric_kind
    Global MMLU Lite Culturally Sensitive · CS Score
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    187/leaderboard.json
    source_page
    https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-cs/versions/1
    measurements
    • name
      CS Score
      unit
      score
      scale
      1
      metric
      Global MMLU Lite Culturally Sensitive · CS Score
      result
      resultCase
      numericResult
      booleanResult
      false
      numericResult
      value
      0.9240625
      standardDeviation
      0
      confidenceInterval
      0
      hasStandardDeviation
      false
      hasConfidenceInterval
      false
      hasUnevenConfidenceInterval
      false
      taskVersionId
      0
      evaluationDate
      2025-11-03T22:31:06.2566667Z
      hasBooleanResult
      false
      hasNumericResult
      true
      hasTaskVersionId
      false
      hasEvaluationDate
      true
      numericResultNullable
      value
      0.9240625
      standardDeviation
      0
      confidenceInterval
      0
      hasStandardDeviation
      false
      hasConfidenceInterval
      false
      hasUnevenConfidenceInterval
      false
      hasNumericResultPublic
      false
      customAdditionalResults
        hasNumericResultPrivate
        false
        task_id
        41747
        task_url
        /benchmarks/tasks/benchmarkler/global-mmlu-lite-cs-score
        sort_order
        DESCENDING
        source_name
        CS Score
        display_type
        rawValues
        task_version
        2
        task_display_type
        RAW_VALUES
        normalization_status
        normalized
      published_at
      2025-11-10T16:52:15.372518Z
      candidate_class
      modelVersion
      source_revision
      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
      publisher_anchor
      google/gemini-2.5-pro
      metric_population
      41
      benchmark_evaluation
      true
      task_metadata_source
      benchmarkVersion
      candidate_metadata_source
      benchmarkVersionMappings
      evaluated at
      2025-11-03T22:31:06.256Z
      source model id
      Gemini 2.5 Pro
      Original result ↗

      Kaggle Benchmarks

      0.93

      source id
      kaggleBenchmarks
      leaderboard name
      Global MMLU Lite Portuguese · Portuguese
      dimension
      reasoning
      metric
      Global MMLU Lite Portuguese · Portuguese
      rank
      15
      aggregation scope
      kaggle
      true
      status
      published
      license
      Apache-2.0
      direction
      higherIsBetter
      conditions
      task
      id
      41746
      name
      Portuguese
      version
      2
      sort_order
      DESCENDING
      display_type
      rawValues
      benchmark
      id
      162
      name
      Global MMLU Lite Portuguese
      type
      INDIVIDUAL
      owner
      cohere-labs
      version
      1
      categories
      • multilingual and cross-lingual capabilities
      • general knowledge and reasoning
      version_id
      144
      maintainer_tier
      Kaggle-maintained
      maintenance_level
      KAGGLE_MAINTAINED
      candidate
      candidate_class
      modelVersion
      record_kind
      aggregate
      attribution
      Kaggle · cohere-labs · Global MMLU Lite Portuguese
      license_url
      https://www.apache.org/licenses/LICENSE-2.0
      metric_kind
      Global MMLU Lite Portuguese · Portuguese
      rank_method
      competition rank among reported evaluations in identical conditions, including unlinked observations
      source_file
      144/leaderboard.json
      source_page
      https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-portuguese/versions/1
      measurements
      • name
        Portuguese
        unit
        score
        scale
        1
        metric
        Global MMLU Lite Portuguese · Portuguese
        result
        resultCase
        numericResult
        booleanResult
        false
        numericResult
        value
        0.93
        standardDeviation
        0
        confidenceInterval
        0.0250039481496016
        hasStandardDeviation
        false
        hasConfidenceInterval
        true
        confidenceIntervalNullable
        0.0250039481496016
        hasUnevenConfidenceInterval
        false
        taskVersionId
        0
        evaluationDate
        2025-10-30T00:00:00Z
        hasBooleanResult
        false
        hasNumericResult
        true
        hasTaskVersionId
        false
        hasEvaluationDate
        true
        numericResultNullable
        value
        0.93
        standardDeviation
        0
        confidenceInterval
        0.0250039481496016
        hasStandardDeviation
        false
        hasConfidenceInterval
        true
        confidenceIntervalNullable
        0.0250039481496016
        hasUnevenConfidenceInterval
        false
        hasNumericResultPublic
        false
        customAdditionalResults
          hasNumericResultPrivate
          false
          task_id
          41746
          task_url
          /benchmarks/tasks/benchmarkler/global-mmlu-lite-portuguese
          sort_order
          DESCENDING
          source_name
          Portuguese
          display_type
          rawValues
          task_version
          2
          task_display_type
          RAW_VALUES
          normalization_status
          normalized
        published_at
        2025-10-17T15:30:30.662890500Z
        candidate_class
        modelVersion
        source_revision
        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
        publisher_anchor
        google/gemini-2.5-pro
        metric_population
        41
        benchmark_evaluation
        true
        task_metadata_source
        benchmarkVersion
        candidate_metadata_source
        benchmarkVersionMappings
        evaluated at
        2025-10-30T00:00:00.000Z
        source model id
        Gemini 2.5 Pro
        confidence lower
        0.9049960518503984
        confidence upper
        0.9550039481496017
        Original result ↗

        Kaggle Benchmarks

        0.928

        source id
        kaggleBenchmarks
        leaderboard name
        Global MMLU Lite Chinese · Chinese
        dimension
        reasoning
        metric
        Global MMLU Lite Chinese · Chinese
        rank
        15
        aggregation scope
        kaggle
        true
        status
        published
        license
        Apache-2.0
        direction
        higherIsBetter
        conditions
        task
        id
        41735
        name
        Chinese
        version
        2
        sort_order
        DESCENDING
        display_type
        rawValues
        benchmark
        id
        166
        name
        Global MMLU Lite Chinese
        type
        INDIVIDUAL
        owner
        cohere-labs
        version
        1
        categories
        • multilingual and cross-lingual capabilities
        • general knowledge and reasoning
        version_id
        148
        maintainer_tier
        Kaggle-maintained
        maintenance_level
        KAGGLE_MAINTAINED
        candidate
        candidate_class
        modelVersion
        record_kind
        aggregate
        attribution
        Kaggle · cohere-labs · Global MMLU Lite Chinese
        license_url
        https://www.apache.org/licenses/LICENSE-2.0
        metric_kind
        Global MMLU Lite Chinese · Chinese
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        source_file
        148/leaderboard.json
        source_page
        https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-chinese/versions/1
        measurements
        • name
          Chinese
          unit
          score
          scale
          1
          metric
          Global MMLU Lite Chinese · Chinese
          result
          resultCase
          numericResult
          booleanResult
          false
          numericResult
          value
          0.9275
          standardDeviation
          0
          confidenceInterval
          0.0254123049217328
          hasStandardDeviation
          false
          hasConfidenceInterval
          true
          confidenceIntervalNullable
          0.0254123049217328
          hasUnevenConfidenceInterval
          false
          taskVersionId
          0
          evaluationDate
          2025-10-30T00:00:00Z
          hasBooleanResult
          false
          hasNumericResult
          true
          hasTaskVersionId
          false
          hasEvaluationDate
          true
          numericResultNullable
          value
          0.9275
          standardDeviation
          0
          confidenceInterval
          0.0254123049217328
          hasStandardDeviation
          false
          hasConfidenceInterval
          true
          confidenceIntervalNullable
          0.0254123049217328
          hasUnevenConfidenceInterval
          false
          hasNumericResultPublic
          false
          customAdditionalResults
            hasNumericResultPrivate
            false
            task_id
            41735
            task_url
            /benchmarks/tasks/benchmarkler/global-mmlu-lite-chinese
            sort_order
            DESCENDING
            source_name
            Chinese
            display_type
            rawValues
            task_version
            2
            task_display_type
            RAW_VALUES
            normalization_status
            normalized
          published_at
          2025-10-17T15:30:22.953186100Z
          candidate_class
          modelVersion
          source_revision
          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
          publisher_anchor
          google/gemini-2.5-pro
          metric_population
          41
          benchmark_evaluation
          true
          task_metadata_source
          benchmarkVersion
          candidate_metadata_source
          benchmarkVersionMappings
          evaluated at
          2025-10-30T00:00:00.000Z
          source model id
          Gemini 2.5 Pro
          confidence lower
          0.9020876950782671
          confidence upper
          0.9529123049217328
          Original result ↗

          Kaggle Benchmarks

          0.928

          source id
          kaggleBenchmarks
          leaderboard name
          Global MMLU Lite English · English
          dimension
          reasoning
          metric
          Global MMLU Lite English · English
          rank
          16
          aggregation scope
          kaggle
          true
          status
          published
          license
          Apache-2.0
          direction
          higherIsBetter
          conditions
          task
          id
          41749
          name
          English
          version
          2
          sort_order
          DESCENDING
          display_type
          rawValues
          benchmark
          id
          153
          name
          Global MMLU Lite English
          type
          INDIVIDUAL
          owner
          cohere-labs
          version
          1
          categories
          • multilingual and cross-lingual capabilities
          • general knowledge and reasoning
          version_id
          135
          maintainer_tier
          Kaggle-maintained
          maintenance_level
          KAGGLE_MAINTAINED
          candidate
          candidate_class
          modelVersion
          record_kind
          aggregate
          attribution
          Kaggle · cohere-labs · Global MMLU Lite English
          license_url
          https://www.apache.org/licenses/LICENSE-2.0
          metric_kind
          Global MMLU Lite English · English
          rank_method
          competition rank among reported evaluations in identical conditions, including unlinked observations
          source_file
          135/leaderboard.json
          source_page
          https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-english/versions/1
          measurements
          • name
            English
            unit
            score
            scale
            1
            metric
            Global MMLU Lite English · English
            result
            resultCase
            numericResult
            booleanResult
            false
            numericResult
            value
            0.9275
            standardDeviation
            0
            confidenceInterval
            0.0254123049217328
            hasStandardDeviation
            false
            hasConfidenceInterval
            true
            confidenceIntervalNullable
            0.0254123049217328
            hasUnevenConfidenceInterval
            false
            taskVersionId
            0
            evaluationDate
            2025-10-30T00:00:00Z
            hasBooleanResult
            false
            hasNumericResult
            true
            hasTaskVersionId
            false
            hasEvaluationDate
            true
            numericResultNullable
            value
            0.9275
            standardDeviation
            0
            confidenceInterval
            0.0254123049217328
            hasStandardDeviation
            false
            hasConfidenceInterval
            true
            confidenceIntervalNullable
            0.0254123049217328
            hasUnevenConfidenceInterval
            false
            hasNumericResultPublic
            false
            customAdditionalResults
              hasNumericResultPrivate
              false
              task_id
              41749
              task_url
              /benchmarks/tasks/benchmarkler/global-mmlu-lite-english
              sort_order
              DESCENDING
              source_name
              English
              display_type
              rawValues
              task_version
              2
              task_display_type
              RAW_VALUES
              normalization_status
              normalized
            published_at
            2025-10-17T15:30:49.982103800Z
            candidate_class
            modelVersion
            source_revision
            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
            publisher_anchor
            google/gemini-2.5-pro
            metric_population
            41
            benchmark_evaluation
            true
            task_metadata_source
            benchmarkVersion
            candidate_metadata_source
            benchmarkVersionMappings
            evaluated at
            2025-10-30T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            0.9020876950782671
            confidence upper
            0.9529123049217328
            Original result ↗

            LMArena

            1,471.047

            source id
            lmarena
            leaderboard name
            LMArena Text Factuality Spanish
            dimension
            factuality
            metric
            Arena rating
            rank
            18
            sample size
            1279
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1454.4626937817754
            confidence upper
            1487.631143136493
            Original result ↗

            LMArena

            1,455.495

            source id
            lmarena
            leaderboard name
            LMArena Text Factuality Polish
            dimension
            factuality
            metric
            Arena rating
            rank
            18
            sample size
            1123
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1437.8397014041427
            confidence upper
            1473.1501457586287
            Original result ↗

            LMArena

            1,415.175

            source id
            lmarena
            leaderboard name
            LMArena Text Factuality Korean
            dimension
            factuality
            metric
            Arena rating
            rank
            20
            sample size
            552
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1391.6190581293686
            confidence upper
            1438.7309466352826
            Original result ↗

            LMArena

            1,486.765

            source id
            lmarena
            leaderboard name
            LMArena Text German
            dimension
            general
            metric
            Arena rating
            rank
            21
            sample size
            2646
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1474.2396768591714
            confidence upper
            1499.290357842165
            Original result ↗

            LMArena

            1,473.747

            source id
            lmarena
            leaderboard name
            LMArena Text Spanish
            dimension
            general
            metric
            Arena rating
            rank
            21
            sample size
            3431
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1462.7376056424157
            confidence upper
            1484.7571369245982
            Original result ↗

            LMArena

            1,501.137

            source id
            lmarena
            leaderboard name
            LMArena Text Factuality Industry Medicine And Healthcare
            dimension
            factuality
            metric
            Arena rating
            rank
            23
            sample size
            2609
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1490.4214351672229
            confidence upper
            1511.8519327381173
            Original result ↗

            OpenRouter Benchmarks

            82.435%

            source id
            openRouterBenchmarks
            leaderboard name
            GPQA Diamond · Accuracy · Google
            dimension
            knowledge
            metric
            GPQA Diamond · Accuracy
            rank
            23
            sample size
            1188
            aggregation scope
            status
            published
            harness
            url
            https://github.com/OpenRouterTeam/benchmark-harness
            name
            OpenRouter benchmark harness
            direction
            higherIsBetter
            run_count
            6
            conditions
            window
            newest 90 days; successful visible nonzero runs
            dataset
            benchmark-leaderboard
            benchmark
            id
            gpqa_diamond
            name
            GPQA Diamond
            record_kind
            aggregate
            provider_name
            Google
            attribution
            Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
            last_run_at
            2026-10-01T19:17:25.275Z
            license_url
            https://creativecommons.org/licenses/by/4.0/
            methodology
            Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
            metric_kind
            GPQA Diamond · Accuracy
            rank_method
            competition rank among reported evaluations in identical conditions, including unlinked observations
            sample_size
            1188
            source_file
            benchmark-leaderboard.json
            source_page
            https://openrouter.ai/benchmarks/gpqa-diamond
            measurements
            accuracy
            0.8243546666666667
            accuracy_std_dev
            0.03111308863734931
            cost_per_task_usd
            0.25956171212121215
            output_tokens_per_task
            25863.887205387204
            latency_seconds_per_task
            233.6810909090909
            published_at
            2026-11-16T01:46:02.653Z
            snapshot_date
            2026-11-16
            source_revision
            7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
            transformations
            TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
            metric_population
            40
            benchmark_evaluation
            true
            openrouter_permaslug
            google/gemini-2.5-pro
            source model id
            google/gemini-2.5-pro
            Original result ↗

            LMArena

            1,478.912

            source id
            lmarena
            leaderboard name
            LMArena Text Polish
            dimension
            general
            metric
            Arena rating
            rank
            26
            sample size
            6489
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1470.4848137125482
            confidence upper
            1487.3401073737236
            Original result ↗

            LMArena

            1,460.887

            source id
            lmarena
            leaderboard name
            LMArena Text Japanese
            dimension
            general
            metric
            Arena rating
            rank
            29
            sample size
            2059
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1446.2306539856543
            confidence upper
            1475.5425934003104
            Original result ↗

            LMArena

            1,454.046

            source id
            lmarena
            leaderboard name
            LMArena Text Creative Writing
            dimension
            general
            metric
            Arena rating
            rank
            30
            sample size
            17581
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1448.9692204098683
            confidence upper
            1459.1220434268325
            Original result ↗

            OpenRouter Benchmarks

            60%

            source id
            openRouterBenchmarks
            leaderboard name
            tau2-bench Verified Airline · Accuracy · Google
            dimension
            agentic
            metric
            tau2-bench Verified Airline · Accuracy
            rank
            30
            sample size
            300
            aggregation scope
            status
            published
            harness
            url
            https://github.com/OpenRouterTeam/benchmark-harness
            name
            OpenRouter benchmark harness
            direction
            higherIsBetter
            run_count
            6
            conditions
            window
            newest 90 days; successful visible nonzero runs
            dataset
            benchmark-leaderboard
            benchmark
            id
            tau_bench_verified_airline
            name
            tau2-bench Verified Airline
            record_kind
            aggregate
            provider_name
            Google
            attribution
            Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
            last_run_at
            2026-10-01T19:04:55.725Z
            license_url
            https://creativecommons.org/licenses/by/4.0/
            methodology
            Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
            metric_kind
            tau2-bench Verified Airline · Accuracy
            rank_method
            competition rank among reported evaluations in identical conditions, including unlinked observations
            sample_size
            300
            source_file
            benchmark-leaderboard.json
            source_page
            https://openrouter.ai/benchmarks/tau2-bench-airline
            measurements
            accuracy
            0.6000001666666667
            accuracy_std_dev
            0.03771239112642186
            cost_per_task_usd
            0.19008634000000002
            output_tokens_per_task
            11452.75
            latency_seconds_per_task
            186.97522
            published_at
            2026-11-16T01:46:02.653Z
            snapshot_date
            2026-11-16
            source_revision
            7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
            transformations
            TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
            metric_population
            36
            benchmark_evaluation
            true
            openrouter_permaslug
            google/gemini-2.5-pro
            source model id
            google/gemini-2.5-pro
            Original result ↗

            LMArena

            1,488.348

            source id
            lmarena
            leaderboard name
            LMArena Text Factuality French
            dimension
            factuality
            metric
            Arena rating
            rank
            31
            sample size
            1018
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1469.8438505555614
            confidence upper
            1506.851304834845
            Original result ↗

            LMArena

            1,476.08

            source id
            lmarena
            leaderboard name
            LMArena Text Industry Medicine And Healthcare
            dimension
            general
            metric
            Arena rating
            rank
            31
            sample size
            7940
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1468.930230943948
            confidence upper
            1483.228997452458
            Original result ↗

            LMArena

            1,480.504

            source id
            lmarena
            leaderboard name
            LMArena Text Industry Legal And Government
            dimension
            general
            metric
            Arena rating
            rank
            32
            sample size
            8892
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1473.7883528981872
            confidence upper
            1487.2197803416043
            Original result ↗

            LMArena

            1,490.935

            source id
            lmarena
            leaderboard name
            LMArena Text Factuality Industry Life And Physical And Social Science
            dimension
            factuality
            metric
            Arena rating
            rank
            33
            sample size
            7127
            evaluated at
            2026-11-09T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            1483.5581126778256
            confidence upper
            1498.3108893259068
            Original result ↗

            Published prices

            1–20 / 60

            google/gemini-2.5-pro ↗

            $0.13

            Inference provider
            cloudflareWorkersAI
            Platform
            Google Vertex AI
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions
            Billing label
            Cached input tokens (per 1M)
            Pricing via
            Cloudflare AI Gateway

            google/gemini-2.5-pro ↗

            $1.25

            Inference provider
            cloudflareWorkersAI
            Platform
            Google Vertex AI
            Unit
            input_tokens_per_million
            Currency
            USD
            Billing conditions
            Billing label
            Input tokens (per 1M)
            Pricing via
            Cloudflare AI Gateway

            google/gemini-2.5-pro ↗

            $10.00

            Inference provider
            cloudflareWorkersAI
            Platform
            Google Vertex AI
            Unit
            output_tokens_per_million
            Currency
            USD
            Billing conditions
            Billing label
            Output tokens (per 1M)
            Pricing via
            Cloudflare AI Gateway

            google/gemini-2.5-pro ↗

            $1.25

            Inference provider
            deepInfra
            Platform
            DeepInfra
            Unit
            input_tokens_per_million
            Currency
            USD
            Billing conditions

            google/gemini-2.5-pro ↗

            $10.00

            Inference provider
            deepInfra
            Platform
            DeepInfra
            Unit
            output_tokens_per_million
            Currency
            USD
            Billing conditions

            google/gemini-2.5-pro ↗

            $0.13

            Inference provider
            openRouter
            Platform
            OpenRouter
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions

            google/gemini-2.5-pro ↗

            $0.38

            Inference provider
            openRouter
            Platform
            OpenRouter
            Unit
            cache_write_tokens_per_million
            Currency
            USD
            Billing conditions

            google/gemini-2.5-pro ↗

            $1.25

            Inference provider
            openRouter
            Platform
            OpenRouter
            Unit
            input_tokens_per_million
            Currency
            USD
            Billing conditions

            google/gemini-2.5-pro ↗

            $10.00

            Inference provider
            openRouter
            Platform
            OpenRouter
            Unit
            output_tokens_per_million
            Currency
            USD
            Billing conditions

            google/gemini-2.5-pro ↗

            $0.25

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions
            model_type
            language
            quote_scope
            gateway_catalog
            billing_scope
            Vercel public catalog; endpoint prices may differ
            context_tokens
            min_inclusive
            200001
            Billing label
            Cached input / 1M
            Pricing via
            Vercel AI Gateway

            google/gemini-2.5-pro ↗

            $0.13

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions
            model_type
            language
            quote_scope
            gateway_catalog
            billing_scope
            Vercel public catalog; endpoint prices may differ
            context_tokens
            max_exclusive
            200001
            min_inclusive
            0
            Billing label
            Cached input / 1M
            Pricing via
            Vercel AI Gateway

            google/gemini-2.5-pro ↗

            $0.22

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions
            model_type
            language
            quote_scope
            gateway_catalog
            service_tier
            priority
            billing_scope
            Vercel public catalog; endpoint prices may differ
            Billing label
            Cached input / 1M
            Pricing via
            Vercel AI Gateway

            google/gemini-2.5-pro ↗

            $0.25

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions
            region
            eu
            model_type
            language
            quote_scope
            gateway_catalog
            billing_scope
            Vercel public catalog; endpoint prices may differ
            context_tokens
            min_inclusive
            200001
            Billing label
            Cached input / 1M
            Pricing via
            Vercel AI Gateway

            google/gemini-2.5-pro ↗

            $0.13

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions
            region
            eu
            model_type
            language
            quote_scope
            gateway_catalog
            billing_scope
            Vercel public catalog; endpoint prices may differ
            context_tokens
            max_exclusive
            200001
            min_inclusive
            0
            Billing label
            Cached input / 1M
            Pricing via
            Vercel AI Gateway

            google/gemini-2.5-pro ↗

            $0.25

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions
            region
            us
            model_type
            language
            quote_scope
            gateway_catalog
            billing_scope
            Vercel public catalog; endpoint prices may differ
            context_tokens
            min_inclusive
            200001
            Billing label
            Cached input / 1M
            Pricing via
            Vercel AI Gateway

            google/gemini-2.5-pro ↗

            $0.13

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            cache_read_tokens_per_million
            Currency
            USD
            Billing conditions
            region
            us
            model_type
            language
            quote_scope
            gateway_catalog
            billing_scope
            Vercel public catalog; endpoint prices may differ
            context_tokens
            max_exclusive
            200001
            min_inclusive
            0
            Billing label
            Cached input / 1M
            Pricing via
            Vercel AI Gateway

            google/gemini-2.5-pro ↗

            $2.50

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            input_tokens_per_million
            Currency
            USD
            Billing conditions
            model_type
            language
            quote_scope
            gateway_catalog
            billing_scope
            Vercel public catalog; endpoint prices may differ
            context_tokens
            min_inclusive
            200001
            Billing label
            Input tokens / 1M
            Pricing via
            Vercel AI Gateway

            google/gemini-2.5-pro ↗

            $1.25

            Inference provider
            vercelAIGatewayCatalog
            Platform
            Vercel AI Gateway
            Unit
            input_tokens_per_million
            Currency
            USD
            Billing conditions
            model_type
            language
            quote_scope
            gateway_catalog
            billing_scope
            Vercel public catalog; endpoint prices may differ
            context_tokens
            max_exclusive
            200001
            min_inclusive
            0
            Billing label
            Input tokens / 1M
            Pricing via
            Vercel AI Gateway

            Provider specifications & access

            1–4 / 4

            gemini-2.5-pro ↗

            source model id
            gemini-2.5-pro
            hosting
            gateway
            platforms
            • Cheaper Inference
            context length
            1048576
            max output tokens
            65535
            architecture
            input_modalities
            • text
            • image
            output_modalities
            • text
            capabilities
            • reasoning
            • structuredOutputs
            • toolCalling
            • visionInput
            • webSearch

            Gemini 2.5 Pro ↗

            source model id
            google/gemini-2.5-pro
            hosting
            third-party
            platforms
            • Google Vertex AI
            context length
            1000000
            max output tokens
            65536
            architecture
            input_modalities
            • text
            output_modalities
            • text
            capabilities
            • reasoning

            Gemini 2.5 Pro ↗

            source model id
            google/gemini-2.5-pro
            hosting
            vercel-gateway
            platforms
            • Vercel AI Gateway
            context length
            1048576
            max output tokens
            65535
            architecture
            input_modalities
            • text
            • image
            • pdf
            output_modalities
            • text
            capabilities
            • promptCaching
            • reasoning
            • structuredOutputs
            • toolCalling
            • visionInput
            • webSearch
            regions
            • eu
            • us

            Gemini 2.5 Pro ↗

            source model id
            publishers/google/models/gemini-2.5-pro@default
            hosting
            vertex-managed
            platforms
            • Google Cloud Vertex AI
            context length
            1048576
            max output tokens
            65536
            architecture
            input_modalities
            • audio
            • image
            • text
            • video
            output_modalities
            • text
            capabilities
            • batch
            • multimodal
            • promptCaching
            • reasoning
            • structuredOutputs
            • toolCalling
            • visionInput

            Safety assessments

            1–1 / 1

            Foundation Model Transparency Index 2025 ↗

            41 / 100

            source id
            stanfordFMTI
            canonical model id
            google/gemini-2.5-pro
            taxonomy
            transparency
            hazard category
            transparency
            hazard name
            Organization transparency
            policy area
            Google
            assessment type
            organization-level disclosure
            metric
            disclosure_score
            result label
            41 / 100
            sample size
            100
            updated at
            2026-11-09T18:08:05.227Z

            Source specification observations

            1–10 / 10

            cloudflareWorkersAI ↗

            field
            context_length
            value
            1000000
            normalized value
            1000000
            scope
            source-model
            source id
            cloudflareWorkersAI
            confidence
            1

            openRouter ↗

            field
            context_length
            value
            1048576
            normalized value
            1048576
            scope
            source-model
            source id
            openRouter
            confidence
            1

            vercelAIGatewayCatalog ↗

            field
            context_length
            value
            1048576
            normalized value
            1048576
            scope
            source-model
            source id
            vercelAIGatewayCatalog
            confidence
            1

            vertexAI ↗

            field
            context_length
            value
            1048576
            normalized value
            1048576
            scope
            source-model
            source id
            vertexAI
            confidence
            1

            openRouter ↗

            field
            knowledge_cutoff
            reported field
            knowledge_cutoff
            value
            2025-01-31
            normalized value
            2025-01-31
            precision
            day
            scope
            source-model
            source id
            openRouter
            confidence
            1

            vercelAIGatewayCatalog ↗

            field
            knowledge_cutoff
            reported field
            knowledge_cutoff
            value
            2025-01
            normalized value
            2025-01
            precision
            month
            scope
            source-model
            source id
            vercelAIGatewayCatalog
            confidence
            1

            cloudflareWorkersAI ↗

            field
            max_output_tokens
            value
            65536
            normalized value
            65536
            scope
            source-model
            source id
            cloudflareWorkersAI
            confidence
            1

            openRouter ↗

            field
            max_output_tokens
            value
            65536
            normalized value
            65536
            scope
            source-model
            source id
            openRouter
            confidence
            1

            vercelAIGatewayCatalog ↗

            field
            max_output_tokens
            value
            65535
            normalized value
            65535
            scope
            source-model
            source id
            vercelAIGatewayCatalog
            confidence
            1

            vertexAI ↗

            field
            max_output_tokens
            value
            65536
            normalized value
            65536
            scope
            source-model
            source id
            vertexAI
            confidence
            1

            Original sources

            1–20 / 20

            Google · Model reference ↗

            DeepInfra · Model page ↗

            Google Cloud · Model page ↗

            OpenRouter · Model page ↗

            Vercel AI Gateway · Model page ↗

            Google Cloud · Documentation ↗

            GitHub · Source snapshot ↗

            api.cheaperinference.com · Source snapshot ↗

            Cloudflare AI · Catalog ↗

            DeepInfra · Catalog ↗

            Epoch AI · Catalog ↗

            GitHub · Catalog ↗

            GitHub · Catalog ↗

            Google Cloud · Catalog ↗

            Kaggle · Catalog ↗

            LMArena · Catalog ↗

            OpenRouter · Catalog ↗

            OpenRouter · Catalog ↗

            Vercel AI Gateway · Catalog ↗

            www.swebench.com · Catalog ↗

            Source links

            1–14 / 14

            Explore more models

            Models

            Loading
            EVIDENCE / COMPARISON

            Benchmarks

            Explore every benchmark

            Loading

            These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

            Install TensorTrace