Google: Gemini 2.5 Pro

Gemini 2.5 Pro is Google’s state-of-the-art AI model designed for advanced reasoning, coding, mathematics, and scientific tasks. It employs “thinking” capabilities, enabling it to reason through resp…

google/gemini-2.5-pro

Gemini 2.5 Pro is Google’s state-of-the-art AI model designed for advanced reasoning, coding, mathematics, and scientific tasks. It employs “thinking” capabilities, enabling it to reason through responses with enhanced accuracy and nuanced context handling. Gemini 2.5 Pro achieves top-tier performance on multiple benchmarks, including first-place positioning on the LMArena leaderboard, reflecting superior human-preference alignment and complex problem-solving abilities.

Specifications & capabilities

Input
text, image, file, audio, video
Output
text
Context window · tokens
1048576
Knowledge cutoff
2025-01
Published model ID
gemini-2.5-pro@default
Capabilities
audioInput, batch, fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput, webSearch
Description source ↗

Evaluations

41–60 / 1,319

Kaggle Benchmarks

68.843%

source id
kaggleBenchmarks
leaderboard name
AssetOpsBench · Elimination Accuracy
dimension
reasoning
metric
AssetOpsBench · Elimination Accuracy
rank
8
aggregation scope
kaggle
true
status
published
license
Apache-2.0
direction
higherIsBetter
conditions
task
id
1041
name
Elimination Accuracy
version
1
sort_order
DESCENDING
display_type
percentages
benchmark
id
243
name
AssetOpsBench
type
INDIVIDUAL
owner
ibm-research
version
1
categories
  • evaluation
  • general knowledge and reasoning
version_id
222
maintainer_tier
Kaggle-maintained
maintenance_level
KAGGLE_MAINTAINED
candidate
candidate_class
modelVersion
record_kind
aggregate
attribution
Kaggle · ibm-research · AssetOpsBench
license_url
https://www.apache.org/licenses/LICENSE-2.0
metric_kind
AssetOpsBench · Elimination Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
222/leaderboard.json
source_page
https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
measurements
  • name
    Elimination Accuracy
    unit
    percent
    scale
    1
    metric
    AssetOpsBench · Elimination Accuracy
    result
    resultCase
    numericResult
    booleanResult
    false
    numericResult
    value
    0.6884335503638452
    standardDeviation
    0
    confidenceInterval
    0
    hasStandardDeviation
    false
    hasConfidenceInterval
    false
    hasUnevenConfidenceInterval
    false
    taskVersionId
    0
    evaluationDate
    2025-11-22T14:59:16.384029Z
    hasBooleanResult
    false
    hasNumericResult
    true
    hasTaskVersionId
    false
    hasEvaluationDate
    true
    numericResultNullable
    value
    0.6884335503638452
    standardDeviation
    0
    confidenceInterval
    0
    hasStandardDeviation
    false
    hasConfidenceInterval
    false
    hasUnevenConfidenceInterval
    false
    hasNumericResultPublic
    false
    customAdditionalResults
      hasNumericResultPrivate
      false
      task_id
      1041
      task_url
      /benchmarks/tasks/benchmarkler/failuresensoriq-eliminationaccuracy-benchmark-task
      sort_order
      DESCENDING
      source_name
      Elimination Accuracy
      display_type
      percentages
      task_version
      1
      task_display_type
      RAW_VALUES
      normalization_status
      normalized
    published_at
    2025-12-01T22:15:26.261663300Z
    candidate_class
    modelVersion
    source_revision
    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
    publisher_anchor
    google/gemini-2.5-pro
    metric_population
    25
    benchmark_evaluation
    true
    task_metadata_source
    benchmarkVersion
    candidate_metadata_source
    benchmarkVersionMappings
    evaluated at
    2025-11-22T14:59:16.384Z
    source model id
    Gemini 2.5 Pro
    Original result ↗

    Kaggle Benchmarks

    66.97%

    source id
    kaggleBenchmarks
    leaderboard name
    AssetOpsBench · F-Score
    dimension
    reasoning
    metric
    AssetOpsBench · F-Score
    rank
    8
    aggregation scope
    kaggle
    true
    status
    published
    license
    Apache-2.0
    direction
    higherIsBetter
    conditions
    task
    id
    1040
    name
    F-Score
    version
    1
    sort_order
    DESCENDING
    display_type
    percentages
    benchmark
    id
    243
    name
    AssetOpsBench
    type
    INDIVIDUAL
    owner
    ibm-research
    version
    1
    categories
    • evaluation
    • general knowledge and reasoning
    version_id
    222
    maintainer_tier
    Kaggle-maintained
    maintenance_level
    KAGGLE_MAINTAINED
    candidate
    candidate_class
    modelVersion
    record_kind
    aggregate
    attribution
    Kaggle · ibm-research · AssetOpsBench
    license_url
    https://www.apache.org/licenses/LICENSE-2.0
    metric_kind
    AssetOpsBench · F-Score
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    222/leaderboard.json
    source_page
    https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
    measurements
    • name
      F-Score
      unit
      percent
      scale
      1
      metric
      AssetOpsBench · F-Score
      result
      resultCase
      numericResult
      booleanResult
      false
      numericResult
      value
      0.6697034163433864
      standardDeviation
      0
      confidenceInterval
      0
      hasStandardDeviation
      false
      hasConfidenceInterval
      false
      hasUnevenConfidenceInterval
      false
      taskVersionId
      0
      evaluationDate
      2025-11-22T14:27:51.636277Z
      hasBooleanResult
      false
      hasNumericResult
      true
      hasTaskVersionId
      false
      hasEvaluationDate
      true
      numericResultNullable
      value
      0.6697034163433864
      standardDeviation
      0
      confidenceInterval
      0
      hasStandardDeviation
      false
      hasConfidenceInterval
      false
      hasUnevenConfidenceInterval
      false
      hasNumericResultPublic
      false
      customAdditionalResults
        hasNumericResultPrivate
        false
        task_id
        1040
        task_url
        /benchmarks/tasks/benchmarkler/failuresensoriq-fscore-benchmark-task
        sort_order
        DESCENDING
        source_name
        F-Score
        display_type
        percentages
        task_version
        1
        task_display_type
        RAW_VALUES
        normalization_status
        normalized
      published_at
      2025-12-01T22:15:26.261663300Z
      candidate_class
      modelVersion
      source_revision
      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
      publisher_anchor
      google/gemini-2.5-pro
      metric_population
      25
      benchmark_evaluation
      true
      task_metadata_source
      benchmarkVersion
      candidate_metadata_source
      benchmarkVersionMappings
      evaluated at
      2025-11-22T14:27:51.636Z
      source model id
      Gemini 2.5 Pro
      Original result ↗

      Kaggle Benchmarks

      62.622%

      source id
      kaggleBenchmarks
      leaderboard name
      AssetOpsBench · Selection Accuracy
      dimension
      reasoning
      metric
      AssetOpsBench · Selection Accuracy
      rank
      8
      aggregation scope
      kaggle
      true
      status
      published
      license
      Apache-2.0
      direction
      higherIsBetter
      conditions
      task
      id
      1042
      name
      Selection Accuracy
      version
      1
      sort_order
      DESCENDING
      display_type
      percentages
      benchmark
      id
      243
      name
      AssetOpsBench
      type
      INDIVIDUAL
      owner
      ibm-research
      version
      1
      categories
      • evaluation
      • general knowledge and reasoning
      version_id
      222
      maintainer_tier
      Kaggle-maintained
      maintenance_level
      KAGGLE_MAINTAINED
      candidate
      candidate_class
      modelVersion
      record_kind
      aggregate
      attribution
      Kaggle · ibm-research · AssetOpsBench
      license_url
      https://www.apache.org/licenses/LICENSE-2.0
      metric_kind
      AssetOpsBench · Selection Accuracy
      rank_method
      competition rank among reported evaluations in identical conditions, including unlinked observations
      source_file
      222/leaderboard.json
      source_page
      https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
      measurements
      • name
        Selection Accuracy
        unit
        percent
        scale
        1
        metric
        AssetOpsBench · Selection Accuracy
        result
        resultCase
        numericResult
        booleanResult
        false
        numericResult
        value
        0.6262188515709642
        standardDeviation
        0
        confidenceInterval
        0
        hasStandardDeviation
        false
        hasConfidenceInterval
        false
        hasUnevenConfidenceInterval
        false
        taskVersionId
        0
        evaluationDate
        2025-11-22T14:59:16.297106Z
        hasBooleanResult
        false
        hasNumericResult
        true
        hasTaskVersionId
        false
        hasEvaluationDate
        true
        numericResultNullable
        value
        0.6262188515709642
        standardDeviation
        0
        confidenceInterval
        0
        hasStandardDeviation
        false
        hasConfidenceInterval
        false
        hasUnevenConfidenceInterval
        false
        hasNumericResultPublic
        false
        customAdditionalResults
          hasNumericResultPrivate
          false
          task_id
          1042
          task_url
          /benchmarks/tasks/benchmarkler/failuresensoriq-selectionaccuracy-benchmark-task
          sort_order
          DESCENDING
          source_name
          Selection Accuracy
          display_type
          percentages
          task_version
          1
          task_display_type
          RAW_VALUES
          normalization_status
          normalized
        published_at
        2025-12-01T22:15:26.261663300Z
        candidate_class
        modelVersion
        source_revision
        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
        publisher_anchor
        google/gemini-2.5-pro
        metric_population
        25
        benchmark_evaluation
        true
        task_metadata_source
        benchmarkVersion
        candidate_metadata_source
        benchmarkVersionMappings
        evaluated at
        2025-11-22T14:59:16.297Z
        source model id
        Gemini 2.5 Pro
        Original result ↗

        OpenRouter Benchmarks

        55%

        source id
        openRouterBenchmarks
        leaderboard name
        tau2-bench Verified Airline · Accuracy · Google AI Studio
        dimension
        agentic
        metric
        tau2-bench Verified Airline · Accuracy
        rank
        8
        sample size
        100
        aggregation scope
        status
        published
        harness
        url
        https://github.com/OpenRouterTeam/benchmark-harness
        name
        OpenRouter benchmark harness
        direction
        higherIsBetter
        run_count
        2
        conditions
        window
        newest 90 days; successful visible nonzero runs
        dataset
        benchmark-leaderboard
        benchmark
        id
        tau_bench_verified_airline
        name
        tau2-bench Verified Airline
        record_kind
        aggregate
        provider_name
        Google AI Studio
        attribution
        Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
        last_run_at
        2026-10-01T19:04:52.649Z
        license_url
        https://creativecommons.org/licenses/by/4.0/
        methodology
        Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
        metric_kind
        tau2-bench Verified Airline · Accuracy
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        sample_size
        100
        source_file
        benchmark-leaderboard.json
        source_page
        https://openrouter.ai/benchmarks/tau2-bench-airline
        measurements
        accuracy
        0.55
        accuracy_std_dev
        0.036667000000000005
        cost_per_task_usd
        0.2373302
        output_tokens_per_task
        12508.34
        latency_seconds_per_task
        164.4292
        published_at
        2026-11-16T01:46:02.653Z
        snapshot_date
        2026-11-16
        source_revision
        7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
        transformations
        TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
        metric_population
        10
        benchmark_evaluation
        true
        openrouter_permaslug
        google/gemini-2.5-pro
        source model id
        google/gemini-2.5-pro
        Original result ↗

        Kaggle Benchmarks

        31.25%

        source id
        kaggleBenchmarks
        leaderboard name
        ITBench · SRE Score
        dimension
        reasoning
        metric
        ITBench · SRE Score
        rank
        8
        aggregation scope
        kaggle
        true
        status
        published
        license
        Apache-2.0
        direction
        higherIsBetter
        conditions
        task
        id
        1172
        name
        SRE Score
        version
        8
        sort_order
        DESCENDING
        display_type
        rawPercentages
        benchmark
        id
        246
        name
        ITBench
        type
        INDIVIDUAL
        owner
        ibm-research
        version
        1
        categories
        • evaluation
        • general knowledge and reasoning
        version_id
        225
        maintainer_tier
        Kaggle-maintained
        maintenance_level
        KAGGLE_MAINTAINED
        candidate
        candidate_class
        modelVersion
        record_kind
        aggregate
        attribution
        Kaggle · ibm-research · ITBench
        license_url
        https://www.apache.org/licenses/LICENSE-2.0
        metric_kind
        ITBench · SRE Score
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        source_file
        225/leaderboard.json
        source_page
        https://www.kaggle.com/benchmarks/ibm-research/itbench/versions/1
        measurements
        • name
          SRE Score
          unit
          percent
          scale
          100
          metric
          ITBench · SRE Score
          result
          resultCase
          numericResult
          booleanResult
          false
          numericResult
          value
          31.25
          standardDeviation
          0
          confidenceInterval
          0
          hasStandardDeviation
          false
          hasConfidenceInterval
          false
          hasUnevenConfidenceInterval
          false
          taskVersionId
          0
          evaluationDate
          2025-12-01T22:27:17.354159Z
          hasBooleanResult
          false
          hasNumericResult
          true
          hasTaskVersionId
          false
          hasEvaluationDate
          true
          numericResultNullable
          value
          31.25
          standardDeviation
          0
          confidenceInterval
          0
          hasStandardDeviation
          false
          hasConfidenceInterval
          false
          hasUnevenConfidenceInterval
          false
          hasNumericResultPublic
          false
          customAdditionalResults
            hasNumericResultPrivate
            false
            task_id
            1172
            task_url
            /benchmarks/tasks/benchmarkler/itbench-sre-main
            sort_order
            DESCENDING
            source_name
            SRE Score
            display_type
            rawPercentages
            task_version
            8
            task_display_type
            RAW_VALUES
            normalization_status
            normalized
          published_at
          2025-12-02T12:36:17.407125500Z
          candidate_class
          modelVersion
          source_revision
          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
          publisher_anchor
          google/gemini-2.5-pro
          metric_population
          13
          benchmark_evaluation
          true
          task_metadata_source
          benchmarkVersion
          candidate_metadata_source
          benchmarkVersionMappings
          evaluated at
          2025-12-01T22:27:17.354Z
          source model id
          Gemini 2.5 Pro
          Original result ↗

          Kaggle Benchmarks

          0.93

          source id
          kaggleBenchmarks
          leaderboard name
          Global MMLU Lite Burmese · Burmese
          dimension
          reasoning
          metric
          Global MMLU Lite Burmese · Burmese
          rank
          9
          aggregation scope
          kaggle
          true
          status
          published
          license
          Apache-2.0
          direction
          higherIsBetter
          conditions
          task
          id
          41736
          name
          Burmese
          version
          2
          sort_order
          DESCENDING
          display_type
          rawValues
          benchmark
          id
          175
          name
          Global MMLU Lite Burmese
          type
          INDIVIDUAL
          owner
          cohere-labs
          version
          1
          categories
          • multilingual and cross-lingual capabilities
          • general knowledge and reasoning
          version_id
          153
          maintainer_tier
          Kaggle-maintained
          maintenance_level
          KAGGLE_MAINTAINED
          candidate
          candidate_class
          modelVersion
          record_kind
          aggregate
          attribution
          Kaggle · cohere-labs · Global MMLU Lite Burmese
          license_url
          https://www.apache.org/licenses/LICENSE-2.0
          metric_kind
          Global MMLU Lite Burmese · Burmese
          rank_method
          competition rank among reported evaluations in identical conditions, including unlinked observations
          source_file
          153/leaderboard.json
          source_page
          https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-burmese/versions/1
          measurements
          • name
            Burmese
            unit
            score
            scale
            1
            metric
            Global MMLU Lite Burmese · Burmese
            result
            resultCase
            numericResult
            booleanResult
            false
            numericResult
            value
            0.93
            standardDeviation
            0
            confidenceInterval
            0.0250039481496016
            hasStandardDeviation
            false
            hasConfidenceInterval
            true
            confidenceIntervalNullable
            0.0250039481496016
            hasUnevenConfidenceInterval
            false
            taskVersionId
            0
            evaluationDate
            2025-10-30T00:00:00Z
            hasBooleanResult
            false
            hasNumericResult
            true
            hasTaskVersionId
            false
            hasEvaluationDate
            true
            numericResultNullable
            value
            0.93
            standardDeviation
            0
            confidenceInterval
            0.0250039481496016
            hasStandardDeviation
            false
            hasConfidenceInterval
            true
            confidenceIntervalNullable
            0.0250039481496016
            hasUnevenConfidenceInterval
            false
            hasNumericResultPublic
            false
            customAdditionalResults
              hasNumericResultPrivate
              false
              task_id
              41736
              task_url
              /benchmarks/tasks/benchmarkler/global-mmlu-lite-burmese
              sort_order
              DESCENDING
              source_name
              Burmese
              display_type
              rawValues
              task_version
              2
              task_display_type
              RAW_VALUES
              normalization_status
              normalized
            published_at
            2025-10-17T15:30:19.694332700Z
            candidate_class
            modelVersion
            source_revision
            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
            publisher_anchor
            google/gemini-2.5-pro
            metric_population
            41
            benchmark_evaluation
            true
            task_metadata_source
            benchmarkVersion
            candidate_metadata_source
            benchmarkVersionMappings
            evaluated at
            2025-10-30T00:00:00.000Z
            source model id
            Gemini 2.5 Pro
            confidence lower
            0.9049960518503984
            confidence upper
            0.9550039481496017
            Original result ↗

            Kaggle Benchmarks

            63.217%

            source id
            kaggleBenchmarks
            leaderboard name
            AssetOpsBench · Pert. Complex Accuracy
            dimension
            reasoning
            metric
            AssetOpsBench · Pert. Complex Accuracy
            rank
            9
            aggregation scope
            kaggle
            true
            status
            published
            license
            Apache-2.0
            direction
            higherIsBetter
            conditions
            task
            id
            1044
            name
            Pert. Complex Accuracy
            version
            1
            sort_order
            DESCENDING
            display_type
            percentages
            benchmark
            id
            243
            name
            AssetOpsBench
            type
            INDIVIDUAL
            owner
            ibm-research
            version
            1
            categories
            • evaluation
            • general knowledge and reasoning
            version_id
            222
            maintainer_tier
            Kaggle-maintained
            maintenance_level
            KAGGLE_MAINTAINED
            candidate
            candidate_class
            modelVersion
            record_kind
            aggregate
            attribution
            Kaggle · ibm-research · AssetOpsBench
            license_url
            https://www.apache.org/licenses/LICENSE-2.0
            metric_kind
            AssetOpsBench · Pert. Complex Accuracy
            rank_method
            competition rank among reported evaluations in identical conditions, including unlinked observations
            source_file
            222/leaderboard.json
            source_page
            https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
            measurements
            • name
              Pert. Complex Accuracy
              unit
              percent
              scale
              1
              metric
              AssetOpsBench · Pert. Complex Accuracy
              result
              resultCase
              numericResult
              booleanResult
              false
              numericResult
              value
              0.6321709786276716
              standardDeviation
              0
              confidenceInterval
              0
              hasStandardDeviation
              false
              hasConfidenceInterval
              false
              hasUnevenConfidenceInterval
              false
              taskVersionId
              0
              evaluationDate
              2025-11-24T03:34:20.061647Z
              hasBooleanResult
              false
              hasNumericResult
              true
              hasTaskVersionId
              false
              hasEvaluationDate
              true
              numericResultNullable
              value
              0.6321709786276716
              standardDeviation
              0
              confidenceInterval
              0
              hasStandardDeviation
              false
              hasConfidenceInterval
              false
              hasUnevenConfidenceInterval
              false
              hasNumericResultPublic
              false
              customAdditionalResults
                hasNumericResultPrivate
                false
                task_id
                1044
                task_url
                /benchmarks/tasks/benchmarkler/failuresensoriq-perturbationcomplex-benchmark-task
                sort_order
                DESCENDING
                source_name
                Pert. Complex Accuracy
                display_type
                percentages
                task_version
                1
                task_display_type
                RAW_VALUES
                normalization_status
                normalized
              published_at
              2025-12-01T22:15:26.261663300Z
              candidate_class
              modelVersion
              source_revision
              b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
              publisher_anchor
              google/gemini-2.5-pro
              metric_population
              25
              benchmark_evaluation
              true
              task_metadata_source
              benchmarkVersion
              candidate_metadata_source
              benchmarkVersionMappings
              evaluated at
              2025-11-24T03:34:20.061Z
              source model id
              Gemini 2.5 Pro
              Original result ↗

              Kaggle Benchmarks

              42.292%

              source id
              kaggleBenchmarks
              leaderboard name
              ITBench · Average Score
              dimension
              reasoning
              metric
              ITBench · Average Score
              rank
              9
              aggregation scope
              kaggle
              true
              status
              published
              license
              Apache-2.0
              direction
              higherIsBetter
              conditions
              task
              id
              1064
              name
              Average Score
              version
              1
              sort_order
              DESCENDING
              display_type
              rawPercentages
              benchmark
              id
              246
              name
              ITBench
              type
              INDIVIDUAL
              owner
              ibm-research
              version
              1
              categories
              • evaluation
              • general knowledge and reasoning
              version_id
              225
              maintainer_tier
              Kaggle-maintained
              maintenance_level
              KAGGLE_MAINTAINED
              candidate
              candidate_class
              modelVersion
              record_kind
              aggregate
              attribution
              Kaggle · ibm-research · ITBench
              license_url
              https://www.apache.org/licenses/LICENSE-2.0
              metric_kind
              ITBench · Average Score
              rank_method
              competition rank among reported evaluations in identical conditions, including unlinked observations
              source_file
              225/leaderboard.json
              source_page
              https://www.kaggle.com/benchmarks/ibm-research/itbench/versions/1
              measurements
              • name
                Average Score
                unit
                percent
                scale
                100
                metric
                ITBench · Average Score
                result
                resultCase
                numericResult
                booleanResult
                false
                numericResult
                value
                42.29166666666667
                standardDeviation
                0
                confidenceInterval
                0
                hasStandardDeviation
                false
                hasConfidenceInterval
                false
                hasUnevenConfidenceInterval
                false
                taskVersionId
                0
                evaluationDate
                2025-12-01T22:27:17.354159Z
                hasBooleanResult
                false
                hasNumericResult
                true
                hasTaskVersionId
                false
                hasEvaluationDate
                true
                numericResultNullable
                value
                42.29166666666667
                standardDeviation
                0
                confidenceInterval
                0
                hasStandardDeviation
                false
                hasConfidenceInterval
                false
                hasUnevenConfidenceInterval
                false
                hasNumericResultPublic
                false
                customAdditionalResults
                  hasNumericResultPrivate
                  false
                  task_id
                  1064
                  sort_order
                  DESCENDING
                  source_name
                  Average Score
                  display_type
                  rawPercentages
                  task_version
                  1
                  task_display_type
                  RAW_PERCENTAGES
                  normalization_status
                  normalized
                published_at
                2025-12-02T12:36:17.407125500Z
                candidate_class
                modelVersion
                source_revision
                b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                publisher_anchor
                google/gemini-2.5-pro
                metric_population
                13
                benchmark_evaluation
                true
                task_metadata_source
                benchmarkVersion
                candidate_metadata_source
                benchmarkVersionMappings
                evaluated at
                2025-12-01T22:27:17.354Z
                source model id
                Gemini 2.5 Pro
                Original result ↗

                Kaggle Benchmarks

                0.943

                source id
                kaggleBenchmarks
                leaderboard name
                Global MMLU Lite French · French
                dimension
                reasoning
                metric
                Global MMLU Lite French · French
                rank
                10
                aggregation scope
                kaggle
                true
                status
                published
                license
                Apache-2.0
                direction
                higherIsBetter
                conditions
                task
                id
                41752
                name
                French
                version
                2
                sort_order
                DESCENDING
                display_type
                rawValues
                benchmark
                id
                156
                name
                Global MMLU Lite French
                type
                INDIVIDUAL
                owner
                cohere-labs
                version
                1
                categories
                • multilingual and cross-lingual capabilities
                • general knowledge and reasoning
                version_id
                138
                maintainer_tier
                Kaggle-maintained
                maintenance_level
                KAGGLE_MAINTAINED
                candidate
                candidate_class
                modelVersion
                record_kind
                aggregate
                attribution
                Kaggle · cohere-labs · Global MMLU Lite French
                license_url
                https://www.apache.org/licenses/LICENSE-2.0
                metric_kind
                Global MMLU Lite French · French
                rank_method
                competition rank among reported evaluations in identical conditions, including unlinked observations
                source_file
                138/leaderboard.json
                source_page
                https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-french/versions/1
                measurements
                • name
                  French
                  unit
                  score
                  scale
                  1
                  metric
                  Global MMLU Lite French · French
                  result
                  resultCase
                  numericResult
                  booleanResult
                  false
                  numericResult
                  value
                  0.9425
                  standardDeviation
                  0
                  confidenceInterval
                  0.0228135408783901
                  hasStandardDeviation
                  false
                  hasConfidenceInterval
                  true
                  confidenceIntervalNullable
                  0.0228135408783901
                  hasUnevenConfidenceInterval
                  false
                  taskVersionId
                  0
                  evaluationDate
                  2025-10-30T00:00:00Z
                  hasBooleanResult
                  false
                  hasNumericResult
                  true
                  hasTaskVersionId
                  false
                  hasEvaluationDate
                  true
                  numericResultNullable
                  value
                  0.9425
                  standardDeviation
                  0
                  confidenceInterval
                  0.0228135408783901
                  hasStandardDeviation
                  false
                  hasConfidenceInterval
                  true
                  confidenceIntervalNullable
                  0.0228135408783901
                  hasUnevenConfidenceInterval
                  false
                  hasNumericResultPublic
                  false
                  customAdditionalResults
                    hasNumericResultPrivate
                    false
                    task_id
                    41752
                    task_url
                    /benchmarks/tasks/benchmarkler/global-mmlu-lite-french
                    sort_order
                    DESCENDING
                    source_name
                    French
                    display_type
                    rawValues
                    task_version
                    2
                    task_display_type
                    RAW_VALUES
                    normalization_status
                    normalized
                  published_at
                  2025-10-17T15:30:43.548947500Z
                  candidate_class
                  modelVersion
                  source_revision
                  b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                  publisher_anchor
                  google/gemini-2.5-pro
                  metric_population
                  41
                  benchmark_evaluation
                  true
                  task_metadata_source
                  benchmarkVersion
                  candidate_metadata_source
                  benchmarkVersionMappings
                  evaluated at
                  2025-10-30T00:00:00.000Z
                  source model id
                  Gemini 2.5 Pro
                  confidence lower
                  0.91968645912161
                  confidence upper
                  0.96531354087839
                  Original result ↗

                  Kaggle Benchmarks

                  0.941

                  source id
                  kaggleBenchmarks
                  leaderboard name
                  Global MMLU Lite Culturally Agnostic · CA Score
                  dimension
                  reasoning
                  metric
                  Global MMLU Lite Culturally Agnostic · CA Score
                  rank
                  10
                  aggregation scope
                  kaggle
                  true
                  status
                  published
                  license
                  Apache-2.0
                  direction
                  higherIsBetter
                  conditions
                  task
                  id
                  41744
                  name
                  CA Score
                  version
                  2
                  sort_order
                  DESCENDING
                  display_type
                  rawValues
                  benchmark
                  id
                  209
                  name
                  Global MMLU Lite Culturally Agnostic
                  type
                  INDIVIDUAL
                  owner
                  cohere-labs
                  version
                  1
                  categories
                  • multilingual and cross-lingual capabilities
                  • general knowledge and reasoning
                  version_id
                  188
                  maintainer_tier
                  Kaggle-maintained
                  maintenance_level
                  KAGGLE_MAINTAINED
                  candidate
                  candidate_class
                  modelVersion
                  record_kind
                  aggregate
                  attribution
                  Kaggle · cohere-labs · Global MMLU Lite Culturally Agnostic
                  license_url
                  https://www.apache.org/licenses/LICENSE-2.0
                  metric_kind
                  Global MMLU Lite Culturally Agnostic · CA Score
                  rank_method
                  competition rank among reported evaluations in identical conditions, including unlinked observations
                  source_file
                  188/leaderboard.json
                  source_page
                  https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-ca/versions/1
                  measurements
                  • name
                    CA Score
                    unit
                    score
                    scale
                    1
                    metric
                    Global MMLU Lite Culturally Agnostic · CA Score
                    result
                    resultCase
                    numericResult
                    booleanResult
                    false
                    numericResult
                    value
                    0.9406249999999999
                    standardDeviation
                    0
                    confidenceInterval
                    0
                    hasStandardDeviation
                    false
                    hasConfidenceInterval
                    false
                    hasUnevenConfidenceInterval
                    false
                    taskVersionId
                    0
                    evaluationDate
                    2025-11-03T22:31:06.2566667Z
                    hasBooleanResult
                    false
                    hasNumericResult
                    true
                    hasTaskVersionId
                    false
                    hasEvaluationDate
                    true
                    numericResultNullable
                    value
                    0.9406249999999999
                    standardDeviation
                    0
                    confidenceInterval
                    0
                    hasStandardDeviation
                    false
                    hasConfidenceInterval
                    false
                    hasUnevenConfidenceInterval
                    false
                    hasNumericResultPublic
                    false
                    customAdditionalResults
                      hasNumericResultPrivate
                      false
                      task_id
                      41744
                      task_url
                      /benchmarks/tasks/benchmarkler/global-mmlu-lite-ca-score
                      sort_order
                      DESCENDING
                      source_name
                      CA Score
                      display_type
                      rawValues
                      task_version
                      2
                      task_display_type
                      RAW_VALUES
                      normalization_status
                      normalized
                    published_at
                    2025-11-10T16:52:12.590864700Z
                    candidate_class
                    modelVersion
                    source_revision
                    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                    publisher_anchor
                    google/gemini-2.5-pro
                    metric_population
                    41
                    benchmark_evaluation
                    true
                    task_metadata_source
                    benchmarkVersion
                    candidate_metadata_source
                    benchmarkVersionMappings
                    evaluated at
                    2025-11-03T22:31:06.256Z
                    source model id
                    Gemini 2.5 Pro
                    Original result ↗

                    Kaggle Benchmarks

                    57.518%

                    source id
                    kaggleBenchmarks
                    leaderboard name
                    AssetOpsBench · Consistency
                    dimension
                    reasoning
                    metric
                    AssetOpsBench · Consistency
                    rank
                    10
                    aggregation scope
                    kaggle
                    true
                    status
                    published
                    license
                    Apache-2.0
                    direction
                    higherIsBetter
                    conditions
                    task
                    id
                    1045
                    name
                    Consistency
                    version
                    1
                    sort_order
                    DESCENDING
                    display_type
                    percentages
                    benchmark
                    id
                    243
                    name
                    AssetOpsBench
                    type
                    INDIVIDUAL
                    owner
                    ibm-research
                    version
                    1
                    categories
                    • evaluation
                    • general knowledge and reasoning
                    version_id
                    222
                    maintainer_tier
                    Kaggle-maintained
                    maintenance_level
                    KAGGLE_MAINTAINED
                    candidate
                    candidate_class
                    modelVersion
                    record_kind
                    aggregate
                    attribution
                    Kaggle · ibm-research · AssetOpsBench
                    license_url
                    https://www.apache.org/licenses/LICENSE-2.0
                    metric_kind
                    AssetOpsBench · Consistency
                    rank_method
                    competition rank among reported evaluations in identical conditions, including unlinked observations
                    source_file
                    222/leaderboard.json
                    source_page
                    https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
                    measurements
                    • name
                      Consistency
                      unit
                      percent
                      scale
                      1
                      metric
                      AssetOpsBench · Consistency
                      result
                      resultCase
                      numericResult
                      booleanResult
                      false
                      numericResult
                      value
                      0.5751781027371579
                      standardDeviation
                      0
                      confidenceInterval
                      0
                      hasStandardDeviation
                      false
                      hasConfidenceInterval
                      false
                      hasUnevenConfidenceInterval
                      false
                      taskVersionId
                      0
                      evaluationDate
                      2025-11-24T03:35:15.458771Z
                      hasBooleanResult
                      false
                      hasNumericResult
                      true
                      hasTaskVersionId
                      false
                      hasEvaluationDate
                      true
                      numericResultNullable
                      value
                      0.5751781027371579
                      standardDeviation
                      0
                      confidenceInterval
                      0
                      hasStandardDeviation
                      false
                      hasConfidenceInterval
                      false
                      hasUnevenConfidenceInterval
                      false
                      hasNumericResultPublic
                      false
                      customAdditionalResults
                        hasNumericResultPrivate
                        false
                        task_id
                        1045
                        task_url
                        /benchmarks/tasks/benchmarkler/failuresensoriq-consistency-benchmark-task
                        sort_order
                        DESCENDING
                        source_name
                        Consistency
                        display_type
                        percentages
                        task_version
                        1
                        task_display_type
                        RAW_VALUES
                        normalization_status
                        normalized
                      published_at
                      2025-12-01T22:15:26.261663300Z
                      candidate_class
                      modelVersion
                      source_revision
                      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                      publisher_anchor
                      google/gemini-2.5-pro
                      metric_population
                      25
                      benchmark_evaluation
                      true
                      task_metadata_source
                      benchmarkVersion
                      candidate_metadata_source
                      benchmarkVersionMappings
                      evaluated at
                      2025-11-24T03:35:15.458Z
                      source model id
                      Gemini 2.5 Pro
                      Original result ↗

                      LMArena

                      1,446.579

                      source id
                      lmarena
                      leaderboard name
                      LMArena Text Factuality German
                      dimension
                      factuality
                      metric
                      Arena rating
                      rank
                      12
                      sample size
                      583
                      evaluated at
                      2026-11-09T00:00:00.000Z
                      source model id
                      Gemini 2.5 Pro
                      confidence lower
                      1422.969491280298
                      confidence upper
                      1470.188753874323
                      Original result ↗

                      Kaggle Benchmarks

                      0.928

                      source id
                      kaggleBenchmarks
                      leaderboard name
                      Global MMLU Lite Korean · Korean
                      dimension
                      reasoning
                      metric
                      Global MMLU Lite Korean · Korean
                      rank
                      12
                      aggregation scope
                      kaggle
                      true
                      status
                      published
                      license
                      Apache-2.0
                      direction
                      higherIsBetter
                      conditions
                      task
                      id
                      41748
                      name
                      Korean
                      version
                      2
                      sort_order
                      DESCENDING
                      display_type
                      rawValues
                      benchmark
                      id
                      161
                      name
                      Global MMLU Lite Korean
                      type
                      INDIVIDUAL
                      owner
                      cohere-labs
                      version
                      1
                      categories
                      • multilingual and cross-lingual capabilities
                      • general knowledge and reasoning
                      version_id
                      143
                      maintainer_tier
                      Kaggle-maintained
                      maintenance_level
                      KAGGLE_MAINTAINED
                      candidate
                      candidate_class
                      modelVersion
                      record_kind
                      aggregate
                      attribution
                      Kaggle · cohere-labs · Global MMLU Lite Korean
                      license_url
                      https://www.apache.org/licenses/LICENSE-2.0
                      metric_kind
                      Global MMLU Lite Korean · Korean
                      rank_method
                      competition rank among reported evaluations in identical conditions, including unlinked observations
                      source_file
                      143/leaderboard.json
                      source_page
                      https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-korean/versions/1
                      measurements
                      • name
                        Korean
                        unit
                        score
                        scale
                        1
                        metric
                        Global MMLU Lite Korean · Korean
                        result
                        resultCase
                        numericResult
                        booleanResult
                        false
                        numericResult
                        value
                        0.9275
                        standardDeviation
                        0
                        confidenceInterval
                        0.0254123049217328
                        hasStandardDeviation
                        false
                        hasConfidenceInterval
                        true
                        confidenceIntervalNullable
                        0.0254123049217328
                        hasUnevenConfidenceInterval
                        false
                        taskVersionId
                        0
                        evaluationDate
                        2025-10-30T00:00:00Z
                        hasBooleanResult
                        false
                        hasNumericResult
                        true
                        hasTaskVersionId
                        false
                        hasEvaluationDate
                        true
                        numericResultNullable
                        value
                        0.9275
                        standardDeviation
                        0
                        confidenceInterval
                        0.0254123049217328
                        hasStandardDeviation
                        false
                        hasConfidenceInterval
                        true
                        confidenceIntervalNullable
                        0.0254123049217328
                        hasUnevenConfidenceInterval
                        false
                        hasNumericResultPublic
                        false
                        customAdditionalResults
                          hasNumericResultPrivate
                          false
                          task_id
                          41748
                          task_url
                          /benchmarks/tasks/benchmarkler/global-mmlu-lite-korean
                          sort_order
                          DESCENDING
                          source_name
                          Korean
                          display_type
                          rawValues
                          task_version
                          2
                          task_display_type
                          RAW_VALUES
                          normalization_status
                          normalized
                        published_at
                        2025-10-17T15:30:33.197926100Z
                        candidate_class
                        modelVersion
                        source_revision
                        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                        publisher_anchor
                        google/gemini-2.5-pro
                        metric_population
                        41
                        benchmark_evaluation
                        true
                        task_metadata_source
                        benchmarkVersion
                        candidate_metadata_source
                        benchmarkVersionMappings
                        evaluated at
                        2025-10-30T00:00:00.000Z
                        source model id
                        Gemini 2.5 Pro
                        confidence lower
                        0.9020876950782671
                        confidence upper
                        0.9529123049217328
                        Original result ↗

                        Kaggle Benchmarks

                        0.928

                        source id
                        kaggleBenchmarks
                        leaderboard name
                        Global MMLU Lite Bengali · Bengali
                        dimension
                        reasoning
                        metric
                        Global MMLU Lite Bengali · Bengali
                        rank
                        12
                        aggregation scope
                        kaggle
                        true
                        status
                        published
                        license
                        Apache-2.0
                        direction
                        higherIsBetter
                        conditions
                        task
                        id
                        41751
                        name
                        Bengali
                        version
                        2
                        sort_order
                        DESCENDING
                        display_type
                        rawValues
                        benchmark
                        id
                        154
                        name
                        Global MMLU Lite Bengali
                        type
                        INDIVIDUAL
                        owner
                        cohere-labs
                        version
                        1
                        categories
                        • multilingual and cross-lingual capabilities
                        • general knowledge and reasoning
                        version_id
                        136
                        maintainer_tier
                        Kaggle-maintained
                        maintenance_level
                        KAGGLE_MAINTAINED
                        candidate
                        candidate_class
                        modelVersion
                        record_kind
                        aggregate
                        attribution
                        Kaggle · cohere-labs · Global MMLU Lite Bengali
                        license_url
                        https://www.apache.org/licenses/LICENSE-2.0
                        metric_kind
                        Global MMLU Lite Bengali · Bengali
                        rank_method
                        competition rank among reported evaluations in identical conditions, including unlinked observations
                        source_file
                        136/leaderboard.json
                        source_page
                        https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-bengali/versions/1
                        measurements
                        • name
                          Bengali
                          unit
                          score
                          scale
                          1
                          metric
                          Global MMLU Lite Bengali · Bengali
                          result
                          resultCase
                          numericResult
                          booleanResult
                          false
                          numericResult
                          value
                          0.9275
                          standardDeviation
                          0
                          confidenceInterval
                          0.0254123049217328
                          hasStandardDeviation
                          false
                          hasConfidenceInterval
                          true
                          confidenceIntervalNullable
                          0.0254123049217328
                          hasUnevenConfidenceInterval
                          false
                          taskVersionId
                          0
                          evaluationDate
                          2025-10-30T00:00:00Z
                          hasBooleanResult
                          false
                          hasNumericResult
                          true
                          hasTaskVersionId
                          false
                          hasEvaluationDate
                          true
                          numericResultNullable
                          value
                          0.9275
                          standardDeviation
                          0
                          confidenceInterval
                          0.0254123049217328
                          hasStandardDeviation
                          false
                          hasConfidenceInterval
                          true
                          confidenceIntervalNullable
                          0.0254123049217328
                          hasUnevenConfidenceInterval
                          false
                          hasNumericResultPublic
                          false
                          customAdditionalResults
                            hasNumericResultPrivate
                            false
                            task_id
                            41751
                            task_url
                            /benchmarks/tasks/benchmarkler/global-mmlu-lite-bengali
                            sort_order
                            DESCENDING
                            source_name
                            Bengali
                            display_type
                            rawValues
                            task_version
                            2
                            task_display_type
                            RAW_VALUES
                            normalization_status
                            normalized
                          published_at
                          2025-10-17T15:30:47.931282Z
                          candidate_class
                          modelVersion
                          source_revision
                          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                          publisher_anchor
                          google/gemini-2.5-pro
                          metric_population
                          41
                          benchmark_evaluation
                          true
                          task_metadata_source
                          benchmarkVersion
                          candidate_metadata_source
                          benchmarkVersionMappings
                          evaluated at
                          2025-10-30T00:00:00.000Z
                          source model id
                          Gemini 2.5 Pro
                          confidence lower
                          0.9020876950782671
                          confidence upper
                          0.9529123049217328
                          Original result ↗

                          Kaggle Benchmarks

                          0.928

                          source id
                          kaggleBenchmarks
                          leaderboard name
                          Global MMLU Lite Hindi · Hindi
                          dimension
                          reasoning
                          metric
                          Global MMLU Lite Hindi · Hindi
                          rank
                          12
                          aggregation scope
                          kaggle
                          true
                          status
                          published
                          license
                          Apache-2.0
                          direction
                          higherIsBetter
                          conditions
                          task
                          id
                          41742
                          name
                          Hindi
                          version
                          2
                          sort_order
                          DESCENDING
                          display_type
                          rawValues
                          benchmark
                          id
                          157
                          name
                          Global MMLU Lite Hindi
                          type
                          INDIVIDUAL
                          owner
                          cohere-labs
                          version
                          1
                          categories
                          • multilingual and cross-lingual capabilities
                          • general knowledge and reasoning
                          version_id
                          139
                          maintainer_tier
                          Kaggle-maintained
                          maintenance_level
                          KAGGLE_MAINTAINED
                          candidate
                          candidate_class
                          modelVersion
                          record_kind
                          aggregate
                          attribution
                          Kaggle · cohere-labs · Global MMLU Lite Hindi
                          license_url
                          https://www.apache.org/licenses/LICENSE-2.0
                          metric_kind
                          Global MMLU Lite Hindi · Hindi
                          rank_method
                          competition rank among reported evaluations in identical conditions, including unlinked observations
                          source_file
                          139/leaderboard.json
                          source_page
                          https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-hindi/versions/1
                          measurements
                          • name
                            Hindi
                            unit
                            score
                            scale
                            1
                            metric
                            Global MMLU Lite Hindi · Hindi
                            result
                            resultCase
                            numericResult
                            booleanResult
                            false
                            numericResult
                            value
                            0.9275
                            standardDeviation
                            0
                            confidenceInterval
                            0.0254123049217328
                            hasStandardDeviation
                            false
                            hasConfidenceInterval
                            true
                            confidenceIntervalNullable
                            0.0254123049217328
                            hasUnevenConfidenceInterval
                            false
                            taskVersionId
                            0
                            evaluationDate
                            2025-10-30T00:00:00Z
                            hasBooleanResult
                            false
                            hasNumericResult
                            true
                            hasTaskVersionId
                            false
                            hasEvaluationDate
                            true
                            numericResultNullable
                            value
                            0.9275
                            standardDeviation
                            0
                            confidenceInterval
                            0.0254123049217328
                            hasStandardDeviation
                            false
                            hasConfidenceInterval
                            true
                            confidenceIntervalNullable
                            0.0254123049217328
                            hasUnevenConfidenceInterval
                            false
                            hasNumericResultPublic
                            false
                            customAdditionalResults
                              hasNumericResultPrivate
                              false
                              task_id
                              41742
                              task_url
                              /benchmarks/tasks/benchmarkler/global-mmlu-lite-hindi
                              sort_order
                              DESCENDING
                              source_name
                              Hindi
                              display_type
                              rawValues
                              task_version
                              2
                              task_display_type
                              RAW_VALUES
                              normalization_status
                              normalized
                            published_at
                            2025-10-17T15:30:41.662768400Z
                            candidate_class
                            modelVersion
                            source_revision
                            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                            publisher_anchor
                            google/gemini-2.5-pro
                            metric_population
                            41
                            benchmark_evaluation
                            true
                            task_metadata_source
                            benchmarkVersion
                            candidate_metadata_source
                            benchmarkVersionMappings
                            evaluated at
                            2025-10-30T00:00:00.000Z
                            source model id
                            Gemini 2.5 Pro
                            confidence lower
                            0.9020876950782671
                            confidence upper
                            0.9529123049217328
                            Original result ↗

                            Kaggle Benchmarks

                            0.925

                            source id
                            kaggleBenchmarks
                            leaderboard name
                            Global MMLU Lite Indonesian · Indonesian
                            dimension
                            reasoning
                            metric
                            Global MMLU Lite Indonesian · Indonesian
                            rank
                            12
                            aggregation scope
                            kaggle
                            true
                            status
                            published
                            license
                            Apache-2.0
                            direction
                            higherIsBetter
                            conditions
                            task
                            id
                            41740
                            name
                            Indonesian
                            version
                            2
                            sort_order
                            DESCENDING
                            display_type
                            rawValues
                            benchmark
                            id
                            158
                            name
                            Global MMLU Lite Indonesian
                            type
                            INDIVIDUAL
                            owner
                            cohere-labs
                            version
                            1
                            categories
                            • multilingual and cross-lingual capabilities
                            • general knowledge and reasoning
                            version_id
                            140
                            maintainer_tier
                            Kaggle-maintained
                            maintenance_level
                            KAGGLE_MAINTAINED
                            candidate
                            candidate_class
                            modelVersion
                            record_kind
                            aggregate
                            attribution
                            Kaggle · cohere-labs · Global MMLU Lite Indonesian
                            license_url
                            https://www.apache.org/licenses/LICENSE-2.0
                            metric_kind
                            Global MMLU Lite Indonesian · Indonesian
                            rank_method
                            competition rank among reported evaluations in identical conditions, including unlinked observations
                            source_file
                            140/leaderboard.json
                            source_page
                            https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-indonesian/versions/1
                            measurements
                            • name
                              Indonesian
                              unit
                              score
                              scale
                              1
                              metric
                              Global MMLU Lite Indonesian · Indonesian
                              result
                              resultCase
                              numericResult
                              booleanResult
                              false
                              numericResult
                              value
                              0.925
                              standardDeviation
                              0
                              confidenceInterval
                              0.0258118773864695
                              hasStandardDeviation
                              false
                              hasConfidenceInterval
                              true
                              confidenceIntervalNullable
                              0.0258118773864695
                              hasUnevenConfidenceInterval
                              false
                              taskVersionId
                              0
                              evaluationDate
                              2025-10-30T00:00:00Z
                              hasBooleanResult
                              false
                              hasNumericResult
                              true
                              hasTaskVersionId
                              false
                              hasEvaluationDate
                              true
                              numericResultNullable
                              value
                              0.925
                              standardDeviation
                              0
                              confidenceInterval
                              0.0258118773864695
                              hasStandardDeviation
                              false
                              hasConfidenceInterval
                              true
                              confidenceIntervalNullable
                              0.0258118773864695
                              hasUnevenConfidenceInterval
                              false
                              hasNumericResultPublic
                              false
                              customAdditionalResults
                                hasNumericResultPrivate
                                false
                                task_id
                                41740
                                task_url
                                /benchmarks/tasks/benchmarkler/global-mmlu-lite-indonesian
                                sort_order
                                DESCENDING
                                source_name
                                Indonesian
                                display_type
                                rawValues
                                task_version
                                2
                                task_display_type
                                RAW_VALUES
                                normalization_status
                                normalized
                              published_at
                              2025-10-17T15:30:39.240452500Z
                              candidate_class
                              modelVersion
                              source_revision
                              b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                              publisher_anchor
                              google/gemini-2.5-pro
                              metric_population
                              41
                              benchmark_evaluation
                              true
                              task_metadata_source
                              benchmarkVersion
                              candidate_metadata_source
                              benchmarkVersionMappings
                              evaluated at
                              2025-10-30T00:00:00.000Z
                              source model id
                              Gemini 2.5 Pro
                              confidence lower
                              0.8991881226135305
                              confidence upper
                              0.9508118773864696
                              Original result ↗

                              Kaggle Benchmarks

                              62.992%

                              source id
                              kaggleBenchmarks
                              leaderboard name
                              AssetOpsBench · Pert. Simple Accuracy
                              dimension
                              reasoning
                              metric
                              AssetOpsBench · Pert. Simple Accuracy
                              rank
                              12
                              aggregation scope
                              kaggle
                              true
                              status
                              published
                              license
                              Apache-2.0
                              direction
                              higherIsBetter
                              conditions
                              task
                              id
                              1043
                              name
                              Pert. Simple Accuracy
                              version
                              1
                              sort_order
                              DESCENDING
                              display_type
                              percentages
                              benchmark
                              id
                              243
                              name
                              AssetOpsBench
                              type
                              INDIVIDUAL
                              owner
                              ibm-research
                              version
                              1
                              categories
                              • evaluation
                              • general knowledge and reasoning
                              version_id
                              222
                              maintainer_tier
                              Kaggle-maintained
                              maintenance_level
                              KAGGLE_MAINTAINED
                              candidate
                              candidate_class
                              modelVersion
                              record_kind
                              aggregate
                              attribution
                              Kaggle · ibm-research · AssetOpsBench
                              license_url
                              https://www.apache.org/licenses/LICENSE-2.0
                              metric_kind
                              AssetOpsBench · Pert. Simple Accuracy
                              rank_method
                              competition rank among reported evaluations in identical conditions, including unlinked observations
                              source_file
                              222/leaderboard.json
                              source_page
                              https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
                              measurements
                              • name
                                Pert. Simple Accuracy
                                unit
                                percent
                                scale
                                1
                                metric
                                AssetOpsBench · Pert. Simple Accuracy
                                result
                                resultCase
                                numericResult
                                booleanResult
                                false
                                numericResult
                                value
                                0.6299212598425197
                                standardDeviation
                                0
                                confidenceInterval
                                0
                                hasStandardDeviation
                                false
                                hasConfidenceInterval
                                false
                                hasUnevenConfidenceInterval
                                false
                                taskVersionId
                                0
                                evaluationDate
                                2025-11-23T14:35:37.103515Z
                                hasBooleanResult
                                false
                                hasNumericResult
                                true
                                hasTaskVersionId
                                false
                                hasEvaluationDate
                                true
                                numericResultNullable
                                value
                                0.6299212598425197
                                standardDeviation
                                0
                                confidenceInterval
                                0
                                hasStandardDeviation
                                false
                                hasConfidenceInterval
                                false
                                hasUnevenConfidenceInterval
                                false
                                hasNumericResultPublic
                                false
                                customAdditionalResults
                                  hasNumericResultPrivate
                                  false
                                  task_id
                                  1043
                                  task_url
                                  /benchmarks/tasks/benchmarkler/failuresensoriq-perturbationsimple-benchmark-task
                                  sort_order
                                  DESCENDING
                                  source_name
                                  Pert. Simple Accuracy
                                  display_type
                                  percentages
                                  task_version
                                  1
                                  task_display_type
                                  RAW_VALUES
                                  normalization_status
                                  normalized
                                published_at
                                2025-12-01T22:15:26.261663300Z
                                candidate_class
                                modelVersion
                                source_revision
                                b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                publisher_anchor
                                google/gemini-2.5-pro
                                metric_population
                                25
                                benchmark_evaluation
                                true
                                task_metadata_source
                                benchmarkVersion
                                candidate_metadata_source
                                benchmarkVersionMappings
                                evaluated at
                                2025-11-23T14:35:37.103Z
                                source model id
                                Gemini 2.5 Pro
                                Original result ↗

                                Kaggle Benchmarks

                                0.94

                                source id
                                kaggleBenchmarks
                                leaderboard name
                                Global MMLU Lite Spanish · Spanish
                                dimension
                                reasoning
                                metric
                                Global MMLU Lite Spanish · Spanish
                                rank
                                13
                                aggregation scope
                                kaggle
                                true
                                status
                                published
                                license
                                Apache-2.0
                                direction
                                higherIsBetter
                                conditions
                                task
                                id
                                41741
                                name
                                Spanish
                                version
                                2
                                sort_order
                                DESCENDING
                                display_type
                                rawValues
                                benchmark
                                id
                                163
                                name
                                Global MMLU Lite Spanish
                                type
                                INDIVIDUAL
                                owner
                                cohere-labs
                                version
                                1
                                categories
                                • multilingual and cross-lingual capabilities
                                • general knowledge and reasoning
                                version_id
                                145
                                maintainer_tier
                                Kaggle-maintained
                                maintenance_level
                                KAGGLE_MAINTAINED
                                candidate
                                candidate_class
                                modelVersion
                                record_kind
                                aggregate
                                attribution
                                Kaggle · cohere-labs · Global MMLU Lite Spanish
                                license_url
                                https://www.apache.org/licenses/LICENSE-2.0
                                metric_kind
                                Global MMLU Lite Spanish · Spanish
                                rank_method
                                competition rank among reported evaluations in identical conditions, including unlinked observations
                                source_file
                                145/leaderboard.json
                                source_page
                                https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-spanish/versions/1
                                measurements
                                • name
                                  Spanish
                                  unit
                                  score
                                  scale
                                  1
                                  metric
                                  Global MMLU Lite Spanish · Spanish
                                  result
                                  resultCase
                                  numericResult
                                  booleanResult
                                  false
                                  numericResult
                                  value
                                  0.94
                                  standardDeviation
                                  0
                                  confidenceInterval
                                  0.0232732828307025
                                  hasStandardDeviation
                                  false
                                  hasConfidenceInterval
                                  true
                                  confidenceIntervalNullable
                                  0.0232732828307025
                                  hasUnevenConfidenceInterval
                                  false
                                  taskVersionId
                                  0
                                  evaluationDate
                                  2025-10-30T00:00:00Z
                                  hasBooleanResult
                                  false
                                  hasNumericResult
                                  true
                                  hasTaskVersionId
                                  false
                                  hasEvaluationDate
                                  true
                                  numericResultNullable
                                  value
                                  0.94
                                  standardDeviation
                                  0
                                  confidenceInterval
                                  0.0232732828307025
                                  hasStandardDeviation
                                  false
                                  hasConfidenceInterval
                                  true
                                  confidenceIntervalNullable
                                  0.0232732828307025
                                  hasUnevenConfidenceInterval
                                  false
                                  hasNumericResultPublic
                                  false
                                  customAdditionalResults
                                    hasNumericResultPrivate
                                    false
                                    task_id
                                    41741
                                    task_url
                                    /benchmarks/tasks/benchmarkler/global-mmlu-lite-spanish
                                    sort_order
                                    DESCENDING
                                    source_name
                                    Spanish
                                    display_type
                                    rawValues
                                    task_version
                                    2
                                    task_display_type
                                    RAW_VALUES
                                    normalization_status
                                    normalized
                                  published_at
                                  2025-10-17T15:30:28.873808200Z
                                  candidate_class
                                  modelVersion
                                  source_revision
                                  b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                  publisher_anchor
                                  google/gemini-2.5-pro
                                  metric_population
                                  41
                                  benchmark_evaluation
                                  true
                                  task_metadata_source
                                  benchmarkVersion
                                  candidate_metadata_source
                                  benchmarkVersionMappings
                                  evaluated at
                                  2025-10-30T00:00:00.000Z
                                  source model id
                                  Gemini 2.5 Pro
                                  confidence lower
                                  0.9167267171692974
                                  confidence upper
                                  0.9632732828307025
                                  Original result ↗

                                  Kaggle Benchmarks

                                  0.938

                                  source id
                                  kaggleBenchmarks
                                  leaderboard name
                                  Global MMLU Lite Japanese · Japanese
                                  dimension
                                  reasoning
                                  metric
                                  Global MMLU Lite Japanese · Japanese
                                  rank
                                  13
                                  aggregation scope
                                  kaggle
                                  true
                                  status
                                  published
                                  license
                                  Apache-2.0
                                  direction
                                  higherIsBetter
                                  conditions
                                  task
                                  id
                                  41745
                                  name
                                  Japanese
                                  version
                                  2
                                  sort_order
                                  DESCENDING
                                  display_type
                                  rawValues
                                  benchmark
                                  id
                                  160
                                  name
                                  Global MMLU Lite Japanese
                                  type
                                  INDIVIDUAL
                                  owner
                                  cohere-labs
                                  version
                                  1
                                  categories
                                  • multilingual and cross-lingual capabilities
                                  • general knowledge and reasoning
                                  version_id
                                  142
                                  maintainer_tier
                                  Kaggle-maintained
                                  maintenance_level
                                  KAGGLE_MAINTAINED
                                  candidate
                                  candidate_class
                                  modelVersion
                                  record_kind
                                  aggregate
                                  attribution
                                  Kaggle · cohere-labs · Global MMLU Lite Japanese
                                  license_url
                                  https://www.apache.org/licenses/LICENSE-2.0
                                  metric_kind
                                  Global MMLU Lite Japanese · Japanese
                                  rank_method
                                  competition rank among reported evaluations in identical conditions, including unlinked observations
                                  source_file
                                  142/leaderboard.json
                                  source_page
                                  https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-japanese/versions/1
                                  measurements
                                  • name
                                    Japanese
                                    unit
                                    score
                                    scale
                                    1
                                    metric
                                    Global MMLU Lite Japanese · Japanese
                                    result
                                    resultCase
                                    numericResult
                                    booleanResult
                                    false
                                    numericResult
                                    value
                                    0.9375
                                    standardDeviation
                                    0
                                    confidenceInterval
                                    0.0237215870977811
                                    hasStandardDeviation
                                    false
                                    hasConfidenceInterval
                                    true
                                    confidenceIntervalNullable
                                    0.0237215870977811
                                    hasUnevenConfidenceInterval
                                    false
                                    taskVersionId
                                    0
                                    evaluationDate
                                    2025-10-30T00:00:00Z
                                    hasBooleanResult
                                    false
                                    hasNumericResult
                                    true
                                    hasTaskVersionId
                                    false
                                    hasEvaluationDate
                                    true
                                    numericResultNullable
                                    value
                                    0.9375
                                    standardDeviation
                                    0
                                    confidenceInterval
                                    0.0237215870977811
                                    hasStandardDeviation
                                    false
                                    hasConfidenceInterval
                                    true
                                    confidenceIntervalNullable
                                    0.0237215870977811
                                    hasUnevenConfidenceInterval
                                    false
                                    hasNumericResultPublic
                                    false
                                    customAdditionalResults
                                      hasNumericResultPrivate
                                      false
                                      task_id
                                      41745
                                      task_url
                                      /benchmarks/tasks/benchmarkler/global-mmlu-lite-japanese
                                      sort_order
                                      DESCENDING
                                      source_name
                                      Japanese
                                      display_type
                                      rawValues
                                      task_version
                                      2
                                      task_display_type
                                      RAW_VALUES
                                      normalization_status
                                      normalized
                                    published_at
                                    2025-10-17T15:30:34.896663Z
                                    candidate_class
                                    modelVersion
                                    source_revision
                                    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                    publisher_anchor
                                    google/gemini-2.5-pro
                                    metric_population
                                    41
                                    benchmark_evaluation
                                    true
                                    task_metadata_source
                                    benchmarkVersion
                                    candidate_metadata_source
                                    benchmarkVersionMappings
                                    evaluated at
                                    2025-10-30T00:00:00.000Z
                                    source model id
                                    Gemini 2.5 Pro
                                    confidence lower
                                    0.913778412902219
                                    confidence upper
                                    0.961221587097781
                                    Original result ↗

                                    Kaggle Benchmarks

                                    0.935

                                    source id
                                    kaggleBenchmarks
                                    leaderboard name
                                    Global MMLU Lite Italian · Italian
                                    dimension
                                    reasoning
                                    metric
                                    Global MMLU Lite Italian · Italian
                                    rank
                                    14
                                    aggregation scope
                                    kaggle
                                    true
                                    status
                                    published
                                    license
                                    Apache-2.0
                                    direction
                                    higherIsBetter
                                    conditions
                                    task
                                    id
                                    41739
                                    name
                                    Italian
                                    version
                                    2
                                    sort_order
                                    DESCENDING
                                    display_type
                                    rawValues
                                    benchmark
                                    id
                                    159
                                    name
                                    Global MMLU Lite Italian
                                    type
                                    INDIVIDUAL
                                    owner
                                    cohere-labs
                                    version
                                    1
                                    categories
                                    • multilingual and cross-lingual capabilities
                                    • general knowledge and reasoning
                                    version_id
                                    141
                                    maintainer_tier
                                    Kaggle-maintained
                                    maintenance_level
                                    KAGGLE_MAINTAINED
                                    candidate
                                    candidate_class
                                    modelVersion
                                    record_kind
                                    aggregate
                                    attribution
                                    Kaggle · cohere-labs · Global MMLU Lite Italian
                                    license_url
                                    https://www.apache.org/licenses/LICENSE-2.0
                                    metric_kind
                                    Global MMLU Lite Italian · Italian
                                    rank_method
                                    competition rank among reported evaluations in identical conditions, including unlinked observations
                                    source_file
                                    141/leaderboard.json
                                    source_page
                                    https://www.kaggle.com/benchmarks/cohere-labs/global-mmlu-lite-italian/versions/1
                                    measurements
                                    • name
                                      Italian
                                      unit
                                      score
                                      scale
                                      1
                                      metric
                                      Global MMLU Lite Italian · Italian
                                      result
                                      resultCase
                                      numericResult
                                      booleanResult
                                      false
                                      numericResult
                                      value
                                      0.935
                                      standardDeviation
                                      0
                                      confidenceInterval
                                      0.0241590904127041
                                      hasStandardDeviation
                                      false
                                      hasConfidenceInterval
                                      true
                                      confidenceIntervalNullable
                                      0.0241590904127041
                                      hasUnevenConfidenceInterval
                                      false
                                      taskVersionId
                                      0
                                      evaluationDate
                                      2025-10-30T00:00:00Z
                                      hasBooleanResult
                                      false
                                      hasNumericResult
                                      true
                                      hasTaskVersionId
                                      false
                                      hasEvaluationDate
                                      true
                                      numericResultNullable
                                      value
                                      0.935
                                      standardDeviation
                                      0
                                      confidenceInterval
                                      0.0241590904127041
                                      hasStandardDeviation
                                      false
                                      hasConfidenceInterval
                                      true
                                      confidenceIntervalNullable
                                      0.0241590904127041
                                      hasUnevenConfidenceInterval
                                      false
                                      hasNumericResultPublic
                                      false
                                      customAdditionalResults
                                        hasNumericResultPrivate
                                        false
                                        task_id
                                        41739
                                        task_url
                                        /benchmarks/tasks/benchmarkler/global-mmlu-lite-italian
                                        sort_order
                                        DESCENDING
                                        source_name
                                        Italian
                                        display_type
                                        rawValues
                                        task_version
                                        2
                                        task_display_type
                                        RAW_VALUES
                                        normalization_status
                                        normalized
                                      published_at
                                      2025-10-17T15:30:36.963753600Z
                                      candidate_class
                                      modelVersion
                                      source_revision
                                      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                      publisher_anchor
                                      google/gemini-2.5-pro
                                      metric_population
                                      41
                                      benchmark_evaluation
                                      true
                                      task_metadata_source
                                      benchmarkVersion
                                      candidate_metadata_source
                                      benchmarkVersionMappings
                                      evaluated at
                                      2025-10-30T00:00:00.000Z
                                      source model id
                                      Gemini 2.5 Pro
                                      confidence lower
                                      0.9108409095872959
                                      confidence upper
                                      0.9591590904127042
                                      Original result ↗

                                      Published prices

                                      1–20 / 60

                                      google/gemini-2.5-pro ↗

                                      $0.13

                                      Inference provider
                                      cloudflareWorkersAI
                                      Platform
                                      Google Vertex AI
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      Billing label
                                      Cached input tokens (per 1M)
                                      Pricing via
                                      Cloudflare AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $1.25

                                      Inference provider
                                      cloudflareWorkersAI
                                      Platform
                                      Google Vertex AI
                                      Unit
                                      input_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      Billing label
                                      Input tokens (per 1M)
                                      Pricing via
                                      Cloudflare AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $10.00

                                      Inference provider
                                      cloudflareWorkersAI
                                      Platform
                                      Google Vertex AI
                                      Unit
                                      output_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      Billing label
                                      Output tokens (per 1M)
                                      Pricing via
                                      Cloudflare AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $1.25

                                      Inference provider
                                      deepInfra
                                      Platform
                                      DeepInfra
                                      Unit
                                      input_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions

                                      google/gemini-2.5-pro ↗

                                      $10.00

                                      Inference provider
                                      deepInfra
                                      Platform
                                      DeepInfra
                                      Unit
                                      output_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions

                                      google/gemini-2.5-pro ↗

                                      $0.13

                                      Inference provider
                                      openRouter
                                      Platform
                                      OpenRouter
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions

                                      google/gemini-2.5-pro ↗

                                      $0.38

                                      Inference provider
                                      openRouter
                                      Platform
                                      OpenRouter
                                      Unit
                                      cache_write_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions

                                      google/gemini-2.5-pro ↗

                                      $1.25

                                      Inference provider
                                      openRouter
                                      Platform
                                      OpenRouter
                                      Unit
                                      input_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions

                                      google/gemini-2.5-pro ↗

                                      $10.00

                                      Inference provider
                                      openRouter
                                      Platform
                                      OpenRouter
                                      Unit
                                      output_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions

                                      google/gemini-2.5-pro ↗

                                      $0.25

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      context_tokens
                                      min_inclusive
                                      200001
                                      Billing label
                                      Cached input / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $0.13

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      context_tokens
                                      max_exclusive
                                      200001
                                      min_inclusive
                                      0
                                      Billing label
                                      Cached input / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $0.22

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      service_tier
                                      priority
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      Billing label
                                      Cached input / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $0.25

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      region
                                      eu
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      context_tokens
                                      min_inclusive
                                      200001
                                      Billing label
                                      Cached input / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $0.13

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      region
                                      eu
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      context_tokens
                                      max_exclusive
                                      200001
                                      min_inclusive
                                      0
                                      Billing label
                                      Cached input / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $0.25

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      region
                                      us
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      context_tokens
                                      min_inclusive
                                      200001
                                      Billing label
                                      Cached input / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $0.13

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      cache_read_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      region
                                      us
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      context_tokens
                                      max_exclusive
                                      200001
                                      min_inclusive
                                      0
                                      Billing label
                                      Cached input / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $2.50

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      input_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      context_tokens
                                      min_inclusive
                                      200001
                                      Billing label
                                      Input tokens / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      google/gemini-2.5-pro ↗

                                      $1.25

                                      Inference provider
                                      vercelAIGatewayCatalog
                                      Platform
                                      Vercel AI Gateway
                                      Unit
                                      input_tokens_per_million
                                      Currency
                                      USD
                                      Billing conditions
                                      model_type
                                      language
                                      quote_scope
                                      gateway_catalog
                                      billing_scope
                                      Vercel public catalog; endpoint prices may differ
                                      context_tokens
                                      max_exclusive
                                      200001
                                      min_inclusive
                                      0
                                      Billing label
                                      Input tokens / 1M
                                      Pricing via
                                      Vercel AI Gateway

                                      Provider specifications & access

                                      1–4 / 4

                                      gemini-2.5-pro ↗

                                      source model id
                                      gemini-2.5-pro
                                      hosting
                                      gateway
                                      platforms
                                      • Cheaper Inference
                                      context length
                                      1048576
                                      max output tokens
                                      65535
                                      architecture
                                      input_modalities
                                      • text
                                      • image
                                      output_modalities
                                      • text
                                      capabilities
                                      • reasoning
                                      • structuredOutputs
                                      • toolCalling
                                      • visionInput
                                      • webSearch

                                      Gemini 2.5 Pro ↗

                                      source model id
                                      google/gemini-2.5-pro
                                      hosting
                                      third-party
                                      platforms
                                      • Google Vertex AI
                                      context length
                                      1000000
                                      max output tokens
                                      65536
                                      architecture
                                      input_modalities
                                      • text
                                      output_modalities
                                      • text
                                      capabilities
                                      • reasoning

                                      Gemini 2.5 Pro ↗

                                      source model id
                                      google/gemini-2.5-pro
                                      hosting
                                      vercel-gateway
                                      platforms
                                      • Vercel AI Gateway
                                      context length
                                      1048576
                                      max output tokens
                                      65535
                                      architecture
                                      input_modalities
                                      • text
                                      • image
                                      • pdf
                                      output_modalities
                                      • text
                                      capabilities
                                      • promptCaching
                                      • reasoning
                                      • structuredOutputs
                                      • toolCalling
                                      • visionInput
                                      • webSearch
                                      regions
                                      • eu
                                      • us

                                      Gemini 2.5 Pro ↗

                                      source model id
                                      publishers/google/models/gemini-2.5-pro@default
                                      hosting
                                      vertex-managed
                                      platforms
                                      • Google Cloud Vertex AI
                                      context length
                                      1048576
                                      max output tokens
                                      65536
                                      architecture
                                      input_modalities
                                      • audio
                                      • image
                                      • text
                                      • video
                                      output_modalities
                                      • text
                                      capabilities
                                      • batch
                                      • multimodal
                                      • promptCaching
                                      • reasoning
                                      • structuredOutputs
                                      • toolCalling
                                      • visionInput

                                      Safety assessments

                                      1–1 / 1

                                      Foundation Model Transparency Index 2025 ↗

                                      41 / 100

                                      source id
                                      stanfordFMTI
                                      canonical model id
                                      google/gemini-2.5-pro
                                      taxonomy
                                      transparency
                                      hazard category
                                      transparency
                                      hazard name
                                      Organization transparency
                                      policy area
                                      Google
                                      assessment type
                                      organization-level disclosure
                                      metric
                                      disclosure_score
                                      result label
                                      41 / 100
                                      sample size
                                      100
                                      updated at
                                      2026-11-09T18:08:05.227Z

                                      Source specification observations

                                      1–10 / 10

                                      cloudflareWorkersAI ↗

                                      field
                                      context_length
                                      value
                                      1000000
                                      normalized value
                                      1000000
                                      scope
                                      source-model
                                      source id
                                      cloudflareWorkersAI
                                      confidence
                                      1

                                      openRouter ↗

                                      field
                                      context_length
                                      value
                                      1048576
                                      normalized value
                                      1048576
                                      scope
                                      source-model
                                      source id
                                      openRouter
                                      confidence
                                      1

                                      vercelAIGatewayCatalog ↗

                                      field
                                      context_length
                                      value
                                      1048576
                                      normalized value
                                      1048576
                                      scope
                                      source-model
                                      source id
                                      vercelAIGatewayCatalog
                                      confidence
                                      1

                                      vertexAI ↗

                                      field
                                      context_length
                                      value
                                      1048576
                                      normalized value
                                      1048576
                                      scope
                                      source-model
                                      source id
                                      vertexAI
                                      confidence
                                      1

                                      openRouter ↗

                                      field
                                      knowledge_cutoff
                                      reported field
                                      knowledge_cutoff
                                      value
                                      2025-01-31
                                      normalized value
                                      2025-01-31
                                      precision
                                      day
                                      scope
                                      source-model
                                      source id
                                      openRouter
                                      confidence
                                      1

                                      vercelAIGatewayCatalog ↗

                                      field
                                      knowledge_cutoff
                                      reported field
                                      knowledge_cutoff
                                      value
                                      2025-01
                                      normalized value
                                      2025-01
                                      precision
                                      month
                                      scope
                                      source-model
                                      source id
                                      vercelAIGatewayCatalog
                                      confidence
                                      1

                                      cloudflareWorkersAI ↗

                                      field
                                      max_output_tokens
                                      value
                                      65536
                                      normalized value
                                      65536
                                      scope
                                      source-model
                                      source id
                                      cloudflareWorkersAI
                                      confidence
                                      1

                                      openRouter ↗

                                      field
                                      max_output_tokens
                                      value
                                      65536
                                      normalized value
                                      65536
                                      scope
                                      source-model
                                      source id
                                      openRouter
                                      confidence
                                      1

                                      vercelAIGatewayCatalog ↗

                                      field
                                      max_output_tokens
                                      value
                                      65535
                                      normalized value
                                      65535
                                      scope
                                      source-model
                                      source id
                                      vercelAIGatewayCatalog
                                      confidence
                                      1

                                      vertexAI ↗

                                      field
                                      max_output_tokens
                                      value
                                      65536
                                      normalized value
                                      65536
                                      scope
                                      source-model
                                      source id
                                      vertexAI
                                      confidence
                                      1

                                      Original sources

                                      1–20 / 20

                                      Google · Model reference ↗

                                      DeepInfra · Model page ↗

                                      Google Cloud · Model page ↗

                                      OpenRouter · Model page ↗

                                      Vercel AI Gateway · Model page ↗

                                      Google Cloud · Documentation ↗

                                      GitHub · Source snapshot ↗

                                      api.cheaperinference.com · Source snapshot ↗

                                      Cloudflare AI · Catalog ↗

                                      DeepInfra · Catalog ↗

                                      Epoch AI · Catalog ↗

                                      GitHub · Catalog ↗

                                      GitHub · Catalog ↗

                                      Google Cloud · Catalog ↗

                                      Kaggle · Catalog ↗

                                      LMArena · Catalog ↗

                                      OpenRouter · Catalog ↗

                                      OpenRouter · Catalog ↗

                                      Vercel AI Gateway · Catalog ↗

                                      www.swebench.com · Catalog ↗

                                      Source links

                                      1–14 / 14

                                      Explore more models

                                      Models

                                      Loading
                                      EVIDENCE / COMPARISON

                                      Benchmarks

                                      Explore every benchmark

                                      Loading

                                      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                                      Install TensorTrace