OpenAI: o3

o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Us…

openai/o3-2025-04-16

o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Use it to think through multi-step problems that involve analysis across text, code, and images.

Specifications & capabilities

Input
image, text, file
Output
text
Context window · tokens
200000
Capabilities
fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗

Evaluations

101–120 / 328

Kaggle Benchmarks

564.947

source id
kaggleBenchmarks
leaderboard name
Nine Men's Morris · Score
dimension
general
metric
Nine Men's Morris · Score
rank
25
aggregation scope
kaggle
true
status
published
license
Apache-2.0
direction
higherIsBetter
conditions
task
id
41675
name
Score
version
1
sort_order
DESCENDING
display_type
counts
benchmark
id
6479
name
Nine Men's Morris
type
GAME
owner
kaggle
version
1
categories
    version_id
    6498
    maintainer_tier
    Kaggle-published
    candidate
    candidate_class
    modelVersion
    collection
    kaggleGameArena
    record_kind
    aggregate
    game_arena
    true
    attribution
    Kaggle · kaggle · Nine Men's Morris
    license_url
    https://www.apache.org/licenses/LICENSE-2.0
    metric_kind
    Nine Men's Morris · Score
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    6498/leaderboard.json
    source_page
    https://www.kaggle.com/benchmarks/kaggle/nine-mens-morris/versions/1
    measurements
    • name
      Score
      unit
      score
      scale
      1
      metric
      Nine Men's Morris · Score
      result
      resultCase
      numericResult
      booleanResult
      false
      numericResult
      value
      564.946923338613
      standardDeviation
      0
      confidenceInterval
      0
      hasStandardDeviation
      false
      hasConfidenceInterval
      false
      unevenConfidenceInterval
      plus
      30.65602957699275
      minus
      35.68289180942145
      hasUnevenConfidenceInterval
      true
      taskVersionId
      0
      evaluationDate
      2026-11-11T20:01:17.6916056Z
      hasBooleanResult
      false
      hasNumericResult
      true
      hasTaskVersionId
      false
      hasEvaluationDate
      true
      numericResultNullable
      value
      564.946923338613
      standardDeviation
      0
      confidenceInterval
      0
      hasStandardDeviation
      false
      hasConfidenceInterval
      false
      unevenConfidenceInterval
      plus
      30.65602957699275
      minus
      35.68289180942145
      hasUnevenConfidenceInterval
      true
      hasNumericResultPublic
      false
      customAdditionalResults
        hasNumericResultPrivate
        false
        task_id
        41675
        sort_order
        DESCENDING
        source_name
        Score
        display_type
        counts
        task_version
        1
        task_display_type
        COUNTS
        normalization_status
        normalized
      published_at
      2026-08-04T20:43:32.035505100Z
      candidate_class
      modelVersion
      methodology_url
      https://www.kaggle.com/game-arena-faq
      source_revision
      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
      publisher_anchor
      openai/o3-2025-04-16
      game_arena_rating
      Game-specific rating; compare within this game and version
      metric_population
      39
      benchmark_evaluation
      true
      task_metadata_source
      benchmarkVersion
      candidate_metadata_source
      benchmarkVersionMappings
      evaluated at
      2026-11-11T20:01:17.691Z
      source model id
      o3
      confidence lower
      529.2640315291916
      confidence upper
      595.6029529156058
      Original result ↗

      Kaggle Benchmarks

      271.531

      source id
      kaggleBenchmarks
      leaderboard name
      Game Arena · Score
      dimension
      general
      metric
      Game Arena · Score
      rank
      25
      aggregation scope
      kaggle
      true
      status
      published
      license
      Apache-2.0
      direction
      higherIsBetter
      conditions
      task
      id
      115
      name
      Score
      version
      1
      sort_order
      DESCENDING
      display_type
      counts
      benchmark
      id
      45
      name
      Game Arena
      type
      SUITE
      owner
      kaggle
      version
      1
      categories
        version_id
        31
        maintainer_tier
        Kaggle-published
        candidate
        candidate_class
        modelVersion
        collection
        kaggleGameArena
        record_kind
        aggregate
        game_arena
        true
        attribution
        Kaggle · kaggle · Game Arena
        license_url
        https://www.apache.org/licenses/LICENSE-2.0
        metric_kind
        Game Arena · Score
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        source_file
        31/leaderboard.json
        source_page
        https://www.kaggle.com/benchmarks/kaggle/game-arena/versions/1
        measurements
        • name
          Score
          unit
          score
          scale
          1
          metric
          Game Arena · Score
          result
          resultCase
          numericResult
          booleanResult
          false
          numericResult
          value
          271.5305997506342
          standardDeviation
          0
          confidenceInterval
          0
          hasStandardDeviation
          false
          hasConfidenceInterval
          false
          unevenConfidenceInterval
          plus
          6.110219814332254
          minus
          6.309314891367251
          hasUnevenConfidenceInterval
          true
          taskVersionId
          0
          evaluationDate
          2026-11-12T17:14:24.8808889Z
          hasBooleanResult
          false
          hasNumericResult
          true
          hasTaskVersionId
          false
          hasEvaluationDate
          true
          numericResultNullable
          value
          271.5305997506342
          standardDeviation
          0
          confidenceInterval
          0
          hasStandardDeviation
          false
          hasConfidenceInterval
          false
          unevenConfidenceInterval
          plus
          6.110219814332254
          minus
          6.309314891367251
          hasUnevenConfidenceInterval
          true
          hasNumericResultPublic
          false
          customAdditionalResults
            hasNumericResultPrivate
            false
            task_id
            115
            sort_order
            DESCENDING
            source_name
            Score
            display_type
            counts
            task_version
            1
            task_display_type
            COUNTS
            normalization_status
            normalized
          published_at
          2026-04-09T15:30:02.186914800Z
          candidate_class
          modelVersion
          methodology_url
          https://www.kaggle.com/game-arena-faq
          source_revision
          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
          publisher_anchor
          openai/o3-2025-04-16
          game_arena_rating
          Pooled Bradley–Terry rating; equal total weight per game
          metric_population
          42
          benchmark_evaluation
          true
          task_metadata_source
          benchmarkVersion
          candidate_metadata_source
          benchmarkVersionMappings
          evaluated at
          2026-11-12T17:14:24.880Z
          source model id
          o3
          confidence lower
          265.221284859267
          confidence upper
          277.6408195649665
          Original result ↗

          Kaggle Benchmarks

          87.5%

          source id
          kaggleBenchmarks
          leaderboard name
          AIME 2025 · aime_task
          dimension
          math
          metric
          AIME 2025 · aime_task
          rank
          25
          aggregation scope
          kaggle
          true
          status
          published
          license
          Apache-2.0
          direction
          higherIsBetter
          conditions
          task
          id
          40087
          name
          aime_task
          version
          6
          sort_order
          DESCENDING
          display_type
          percentages
          benchmark
          id
          125
          name
          AIME 2025
          type
          INDIVIDUAL
          owner
          open-benchmarks
          version
          1
          categories
          • general knowledge and reasoning
          • math
          version_id
          111
          maintainer_tier
          Kaggle-maintained
          maintenance_level
          KAGGLE_MAINTAINED
          candidate
          candidate_class
          modelVersion
          record_kind
          aggregate
          attribution
          Kaggle · open-benchmarks · AIME 2025
          license_url
          https://www.apache.org/licenses/LICENSE-2.0
          metric_kind
          AIME 2025 · aime_task
          rank_method
          competition rank among reported evaluations in identical conditions, including unlinked observations
          source_file
          111/leaderboard.json
          source_page
          https://www.kaggle.com/benchmarks/open-benchmarks/aime-2025/versions/1
          measurements
          • name
            aime_task
            unit
            percent
            scale
            1
            metric
            AIME 2025 · aime_task
            result
            resultCase
            numericResult
            booleanResult
            false
            numericResult
            value
            0.875
            standardDeviation
            0
            confidenceInterval
            0
            hasStandardDeviation
            false
            hasConfidenceInterval
            false
            hasUnevenConfidenceInterval
            false
            taskVersionId
            0
            evaluationDate
            2025-07-07T00:00:00Z
            hasBooleanResult
            false
            hasNumericResult
            true
            hasTaskVersionId
            false
            hasEvaluationDate
            true
            numericResultNullable
            value
            0.875
            standardDeviation
            0
            confidenceInterval
            0
            hasStandardDeviation
            false
            hasConfidenceInterval
            false
            hasUnevenConfidenceInterval
            false
            hasNumericResultPublic
            false
            customAdditionalResults
              hasNumericResultPrivate
              false
              task_id
              40087
              task_url
              /benchmarks/tasks/benchmarkler/aime-task
              sort_order
              DESCENDING
              source_name
              aime_task
              display_type
              percentages
              task_version
              6
              task_display_type
              PERCENTAGES
              normalization_status
              normalized
            published_at
            2025-07-15T11:49:31.790Z
            candidate_class
            modelVersion
            source_revision
            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
            publisher_anchor
            openai/o3-2025-04-16
            metric_population
            68
            benchmark_evaluation
            true
            task_metadata_source
            benchmarkVersion
            candidate_metadata_source
            benchmarkVersionMappings
            evaluated at
            2025-07-07T00:00:00.000Z
            source model id
            o3
            Original result ↗

            Kaggle Benchmarks

            240.724

            source id
            kaggleBenchmarks
            leaderboard name
            Ultimate Tic-Tac-Toe · Score
            dimension
            general
            metric
            Ultimate Tic-Tac-Toe · Score
            rank
            26
            aggregation scope
            kaggle
            true
            status
            published
            license
            Apache-2.0
            direction
            higherIsBetter
            conditions
            task
            id
            38391
            name
            Score
            version
            1
            sort_order
            DESCENDING
            display_type
            counts
            benchmark
            id
            6202
            name
            Ultimate Tic-Tac-Toe
            type
            GAME
            owner
            kaggle
            version
            1
            categories
              version_id
              6226
              maintainer_tier
              Kaggle-published
              candidate
              candidate_class
              modelVersion
              collection
              kaggleGameArena
              record_kind
              aggregate
              game_arena
              true
              attribution
              Kaggle · kaggle · Ultimate Tic-Tac-Toe
              license_url
              https://www.apache.org/licenses/LICENSE-2.0
              metric_kind
              Ultimate Tic-Tac-Toe · Score
              rank_method
              competition rank among reported evaluations in identical conditions, including unlinked observations
              source_file
              6226/leaderboard.json
              source_page
              https://www.kaggle.com/benchmarks/kaggle/ultimate-tic-tac-toe/versions/1
              measurements
              • name
                Score
                unit
                score
                scale
                1
                metric
                Ultimate Tic-Tac-Toe · Score
                result
                resultCase
                numericResult
                booleanResult
                false
                numericResult
                value
                240.72428005775396
                standardDeviation
                0
                confidenceInterval
                0
                hasStandardDeviation
                false
                hasConfidenceInterval
                false
                unevenConfidenceInterval
                plus
                20.624959807400387
                minus
                24.601573407941572
                hasUnevenConfidenceInterval
                true
                taskVersionId
                0
                evaluationDate
                2026-11-11T19:59:56.7752345Z
                hasBooleanResult
                false
                hasNumericResult
                true
                hasTaskVersionId
                false
                hasEvaluationDate
                true
                numericResultNullable
                value
                240.72428005775396
                standardDeviation
                0
                confidenceInterval
                0
                hasStandardDeviation
                false
                hasConfidenceInterval
                false
                unevenConfidenceInterval
                plus
                20.624959807400387
                minus
                24.601573407941572
                hasUnevenConfidenceInterval
                true
                hasNumericResultPublic
                false
                customAdditionalResults
                  hasNumericResultPrivate
                  false
                  task_id
                  38391
                  sort_order
                  DESCENDING
                  source_name
                  Score
                  display_type
                  counts
                  task_version
                  1
                  task_display_type
                  COUNTS
                  normalization_status
                  normalized
                published_at
                2026-07-28T19:19:16.035653200Z
                candidate_class
                modelVersion
                methodology_url
                https://www.kaggle.com/game-arena-faq
                source_revision
                b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                publisher_anchor
                openai/o3-2025-04-16
                game_arena_rating
                Game-specific rating; compare within this game and version
                metric_population
                39
                benchmark_evaluation
                true
                task_metadata_source
                benchmarkVersion
                candidate_metadata_source
                benchmarkVersionMappings
                evaluated at
                2026-11-11T19:59:56.775Z
                source model id
                o3
                confidence lower
                216.1227066498124
                confidence upper
                261.34923986515435
                Original result ↗

                Kaggle Benchmarks

                222.533

                source id
                kaggleBenchmarks
                leaderboard name
                Dark Hex · Score
                dimension
                general
                metric
                Dark Hex · Score
                rank
                26
                aggregation scope
                kaggle
                true
                status
                published
                license
                Apache-2.0
                direction
                higherIsBetter
                conditions
                task
                id
                33066
                name
                Score
                version
                1
                sort_order
                DESCENDING
                display_type
                counts
                benchmark
                id
                5571
                name
                Dark Hex
                type
                GAME
                owner
                kaggle
                version
                1
                categories
                  version_id
                  5594
                  maintainer_tier
                  Kaggle-published
                  candidate
                  candidate_class
                  modelVersion
                  collection
                  kaggleGameArena
                  record_kind
                  aggregate
                  game_arena
                  true
                  attribution
                  Kaggle · kaggle · Dark Hex
                  license_url
                  https://www.apache.org/licenses/LICENSE-2.0
                  metric_kind
                  Dark Hex · Score
                  rank_method
                  competition rank among reported evaluations in identical conditions, including unlinked observations
                  source_file
                  5594/leaderboard.json
                  source_page
                  https://www.kaggle.com/benchmarks/kaggle/dark-hex/versions/1
                  measurements
                  • name
                    Score
                    unit
                    score
                    scale
                    1
                    metric
                    Dark Hex · Score
                    result
                    resultCase
                    numericResult
                    booleanResult
                    false
                    numericResult
                    value
                    222.5332637658962
                    standardDeviation
                    0
                    confidenceInterval
                    0
                    hasStandardDeviation
                    false
                    hasConfidenceInterval
                    false
                    unevenConfidenceInterval
                    plus
                    25.465670157266743
                    minus
                    24.05808869959236
                    hasUnevenConfidenceInterval
                    true
                    taskVersionId
                    0
                    evaluationDate
                    2026-11-11T19:56:36.2751281Z
                    hasBooleanResult
                    false
                    hasNumericResult
                    true
                    hasTaskVersionId
                    false
                    hasEvaluationDate
                    true
                    numericResultNullable
                    value
                    222.5332637658962
                    standardDeviation
                    0
                    confidenceInterval
                    0
                    hasStandardDeviation
                    false
                    hasConfidenceInterval
                    false
                    unevenConfidenceInterval
                    plus
                    25.465670157266743
                    minus
                    24.05808869959236
                    hasUnevenConfidenceInterval
                    true
                    hasNumericResultPublic
                    false
                    customAdditionalResults
                      hasNumericResultPrivate
                      false
                      task_id
                      33066
                      sort_order
                      DESCENDING
                      source_name
                      Score
                      display_type
                      counts
                      task_version
                      1
                      task_display_type
                      COUNTS
                      normalization_status
                      normalized
                    published_at
                    2026-06-04T13:40:48.173580300Z
                    candidate_class
                    modelVersion
                    methodology_url
                    https://www.kaggle.com/game-arena-faq
                    source_revision
                    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                    publisher_anchor
                    openai/o3-2025-04-16
                    game_arena_rating
                    Game-specific rating; compare within this game and version
                    metric_population
                    41
                    benchmark_evaluation
                    true
                    task_metadata_source
                    benchmarkVersion
                    candidate_metadata_source
                    benchmarkVersionMappings
                    evaluated at
                    2026-11-11T19:56:36.275Z
                    source model id
                    o3
                    confidence lower
                    198.47517506630385
                    confidence upper
                    247.99893392316295
                    Original result ↗

                    BFCL

                    50.5%

                    source id
                    bfcl
                    leaderboard name
                    BFCL Web Search
                    dimension
                    agentic
                    metric
                    Web-search accuracy
                    rank
                    26
                    Original result ↗

                    Kaggle Benchmarks

                    276.606

                    source id
                    kaggleBenchmarks
                    leaderboard name
                    Word Art · Score
                    dimension
                    general
                    metric
                    Word Art · Score
                    rank
                    27
                    aggregation scope
                    kaggle
                    true
                    status
                    published
                    license
                    Apache-2.0
                    direction
                    higherIsBetter
                    conditions
                    task
                    id
                    44050
                    name
                    Score
                    version
                    1
                    sort_order
                    DESCENDING
                    display_type
                    counts
                    benchmark
                    id
                    6710
                    name
                    Word Art
                    type
                    GAME
                    owner
                    kaggle
                    version
                    1
                    categories
                      version_id
                      6729
                      maintainer_tier
                      Kaggle-published
                      candidate
                      candidate_class
                      modelVersion
                      collection
                      kaggleGameArena
                      record_kind
                      aggregate
                      game_arena
                      true
                      attribution
                      Kaggle · kaggle · Word Art
                      license_url
                      https://www.apache.org/licenses/LICENSE-2.0
                      metric_kind
                      Word Art · Score
                      rank_method
                      competition rank among reported evaluations in identical conditions, including unlinked observations
                      source_file
                      6729/leaderboard.json
                      source_page
                      https://www.kaggle.com/benchmarks/kaggle/word-art/versions/1
                      measurements
                      • name
                        Score
                        unit
                        score
                        scale
                        1
                        metric
                        Word Art · Score
                        result
                        resultCase
                        numericResult
                        booleanResult
                        false
                        numericResult
                        value
                        276.60638141159643
                        standardDeviation
                        0
                        confidenceInterval
                        0
                        hasStandardDeviation
                        false
                        hasConfidenceInterval
                        false
                        unevenConfidenceInterval
                        plus
                        135.49198065673954
                        minus
                        128.7949872317654
                        hasUnevenConfidenceInterval
                        true
                        taskVersionId
                        0
                        evaluationDate
                        2026-11-11T20:02:33.211618Z
                        hasBooleanResult
                        false
                        hasNumericResult
                        true
                        hasTaskVersionId
                        false
                        hasEvaluationDate
                        true
                        numericResultNullable
                        value
                        276.60638141159643
                        standardDeviation
                        0
                        confidenceInterval
                        0
                        hasStandardDeviation
                        false
                        hasConfidenceInterval
                        false
                        unevenConfidenceInterval
                        plus
                        135.49198065673954
                        minus
                        128.7949872317654
                        hasUnevenConfidenceInterval
                        true
                        hasNumericResultPublic
                        false
                        customAdditionalResults
                          hasNumericResultPrivate
                          false
                          task_id
                          44050
                          sort_order
                          DESCENDING
                          source_name
                          Score
                          display_type
                          counts
                          task_version
                          1
                          task_display_type
                          COUNTS
                          normalization_status
                          normalized
                        published_at
                        2026-08-18T17:14:51.483049400Z
                        candidate_class
                        modelVersion
                        methodology_url
                        https://www.kaggle.com/game-arena-faq
                        source_revision
                        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                        publisher_anchor
                        openai/o3-2025-04-16
                        game_arena_rating
                        Game-specific rating; compare within this game and version
                        metric_population
                        39
                        benchmark_evaluation
                        true
                        task_metadata_source
                        benchmarkVersion
                        candidate_metadata_source
                        benchmarkVersionMappings
                        evaluated at
                        2026-11-11T20:02:33.211Z
                        source model id
                        o3
                        confidence lower
                        147.81139417983104
                        confidence upper
                        412.098362068336
                        Original result ↗

                        Kaggle Benchmarks

                        0.092

                        source id
                        kaggleBenchmarks
                        leaderboard name
                        SciCode Main With Background · SciCode Main with Background
                        dimension
                        coding
                        metric
                        SciCode Main With Background · SciCode Main with Background
                        rank
                        27
                        aggregation scope
                        kaggle
                        true
                        status
                        published
                        license
                        Apache-2.0
                        direction
                        higherIsBetter
                        conditions
                        task
                        id
                        40201
                        name
                        SciCode Main with Background
                        version
                        3
                        sort_order
                        DESCENDING
                        display_type
                        rawValues
                        benchmark
                        id
                        13
                        name
                        SciCode Main With Background
                        type
                        INDIVIDUAL
                        owner
                        open-benchmarks
                        version
                        1
                        categories
                        • general knowledge and reasoning
                        • coding
                        • science
                        version_id
                        13
                        maintainer_tier
                        Kaggle-maintained
                        maintenance_level
                        KAGGLE_MAINTAINED
                        candidate
                        candidate_class
                        modelVersion
                        record_kind
                        aggregate
                        attribution
                        Kaggle · open-benchmarks · SciCode Main With Background
                        license_url
                        https://www.apache.org/licenses/LICENSE-2.0
                        metric_kind
                        SciCode Main With Background · SciCode Main with Background
                        rank_method
                        competition rank among reported evaluations in identical conditions, including unlinked observations
                        source_file
                        13/leaderboard.json
                        source_page
                        https://www.kaggle.com/benchmarks/open-benchmarks/scicode-main-with-background/versions/1
                        measurements
                        • name
                          SciCode Main with Background
                          unit
                          score
                          scale
                          1
                          metric
                          SciCode Main With Background · SciCode Main with Background
                          result
                          resultCase
                          numericResult
                          booleanResult
                          false
                          numericResult
                          value
                          0.0923076923076923
                          standardDeviation
                          0
                          confidenceInterval
                          0.0703687257914088
                          hasStandardDeviation
                          false
                          hasConfidenceInterval
                          true
                          confidenceIntervalNullable
                          0.0703687257914088
                          hasUnevenConfidenceInterval
                          false
                          taskVersionId
                          0
                          evaluationDate
                          2025-05-22T00:00:00Z
                          hasBooleanResult
                          false
                          hasNumericResult
                          true
                          hasTaskVersionId
                          false
                          hasEvaluationDate
                          true
                          numericResultNullable
                          value
                          0.0923076923076923
                          standardDeviation
                          0
                          confidenceInterval
                          0.0703687257914088
                          hasStandardDeviation
                          false
                          hasConfidenceInterval
                          true
                          confidenceIntervalNullable
                          0.0703687257914088
                          hasUnevenConfidenceInterval
                          false
                          hasNumericResultPublic
                          false
                          customAdditionalResults
                            hasNumericResultPrivate
                            false
                            task_id
                            40201
                            task_url
                            /benchmarks/tasks/benchmarkler/scicode-background-scicode-main-with-background
                            sort_order
                            DESCENDING
                            source_name
                            SciCode Main with Background
                            display_type
                            rawValues
                            task_version
                            3
                            task_display_type
                            RAW_VALUES
                            normalization_status
                            normalized
                          published_at
                          2025-07-15T11:49:31.790Z
                          candidate_class
                          modelVersion
                          source_revision
                          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                          publisher_anchor
                          openai/o3-2025-04-16
                          metric_population
                          57
                          benchmark_evaluation
                          true
                          task_metadata_source
                          benchmarkVersion
                          candidate_metadata_source
                          benchmarkVersionMappings
                          evaluated at
                          2025-05-22T00:00:00.000Z
                          source model id
                          o3
                          confidence lower
                          0.021938966516283492
                          confidence upper
                          0.1626764180991011
                          Original result ↗

                          Kaggle Benchmarks

                          0%

                          source id
                          kaggleBenchmarks
                          leaderboard name
                          MultiLoKo Urdu · EM Urdu
                          dimension
                          reasoning
                          metric
                          MultiLoKo Urdu · EM Urdu
                          rank
                          27
                          aggregation scope
                          kaggle
                          true
                          status
                          published
                          license
                          Apache-2.0
                          direction
                          higherIsBetter
                          conditions
                          task
                          id
                          58
                          name
                          EM Urdu
                          version
                          1
                          sort_order
                          DESCENDING
                          display_type
                          percentages
                          benchmark
                          id
                          81
                          name
                          MultiLoKo Urdu
                          type
                          INDIVIDUAL
                          owner
                          metaresearch
                          version
                          1
                          categories
                          • multilingual and cross-lingual capabilities
                          • general knowledge and reasoning
                          version_id
                          67
                          maintainer_tier
                          Kaggle-maintained
                          maintenance_level
                          KAGGLE_MAINTAINED
                          candidate
                          candidate_class
                          modelVersion
                          record_kind
                          aggregate
                          attribution
                          Kaggle · metaresearch · MultiLoKo Urdu
                          license_url
                          https://www.apache.org/licenses/LICENSE-2.0
                          metric_kind
                          MultiLoKo Urdu · EM Urdu
                          rank_method
                          competition rank among reported evaluations in identical conditions, including unlinked observations
                          source_file
                          67/leaderboard.json
                          source_page
                          https://www.kaggle.com/benchmarks/metaresearch/multiloko-urdu/versions/1
                          measurements
                          • name
                            EM Urdu
                            unit
                            percent
                            scale
                            1
                            metric
                            MultiLoKo Urdu · EM Urdu
                            result
                            resultCase
                            numericResult
                            booleanResult
                            false
                            numericResult
                            value
                            0
                            standardDeviation
                            0
                            confidenceInterval
                            0
                            hasStandardDeviation
                            false
                            hasConfidenceInterval
                            true
                            confidenceIntervalNullable
                            0
                            hasUnevenConfidenceInterval
                            false
                            taskVersionId
                            0
                            evaluationDate
                            2025-06-16T00:00:00Z
                            hasBooleanResult
                            false
                            hasNumericResult
                            true
                            hasTaskVersionId
                            false
                            hasEvaluationDate
                            true
                            numericResultNullable
                            value
                            0
                            standardDeviation
                            0
                            confidenceInterval
                            0
                            hasStandardDeviation
                            false
                            hasConfidenceInterval
                            true
                            confidenceIntervalNullable
                            0
                            hasUnevenConfidenceInterval
                            false
                            hasNumericResultPublic
                            false
                            customAdditionalResults
                              hasNumericResultPrivate
                              false
                              task_id
                              58
                              sort_order
                              DESCENDING
                              source_name
                              EM Urdu
                              display_type
                              percentages
                              task_version
                              1
                              task_display_type
                              PERCENTAGES
                              normalization_status
                              normalized
                            published_at
                            2025-07-15T11:49:31.790Z
                            candidate_class
                            modelVersion
                            source_revision
                            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                            publisher_anchor
                            openai/o3-2025-04-16
                            metric_population
                            51
                            benchmark_evaluation
                            true
                            task_metadata_source
                            benchmarkVersion
                            candidate_metadata_source
                            benchmarkVersionMappings
                            evaluated at
                            2025-06-16T00:00:00.000Z
                            source model id
                            o3
                            confidence lower
                            0
                            confidence upper
                            0
                            Original result ↗

                            Kaggle Benchmarks

                            554.601

                            source id
                            kaggleBenchmarks
                            leaderboard name
                            Go · Score
                            dimension
                            general
                            metric
                            Go · Score
                            rank
                            28
                            aggregation scope
                            kaggle
                            true
                            status
                            published
                            license
                            Apache-2.0
                            direction
                            higherIsBetter
                            conditions
                            task
                            id
                            38797
                            name
                            Score
                            version
                            1
                            sort_order
                            DESCENDING
                            display_type
                            counts
                            benchmark
                            id
                            6234
                            name
                            Go
                            type
                            GAME
                            owner
                            kaggle
                            version
                            1
                            categories
                              version_id
                              6258
                              maintainer_tier
                              Kaggle-published
                              candidate
                              candidate_class
                              modelVersion
                              collection
                              kaggleGameArena
                              record_kind
                              aggregate
                              game_arena
                              true
                              attribution
                              Kaggle · kaggle · Go
                              license_url
                              https://www.apache.org/licenses/LICENSE-2.0
                              metric_kind
                              Go · Score
                              rank_method
                              competition rank among reported evaluations in identical conditions, including unlinked observations
                              source_file
                              6258/leaderboard.json
                              source_page
                              https://www.kaggle.com/benchmarks/kaggle/go/versions/1
                              measurements
                              • name
                                Score
                                unit
                                score
                                scale
                                1
                                metric
                                Go · Score
                                result
                                resultCase
                                numericResult
                                booleanResult
                                false
                                numericResult
                                value
                                554.6011567652583
                                standardDeviation
                                0
                                confidenceInterval
                                0
                                hasStandardDeviation
                                false
                                hasConfidenceInterval
                                false
                                unevenConfidenceInterval
                                plus
                                53.571610529952295
                                minus
                                73.6504478450662
                                hasUnevenConfidenceInterval
                                true
                                taskVersionId
                                0
                                evaluationDate
                                2026-11-12T17:13:46.623819Z
                                hasBooleanResult
                                false
                                hasNumericResult
                                true
                                hasTaskVersionId
                                false
                                hasEvaluationDate
                                true
                                numericResultNullable
                                value
                                554.6011567652583
                                standardDeviation
                                0
                                confidenceInterval
                                0
                                hasStandardDeviation
                                false
                                hasConfidenceInterval
                                false
                                unevenConfidenceInterval
                                plus
                                53.571610529952295
                                minus
                                73.6504478450662
                                hasUnevenConfidenceInterval
                                true
                                hasNumericResultPublic
                                false
                                customAdditionalResults
                                  hasNumericResultPrivate
                                  false
                                  task_id
                                  38797
                                  sort_order
                                  DESCENDING
                                  source_name
                                  Score
                                  display_type
                                  counts
                                  task_version
                                  1
                                  task_display_type
                                  COUNTS
                                  normalization_status
                                  normalized
                                published_at
                                2026-09-22T18:31:46.229644200Z
                                candidate_class
                                modelVersion
                                methodology_url
                                https://www.kaggle.com/game-arena-faq
                                source_revision
                                b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                publisher_anchor
                                openai/o3-2025-04-16
                                game_arena_rating
                                Game-specific rating; compare within this game and version
                                metric_population
                                39
                                benchmark_evaluation
                                true
                                task_metadata_source
                                benchmarkVersion
                                candidate_metadata_source
                                benchmarkVersionMappings
                                evaluated at
                                2026-11-12T17:13:46.623Z
                                source model id
                                o3
                                confidence lower
                                480.95070892019214
                                confidence upper
                                608.1727672952106
                                Original result ↗

                                Kaggle Benchmarks

                                293.37

                                source id
                                kaggleBenchmarks
                                leaderboard name
                                Four in a Row · Score
                                dimension
                                general
                                metric
                                Four in a Row · Score
                                rank
                                28
                                aggregation scope
                                kaggle
                                true
                                status
                                published
                                license
                                Apache-2.0
                                direction
                                higherIsBetter
                                conditions
                                task
                                id
                                2965
                                name
                                Score
                                version
                                1
                                sort_order
                                DESCENDING
                                display_type
                                counts
                                benchmark
                                id
                                862
                                name
                                Four in a Row
                                type
                                GAME
                                owner
                                kaggle
                                version
                                1
                                categories
                                  version_id
                                  844
                                  maintainer_tier
                                  Kaggle-published
                                  candidate
                                  candidate_class
                                  modelVersion
                                  collection
                                  kaggleGameArena
                                  record_kind
                                  aggregate
                                  game_arena
                                  true
                                  attribution
                                  Kaggle · kaggle · Four in a Row
                                  license_url
                                  https://www.apache.org/licenses/LICENSE-2.0
                                  metric_kind
                                  Four in a Row · Score
                                  rank_method
                                  competition rank among reported evaluations in identical conditions, including unlinked observations
                                  source_file
                                  844/leaderboard.json
                                  source_page
                                  https://www.kaggle.com/benchmarks/kaggle/four-in-a-row/versions/1
                                  measurements
                                  • name
                                    Score
                                    unit
                                    score
                                    scale
                                    1
                                    metric
                                    Four in a Row · Score
                                    result
                                    resultCase
                                    numericResult
                                    booleanResult
                                    false
                                    numericResult
                                    value
                                    293.36994581347494
                                    standardDeviation
                                    0
                                    confidenceInterval
                                    0
                                    hasStandardDeviation
                                    false
                                    hasConfidenceInterval
                                    false
                                    unevenConfidenceInterval
                                    plus
                                    27.861996569074563
                                    minus
                                    35.21686982664619
                                    hasUnevenConfidenceInterval
                                    true
                                    taskVersionId
                                    0
                                    evaluationDate
                                    2026-11-11T20:03:06.5646949Z
                                    hasBooleanResult
                                    false
                                    hasNumericResult
                                    true
                                    hasTaskVersionId
                                    false
                                    hasEvaluationDate
                                    true
                                    numericResultNullable
                                    value
                                    293.36994581347494
                                    standardDeviation
                                    0
                                    confidenceInterval
                                    0
                                    hasStandardDeviation
                                    false
                                    hasConfidenceInterval
                                    false
                                    unevenConfidenceInterval
                                    plus
                                    27.861996569074563
                                    minus
                                    35.21686982664619
                                    hasUnevenConfidenceInterval
                                    true
                                    hasNumericResultPublic
                                    false
                                    customAdditionalResults
                                      hasNumericResultPrivate
                                      false
                                      task_id
                                      2965
                                      sort_order
                                      DESCENDING
                                      source_name
                                      Score
                                      display_type
                                      counts
                                      task_version
                                      1
                                      task_display_type
                                      COUNTS
                                      normalization_status
                                      normalized
                                    published_at
                                    2026-02-19T18:00:05.809078100Z
                                    candidate_class
                                    modelVersion
                                    methodology_url
                                    https://www.kaggle.com/game-arena-faq
                                    source_revision
                                    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                    publisher_anchor
                                    openai/o3-2025-04-16
                                    game_arena_rating
                                    Game-specific rating; compare within this game and version
                                    metric_population
                                    43
                                    benchmark_evaluation
                                    true
                                    task_metadata_source
                                    benchmarkVersion
                                    candidate_metadata_source
                                    benchmarkVersionMappings
                                    evaluated at
                                    2026-11-11T20:03:06.564Z
                                    source model id
                                    o3
                                    confidence lower
                                    258.15307598682875
                                    confidence upper
                                    321.2319423825495
                                    Original result ↗

                                    Kaggle Benchmarks

                                    292.594

                                    source id
                                    kaggleBenchmarks
                                    leaderboard name
                                    Lines of Action · Score
                                    dimension
                                    general
                                    metric
                                    Lines of Action · Score
                                    rank
                                    28
                                    aggregation scope
                                    kaggle
                                    true
                                    status
                                    published
                                    license
                                    Apache-2.0
                                    direction
                                    higherIsBetter
                                    conditions
                                    task
                                    id
                                    34467
                                    name
                                    Score
                                    version
                                    1
                                    sort_order
                                    DESCENDING
                                    display_type
                                    counts
                                    benchmark
                                    id
                                    5814
                                    name
                                    Lines of Action
                                    type
                                    GAME
                                    owner
                                    kaggle
                                    version
                                    1
                                    categories
                                      version_id
                                      5837
                                      maintainer_tier
                                      Kaggle-published
                                      candidate
                                      candidate_class
                                      modelVersion
                                      collection
                                      kaggleGameArena
                                      record_kind
                                      aggregate
                                      game_arena
                                      true
                                      attribution
                                      Kaggle · kaggle · Lines of Action
                                      license_url
                                      https://www.apache.org/licenses/LICENSE-2.0
                                      metric_kind
                                      Lines of Action · Score
                                      rank_method
                                      competition rank among reported evaluations in identical conditions, including unlinked observations
                                      source_file
                                      5837/leaderboard.json
                                      source_page
                                      https://www.kaggle.com/benchmarks/kaggle/lines-of-action/versions/1
                                      measurements
                                      • name
                                        Score
                                        unit
                                        score
                                        scale
                                        1
                                        metric
                                        Lines of Action · Score
                                        result
                                        resultCase
                                        numericResult
                                        booleanResult
                                        false
                                        numericResult
                                        value
                                        292.5936737794559
                                        standardDeviation
                                        0
                                        confidenceInterval
                                        0
                                        hasStandardDeviation
                                        false
                                        hasConfidenceInterval
                                        false
                                        unevenConfidenceInterval
                                        plus
                                        45.70529830623434
                                        minus
                                        61.238266806274225
                                        hasUnevenConfidenceInterval
                                        true
                                        taskVersionId
                                        0
                                        evaluationDate
                                        2026-11-11T19:57:59.1350406Z
                                        hasBooleanResult
                                        false
                                        hasNumericResult
                                        true
                                        hasTaskVersionId
                                        false
                                        hasEvaluationDate
                                        true
                                        numericResultNullable
                                        value
                                        292.5936737794559
                                        standardDeviation
                                        0
                                        confidenceInterval
                                        0
                                        hasStandardDeviation
                                        false
                                        hasConfidenceInterval
                                        false
                                        unevenConfidenceInterval
                                        plus
                                        45.70529830623434
                                        minus
                                        61.238266806274225
                                        hasUnevenConfidenceInterval
                                        true
                                        hasNumericResultPublic
                                        false
                                        customAdditionalResults
                                          hasNumericResultPrivate
                                          false
                                          task_id
                                          34467
                                          sort_order
                                          DESCENDING
                                          source_name
                                          Score
                                          display_type
                                          counts
                                          task_version
                                          1
                                          task_display_type
                                          COUNTS
                                          normalization_status
                                          normalized
                                        published_at
                                        2026-07-16T23:03:18.619026400Z
                                        candidate_class
                                        modelVersion
                                        methodology_url
                                        https://www.kaggle.com/game-arena-faq
                                        source_revision
                                        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                        publisher_anchor
                                        openai/o3-2025-04-16
                                        game_arena_rating
                                        Game-specific rating; compare within this game and version
                                        metric_population
                                        39
                                        benchmark_evaluation
                                        true
                                        task_metadata_source
                                        benchmarkVersion
                                        candidate_metadata_source
                                        benchmarkVersionMappings
                                        evaluated at
                                        2026-11-11T19:57:59.135Z
                                        source model id
                                        o3
                                        confidence lower
                                        231.3554069731817
                                        confidence upper
                                        338.29897208569025
                                        Original result ↗

                                        BFCL

                                        86.13%

                                        source id
                                        bfcl
                                        leaderboard name
                                        BFCL Irrelevance Detection
                                        dimension
                                        toolUse
                                        metric
                                        Irrelevance detection
                                        rank
                                        28
                                        Original result ↗

                                        Kaggle Benchmarks

                                        143.767

                                        source id
                                        kaggleBenchmarks
                                        leaderboard name
                                        Clobber · Score
                                        dimension
                                        general
                                        metric
                                        Clobber · Score
                                        rank
                                        31
                                        aggregation scope
                                        kaggle
                                        true
                                        status
                                        published
                                        license
                                        Apache-2.0
                                        direction
                                        higherIsBetter
                                        conditions
                                        task
                                        id
                                        34472
                                        name
                                        Score
                                        version
                                        1
                                        sort_order
                                        DESCENDING
                                        display_type
                                        counts
                                        benchmark
                                        id
                                        5816
                                        name
                                        Clobber
                                        type
                                        GAME
                                        owner
                                        kaggle
                                        version
                                        1
                                        categories
                                          version_id
                                          5839
                                          maintainer_tier
                                          Kaggle-published
                                          candidate
                                          candidate_class
                                          modelVersion
                                          collection
                                          kaggleGameArena
                                          record_kind
                                          aggregate
                                          game_arena
                                          true
                                          attribution
                                          Kaggle · kaggle · Clobber
                                          license_url
                                          https://www.apache.org/licenses/LICENSE-2.0
                                          metric_kind
                                          Clobber · Score
                                          rank_method
                                          competition rank among reported evaluations in identical conditions, including unlinked observations
                                          source_file
                                          5839/leaderboard.json
                                          source_page
                                          https://www.kaggle.com/benchmarks/kaggle/clobber/versions/1
                                          measurements
                                          • name
                                            Score
                                            unit
                                            score
                                            scale
                                            1
                                            metric
                                            Clobber · Score
                                            result
                                            resultCase
                                            numericResult
                                            booleanResult
                                            false
                                            numericResult
                                            value
                                            143.76723166045315
                                            standardDeviation
                                            0
                                            confidenceInterval
                                            0
                                            hasStandardDeviation
                                            false
                                            hasConfidenceInterval
                                            false
                                            unevenConfidenceInterval
                                            plus
                                            38.26638387488663
                                            minus
                                            41.07049556980107
                                            hasUnevenConfidenceInterval
                                            true
                                            taskVersionId
                                            0
                                            evaluationDate
                                            2026-11-11T19:57:33.0562958Z
                                            hasBooleanResult
                                            false
                                            hasNumericResult
                                            true
                                            hasTaskVersionId
                                            false
                                            hasEvaluationDate
                                            true
                                            numericResultNullable
                                            value
                                            143.76723166045315
                                            standardDeviation
                                            0
                                            confidenceInterval
                                            0
                                            hasStandardDeviation
                                            false
                                            hasConfidenceInterval
                                            false
                                            unevenConfidenceInterval
                                            plus
                                            38.26638387488663
                                            minus
                                            41.07049556980107
                                            hasUnevenConfidenceInterval
                                            true
                                            hasNumericResultPublic
                                            false
                                            customAdditionalResults
                                              hasNumericResultPrivate
                                              false
                                              task_id
                                              34472
                                              sort_order
                                              DESCENDING
                                              source_name
                                              Score
                                              display_type
                                              counts
                                              task_version
                                              1
                                              task_display_type
                                              COUNTS
                                              normalization_status
                                              normalized
                                            published_at
                                            2026-06-25T16:16:58.883878300Z
                                            candidate_class
                                            modelVersion
                                            methodology_url
                                            https://www.kaggle.com/game-arena-faq
                                            source_revision
                                            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                            publisher_anchor
                                            openai/o3-2025-04-16
                                            game_arena_rating
                                            Game-specific rating; compare within this game and version
                                            metric_population
                                            39
                                            benchmark_evaluation
                                            true
                                            task_metadata_source
                                            benchmarkVersion
                                            candidate_metadata_source
                                            benchmarkVersionMappings
                                            evaluated at
                                            2026-11-11T19:57:33.056Z
                                            source model id
                                            o3
                                            confidence lower
                                            102.69673609065208
                                            confidence upper
                                            182.03361553533978
                                            Original result ↗

                                            Kaggle Benchmarks

                                            9.231%

                                            source id
                                            kaggleBenchmarks
                                            leaderboard name
                                            SciCode · SciCode Main With Background
                                            dimension
                                            coding
                                            metric
                                            SciCode · SciCode Main With Background
                                            rank
                                            31
                                            aggregation scope
                                            kaggle
                                            true
                                            status
                                            published
                                            license
                                            Apache-2.0
                                            direction
                                            higherIsBetter
                                            conditions
                                            task
                                            id
                                            40201
                                            name
                                            SciCode Main With Background
                                            version
                                            3
                                            sort_order
                                            DESCENDING
                                            display_type
                                            percentages
                                            benchmark
                                            id
                                            16
                                            name
                                            SciCode
                                            type
                                            SUITE
                                            owner
                                            open-benchmarks
                                            version
                                            1
                                            categories
                                            • general knowledge and reasoning
                                            • coding
                                            • science
                                            version_id
                                            16
                                            maintainer_tier
                                            Kaggle-maintained
                                            maintenance_level
                                            KAGGLE_MAINTAINED
                                            candidate
                                            candidate_class
                                            modelVersion
                                            record_kind
                                            aggregate
                                            attribution
                                            Kaggle · open-benchmarks · SciCode
                                            license_url
                                            https://www.apache.org/licenses/LICENSE-2.0
                                            metric_kind
                                            SciCode · SciCode Main With Background
                                            rank_method
                                            competition rank among reported evaluations in identical conditions, including unlinked observations
                                            source_file
                                            16/leaderboard.json
                                            source_page
                                            https://www.kaggle.com/benchmarks/open-benchmarks/scicode/versions/1
                                            measurements
                                            • name
                                              SciCode Main With Background
                                              unit
                                              percent
                                              scale
                                              1
                                              metric
                                              SciCode · SciCode Main With Background
                                              result
                                              resultCase
                                              numericResult
                                              booleanResult
                                              false
                                              numericResult
                                              value
                                              0.0923076923076923
                                              standardDeviation
                                              0
                                              confidenceInterval
                                              0.0703687257914088
                                              hasStandardDeviation
                                              false
                                              hasConfidenceInterval
                                              true
                                              confidenceIntervalNullable
                                              0.0703687257914088
                                              hasUnevenConfidenceInterval
                                              false
                                              taskVersionId
                                              0
                                              evaluationDate
                                              2025-05-22T00:00:00Z
                                              hasBooleanResult
                                              false
                                              hasNumericResult
                                              true
                                              hasTaskVersionId
                                              false
                                              hasEvaluationDate
                                              true
                                              numericResultNullable
                                              value
                                              0.0923076923076923
                                              standardDeviation
                                              0
                                              confidenceInterval
                                              0.0703687257914088
                                              hasStandardDeviation
                                              false
                                              hasConfidenceInterval
                                              true
                                              confidenceIntervalNullable
                                              0.0703687257914088
                                              hasUnevenConfidenceInterval
                                              false
                                              hasNumericResultPublic
                                              false
                                              customAdditionalResults
                                                hasNumericResultPrivate
                                                false
                                                task_id
                                                40201
                                                task_url
                                                /benchmarks/open-benchmarks/scicode-main-with-background/versions/1
                                                sort_order
                                                DESCENDING
                                                source_name
                                                SciCode Main With Background
                                                display_type
                                                percentages
                                                task_version
                                                3
                                                task_display_type
                                                RAW_VALUES
                                                normalization_status
                                                normalized
                                              published_at
                                              2025-10-21T20:29:26.251536900Z
                                              candidate_class
                                              modelVersion
                                              source_revision
                                              b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                              publisher_anchor
                                              openai/o3-2025-04-16
                                              metric_population
                                              62
                                              benchmark_evaluation
                                              true
                                              task_metadata_source
                                              benchmarkVersion
                                              candidate_metadata_source
                                              benchmarkVersionMappings
                                              evaluated at
                                              2025-05-22T00:00:00.000Z
                                              source model id
                                              o3
                                              confidence lower
                                              0.021938966516283492
                                              confidence upper
                                              0.1626764180991011
                                              Original result ↗

                                              Kaggle Benchmarks

                                              599.092

                                              source id
                                              kaggleBenchmarks
                                              leaderboard name
                                              Dots and Boxes · Score
                                              dimension
                                              general
                                              metric
                                              Dots and Boxes · Score
                                              rank
                                              33
                                              aggregation scope
                                              kaggle
                                              true
                                              status
                                              published
                                              license
                                              Apache-2.0
                                              direction
                                              higherIsBetter
                                              conditions
                                              task
                                              id
                                              36964
                                              name
                                              Score
                                              version
                                              1
                                              sort_order
                                              DESCENDING
                                              display_type
                                              counts
                                              benchmark
                                              id
                                              6023
                                              name
                                              Dots and Boxes
                                              type
                                              GAME
                                              owner
                                              kaggle
                                              version
                                              1
                                              categories
                                                version_id
                                                6046
                                                maintainer_tier
                                                Kaggle-published
                                                candidate
                                                candidate_class
                                                modelVersion
                                                collection
                                                kaggleGameArena
                                                record_kind
                                                aggregate
                                                game_arena
                                                true
                                                attribution
                                                Kaggle · kaggle · Dots and Boxes
                                                license_url
                                                https://www.apache.org/licenses/LICENSE-2.0
                                                metric_kind
                                                Dots and Boxes · Score
                                                rank_method
                                                competition rank among reported evaluations in identical conditions, including unlinked observations
                                                source_file
                                                6046/leaderboard.json
                                                source_page
                                                https://www.kaggle.com/benchmarks/kaggle/dots-and-boxes/versions/1
                                                measurements
                                                • name
                                                  Score
                                                  unit
                                                  score
                                                  scale
                                                  1
                                                  metric
                                                  Dots and Boxes · Score
                                                  result
                                                  resultCase
                                                  numericResult
                                                  booleanResult
                                                  false
                                                  numericResult
                                                  value
                                                  599.0918630140468
                                                  standardDeviation
                                                  0
                                                  confidenceInterval
                                                  0
                                                  hasStandardDeviation
                                                  false
                                                  hasConfidenceInterval
                                                  false
                                                  unevenConfidenceInterval
                                                  plus
                                                  33.858217825332986
                                                  minus
                                                  41.63358666183876
                                                  hasUnevenConfidenceInterval
                                                  true
                                                  taskVersionId
                                                  0
                                                  evaluationDate
                                                  2026-11-11T19:59:23.3393358Z
                                                  hasBooleanResult
                                                  false
                                                  hasNumericResult
                                                  true
                                                  hasTaskVersionId
                                                  false
                                                  hasEvaluationDate
                                                  true
                                                  numericResultNullable
                                                  value
                                                  599.0918630140468
                                                  standardDeviation
                                                  0
                                                  confidenceInterval
                                                  0
                                                  hasStandardDeviation
                                                  false
                                                  hasConfidenceInterval
                                                  false
                                                  unevenConfidenceInterval
                                                  plus
                                                  33.858217825332986
                                                  minus
                                                  41.63358666183876
                                                  hasUnevenConfidenceInterval
                                                  true
                                                  hasNumericResultPublic
                                                  false
                                                  customAdditionalResults
                                                    hasNumericResultPrivate
                                                    false
                                                    task_id
                                                    36964
                                                    sort_order
                                                    DESCENDING
                                                    source_name
                                                    Score
                                                    display_type
                                                    counts
                                                    task_version
                                                    1
                                                    task_display_type
                                                    COUNTS
                                                    normalization_status
                                                    normalized
                                                  published_at
                                                  2026-06-17T18:07:42.213532800Z
                                                  candidate_class
                                                  modelVersion
                                                  methodology_url
                                                  https://www.kaggle.com/game-arena-faq
                                                  source_revision
                                                  b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                                  publisher_anchor
                                                  openai/o3-2025-04-16
                                                  game_arena_rating
                                                  Game-specific rating; compare within this game and version
                                                  metric_population
                                                  40
                                                  benchmark_evaluation
                                                  true
                                                  task_metadata_source
                                                  benchmarkVersion
                                                  candidate_metadata_source
                                                  benchmarkVersionMappings
                                                  evaluated at
                                                  2026-11-11T19:59:23.339Z
                                                  source model id
                                                  o3
                                                  confidence lower
                                                  557.458276352208
                                                  confidence upper
                                                  632.9500808393798
                                                  Original result ↗

                                                  Kaggle Benchmarks

                                                  154.195

                                                  source id
                                                  kaggleBenchmarks
                                                  leaderboard name
                                                  Coin Game · Score
                                                  dimension
                                                  general
                                                  metric
                                                  Coin Game · Score
                                                  rank
                                                  37
                                                  aggregation scope
                                                  kaggle
                                                  true
                                                  status
                                                  published
                                                  license
                                                  Apache-2.0
                                                  direction
                                                  higherIsBetter
                                                  conditions
                                                  task
                                                  id
                                                  34561
                                                  name
                                                  Score
                                                  version
                                                  1
                                                  sort_order
                                                  DESCENDING
                                                  display_type
                                                  counts
                                                  benchmark
                                                  id
                                                  5821
                                                  name
                                                  Coin Game
                                                  type
                                                  GAME
                                                  owner
                                                  kaggle
                                                  version
                                                  1
                                                  categories
                                                    version_id
                                                    5844
                                                    maintainer_tier
                                                    Kaggle-published
                                                    candidate
                                                    candidate_class
                                                    modelVersion
                                                    collection
                                                    kaggleGameArena
                                                    record_kind
                                                    aggregate
                                                    game_arena
                                                    true
                                                    attribution
                                                    Kaggle · kaggle · Coin Game
                                                    license_url
                                                    https://www.apache.org/licenses/LICENSE-2.0
                                                    metric_kind
                                                    Coin Game · Score
                                                    rank_method
                                                    competition rank among reported evaluations in identical conditions, including unlinked observations
                                                    source_file
                                                    5844/leaderboard.json
                                                    source_page
                                                    https://www.kaggle.com/benchmarks/kaggle/coin-game/versions/1
                                                    measurements
                                                    • name
                                                      Score
                                                      unit
                                                      score
                                                      scale
                                                      1
                                                      metric
                                                      Coin Game · Score
                                                      result
                                                      resultCase
                                                      numericResult
                                                      booleanResult
                                                      false
                                                      numericResult
                                                      value
                                                      154.1948636129182
                                                      standardDeviation
                                                      0
                                                      confidenceInterval
                                                      0
                                                      hasStandardDeviation
                                                      false
                                                      hasConfidenceInterval
                                                      false
                                                      unevenConfidenceInterval
                                                      plus
                                                      29.265849300068595
                                                      minus
                                                      25.160096254565616
                                                      hasUnevenConfidenceInterval
                                                      true
                                                      taskVersionId
                                                      0
                                                      evaluationDate
                                                      2026-11-11T19:58:27.1740754Z
                                                      hasBooleanResult
                                                      false
                                                      hasNumericResult
                                                      true
                                                      hasTaskVersionId
                                                      false
                                                      hasEvaluationDate
                                                      true
                                                      numericResultNullable
                                                      value
                                                      154.1948636129182
                                                      standardDeviation
                                                      0
                                                      confidenceInterval
                                                      0
                                                      hasStandardDeviation
                                                      false
                                                      hasConfidenceInterval
                                                      false
                                                      unevenConfidenceInterval
                                                      plus
                                                      29.265849300068595
                                                      minus
                                                      25.160096254565616
                                                      hasUnevenConfidenceInterval
                                                      true
                                                      hasNumericResultPublic
                                                      false
                                                      customAdditionalResults
                                                        hasNumericResultPrivate
                                                        false
                                                        task_id
                                                        34561
                                                        sort_order
                                                        DESCENDING
                                                        source_name
                                                        Score
                                                        display_type
                                                        counts
                                                        task_version
                                                        1
                                                        task_display_type
                                                        COUNTS
                                                        normalization_status
                                                        normalized
                                                      published_at
                                                      2026-07-08T17:58:41.243600500Z
                                                      candidate_class
                                                      modelVersion
                                                      methodology_url
                                                      https://www.kaggle.com/game-arena-faq
                                                      source_revision
                                                      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                                      publisher_anchor
                                                      openai/o3-2025-04-16
                                                      game_arena_rating
                                                      Game-specific rating; compare within this game and version
                                                      metric_population
                                                      39
                                                      benchmark_evaluation
                                                      true
                                                      task_metadata_source
                                                      benchmarkVersion
                                                      candidate_metadata_source
                                                      benchmarkVersionMappings
                                                      evaluated at
                                                      2026-11-11T19:58:27.174Z
                                                      source model id
                                                      o3
                                                      confidence lower
                                                      129.03476735835258
                                                      confidence upper
                                                      183.4607129129868
                                                      Original result ↗

                                                      BFCL

                                                      83.98%

                                                      source id
                                                      bfcl
                                                      leaderboard name
                                                      BFCL Irrelevance Detection
                                                      dimension
                                                      toolUse
                                                      metric
                                                      Irrelevance detection
                                                      rank
                                                      39
                                                      Original result ↗

                                                      Published prices

                                                      1–8 / 8

                                                      openai/o3 ↗

                                                      $0.50

                                                      Inference provider
                                                      openRouter
                                                      Platform
                                                      OpenRouter
                                                      Unit
                                                      cache_read_tokens_per_million
                                                      Currency
                                                      USD
                                                      Billing conditions

                                                      openai/o3 ↗

                                                      $2.00

                                                      Inference provider
                                                      openRouter
                                                      Platform
                                                      OpenRouter
                                                      Unit
                                                      input_tokens_per_million
                                                      Currency
                                                      USD
                                                      Billing conditions

                                                      openai/o3 ↗

                                                      $8.00

                                                      Inference provider
                                                      openRouter
                                                      Platform
                                                      OpenRouter
                                                      Unit
                                                      output_tokens_per_million
                                                      Currency
                                                      USD
                                                      Billing conditions

                                                      openai/o3 ↗

                                                      $0.01

                                                      Inference provider
                                                      openRouter
                                                      Platform
                                                      OpenRouter
                                                      Unit
                                                      web_search
                                                      Currency
                                                      USD
                                                      Billing conditions

                                                      openai/o3:batch ↗

                                                      $0.25

                                                      Inference provider
                                                      openRouter
                                                      Platform
                                                      OpenRouter
                                                      Unit
                                                      cache_read_tokens_per_million
                                                      Currency
                                                      USD
                                                      Billing conditions
                                                      mode
                                                      batch

                                                      openai/o3:batch ↗

                                                      $1.00

                                                      Inference provider
                                                      openRouter
                                                      Platform
                                                      OpenRouter
                                                      Unit
                                                      input_tokens_per_million
                                                      Currency
                                                      USD
                                                      Billing conditions
                                                      mode
                                                      batch

                                                      openai/o3:batch ↗

                                                      $4.00

                                                      Inference provider
                                                      openRouter
                                                      Platform
                                                      OpenRouter
                                                      Unit
                                                      output_tokens_per_million
                                                      Currency
                                                      USD
                                                      Billing conditions
                                                      mode
                                                      batch

                                                      openai/o3:batch ↗

                                                      $0.01

                                                      Inference provider
                                                      openRouter
                                                      Platform
                                                      OpenRouter
                                                      Unit
                                                      web_search
                                                      Currency
                                                      USD
                                                      Billing conditions
                                                      mode
                                                      batch

                                                      Provider specifications & access

                                                      1–1 / 1

                                                      OpenAI o3 · azure-openai v2025-04-16 ↗

                                                      source model id
                                                      azure-openai/o3@2025-04-16
                                                      hosting
                                                      foundry-managed-compute
                                                      platforms
                                                      • Microsoft Foundry
                                                      license
                                                      custom
                                                      context length
                                                      200000
                                                      max output tokens
                                                      100000
                                                      architecture
                                                      input_modalities
                                                      • text
                                                      • image
                                                      output_modalities
                                                      • text
                                                      capabilities
                                                      • reasoning
                                                      • toolCalling

                                                      Safety assessments

                                                      1–1 / 1

                                                      Foundation Model Transparency Index 2025 ↗

                                                      35 / 100

                                                      source id
                                                      stanfordFMTI
                                                      canonical model id
                                                      openai/o3-2025-04-16
                                                      taxonomy
                                                      transparency
                                                      hazard category
                                                      transparency
                                                      hazard name
                                                      Organization transparency
                                                      policy area
                                                      OpenAI
                                                      assessment type
                                                      organization-level disclosure
                                                      metric
                                                      disclosure_score
                                                      result label
                                                      35 / 100
                                                      sample size
                                                      100
                                                      updated at
                                                      2026-11-09T18:08:05.227Z

                                                      Source specification observations

                                                      1–9 / 9

                                                      azureFoundry ↗

                                                      field
                                                      context_length
                                                      value
                                                      200000
                                                      normalized value
                                                      200000
                                                      scope
                                                      source-model
                                                      source id
                                                      azureFoundry
                                                      confidence
                                                      1

                                                      openRouter ↗

                                                      field
                                                      context_length
                                                      value
                                                      200000
                                                      normalized value
                                                      200000
                                                      scope
                                                      source-model
                                                      source id
                                                      openRouter
                                                      confidence
                                                      1

                                                      openRouter ↗

                                                      field
                                                      context_length
                                                      value
                                                      200000
                                                      normalized value
                                                      200000
                                                      scope
                                                      source-model
                                                      source id
                                                      openRouter
                                                      confidence
                                                      1

                                                      openRouter ↗

                                                      field
                                                      knowledge_cutoff
                                                      reported field
                                                      knowledge_cutoff
                                                      value
                                                      2024-06-30
                                                      normalized value
                                                      2024-06-30
                                                      precision
                                                      day
                                                      scope
                                                      source-model
                                                      source id
                                                      openRouter
                                                      confidence
                                                      1

                                                      openRouter ↗

                                                      field
                                                      knowledge_cutoff
                                                      reported field
                                                      knowledge_cutoff
                                                      value
                                                      2024-06-30
                                                      normalized value
                                                      2024-06-30
                                                      precision
                                                      day
                                                      scope
                                                      source-model
                                                      source id
                                                      openRouter
                                                      confidence
                                                      1

                                                      azureFoundry ↗

                                                      field
                                                      max_output_tokens
                                                      value
                                                      100000
                                                      normalized value
                                                      100000
                                                      scope
                                                      source-model
                                                      source id
                                                      azureFoundry
                                                      confidence
                                                      1

                                                      openRouter ↗

                                                      field
                                                      max_output_tokens
                                                      value
                                                      100000
                                                      normalized value
                                                      100000
                                                      scope
                                                      source-model
                                                      source id
                                                      openRouter
                                                      confidence
                                                      1

                                                      openRouter ↗

                                                      field
                                                      max_output_tokens
                                                      value
                                                      100000
                                                      normalized value
                                                      100000
                                                      scope
                                                      source-model
                                                      source id
                                                      openRouter
                                                      confidence
                                                      1

                                                      azureFoundry ↗

                                                      field
                                                      knowledge_cutoff
                                                      reported field
                                                      training_data_cutoff
                                                      value
                                                      May 2024
                                                      normalized value
                                                      2024-05
                                                      precision
                                                      month
                                                      scope
                                                      source-model
                                                      source id
                                                      azureFoundry
                                                      confidence
                                                      1

                                                      Original sources

                                                      1–13 / 13

                                                      OpenAI · Model reference ↗

                                                      Microsoft Foundry · Model page ↗

                                                      OpenRouter · Model page ↗

                                                      OpenRouter · Model page ↗

                                                      Microsoft Foundry · Model documentation ↗

                                                      GitHub · Catalog ↗

                                                      GitHub · Catalog ↗

                                                      Kaggle · Catalog ↗

                                                      LMArena · Catalog ↗

                                                      Microsoft Foundry · Catalog ↗

                                                      OpenRouter · Catalog ↗

                                                      gorilla.cs.berkeley.edu · Catalog ↗

                                                      www.swebench.com · Catalog ↗

                                                      Source links

                                                      1–10 / 10

                                                      Explore more models

                                                      Models

                                                      Loading
                                                      EVIDENCE / COMPARISON

                                                      Benchmarks

                                                      Explore every benchmark

                                                      Loading

                                                      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                                                      Install TensorTrace