OpenAI: o3

o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Us…

openai/o3-2025-04-16

o3 is a well-rounded and powerful model across domains. It sets a new standard for math, science, coding, and visual reasoning tasks. It also excels at technical writing and instruction-following. Use it to think through multi-step problems that involve analysis across text, code, and images.

Specifications & capabilities

Input
image, text, file
Output
text
Context window · tokens
200000
Capabilities
fileInput, multimodal, promptCaching, reasoning, structuredOutputs, toolCalling, visionInput
Description source ↗

Evaluations

21–40 / 328

LLM Stats · GitHub archive

0.497

source id
llmStatsArchive
leaderboard name
BrowseComp · Normalized (reported)
dimension
general
metric
BrowseComp · Normalized (reported)
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
BrowseComp
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • search
created_at
2025-07-28T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2504.12516
updated_at
2025-07-28T00:00:00.000000+00:00
description
BrowseComp is a benchmark comprising 1,266 questions that challenge AI agents to persistently navigate the internet in search of hard-to-find, entangled information. The benchmark measures agents' ability to exercise persistence in information gathering, demonstrate creativity in web navigation, and find concise, verifiable answers. Despite the difficulty of the questions, BrowseComp is simple and easy-to-use, as predicted answers are short and easily verifiable against reference answers.
benchmark_id
browsecomp
multilingual
false
analysis_method
accuracy (with python + browsing)
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
BrowseComp · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o3-2025-04-16/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/introducing-o3-and-o4-mini/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o3-2025-04-16
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o3-2025-04-16
Original result ↗

LLM Stats · GitHub archive

0.497

source id
llmStatsArchive
leaderboard name
BrowseComp · Raw
dimension
general
metric
BrowseComp · Raw
rank
2
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
BrowseComp
language
en
modality
text
verified
false
max_score
1
categories
  • reasoning
  • search
created_at
2025-07-28T00:00:00.000000+00:00
paper_link
https://arxiv.org/abs/2504.12516
updated_at
2025-07-28T00:00:00.000000+00:00
description
BrowseComp is a benchmark comprising 1,266 questions that challenge AI agents to persistently navigate the internet in search of hard-to-find, entangled information. The benchmark measures agents' ability to exercise persistence in information gathering, demonstrate creativity in web navigation, and find concise, verifiable answers. Despite the difficulty of the questions, BrowseComp is simple and easy-to-use, as predicted answers are short and easily verifiable against reference answers.
benchmark_id
browsecomp
multilingual
false
analysis_method
accuracy (with python + browsing)
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
BrowseComp · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/openai/models/o3-2025-04-16/benchmarks.json
archive_kind
llmStats
evidence_url
https://openai.com/index/introducing-o3-and-o4-mini/
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
openai/o3-2025-04-16
metric_population
2
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
o3-2025-04-16
Original result ↗

Kaggle Benchmarks

46.8%

source id
kaggleBenchmarks
leaderboard name
MultiLoKo Korean · EM Korean
dimension
reasoning
metric
MultiLoKo Korean · EM Korean
rank
2
aggregation scope
kaggle
true
status
published
license
Apache-2.0
direction
higherIsBetter
conditions
task
id
44
name
EM Korean
version
1
sort_order
DESCENDING
display_type
percentages
benchmark
id
67
name
MultiLoKo Korean
type
INDIVIDUAL
owner
metaresearch
version
1
categories
  • multilingual and cross-lingual capabilities
  • general knowledge and reasoning
version_id
53
maintainer_tier
Kaggle-maintained
maintenance_level
KAGGLE_MAINTAINED
candidate
candidate_class
modelVersion
record_kind
aggregate
attribution
Kaggle · metaresearch · MultiLoKo Korean
license_url
https://www.apache.org/licenses/LICENSE-2.0
metric_kind
MultiLoKo Korean · EM Korean
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
53/leaderboard.json
source_page
https://www.kaggle.com/benchmarks/metaresearch/multiloko-korean/versions/1
measurements
  • name
    EM Korean
    unit
    percent
    scale
    1
    metric
    MultiLoKo Korean · EM Korean
    result
    resultCase
    numericResult
    booleanResult
    false
    numericResult
    value
    0.468
    standardDeviation
    0
    confidenceInterval
    0.0618524389605176
    hasStandardDeviation
    false
    hasConfidenceInterval
    true
    confidenceIntervalNullable
    0.0618524389605176
    hasUnevenConfidenceInterval
    false
    taskVersionId
    0
    evaluationDate
    2025-06-16T00:00:00Z
    hasBooleanResult
    false
    hasNumericResult
    true
    hasTaskVersionId
    false
    hasEvaluationDate
    true
    numericResultNullable
    value
    0.468
    standardDeviation
    0
    confidenceInterval
    0.0618524389605176
    hasStandardDeviation
    false
    hasConfidenceInterval
    true
    confidenceIntervalNullable
    0.0618524389605176
    hasUnevenConfidenceInterval
    false
    hasNumericResultPublic
    false
    customAdditionalResults
      hasNumericResultPrivate
      false
      task_id
      44
      sort_order
      DESCENDING
      source_name
      EM Korean
      display_type
      percentages
      task_version
      1
      task_display_type
      PERCENTAGES
      normalization_status
      normalized
    published_at
    2025-07-15T11:49:31.790Z
    candidate_class
    modelVersion
    source_revision
    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
    publisher_anchor
    openai/o3-2025-04-16
    metric_population
    51
    benchmark_evaluation
    true
    task_metadata_source
    benchmarkVersion
    candidate_metadata_source
    benchmarkVersionMappings
    evaluated at
    2025-06-16T00:00:00.000Z
    source model id
    o3
    confidence lower
    0.4061475610394824
    confidence upper
    0.5298524389605176
    Original result ↗

    SWE-bench

    35.98%

    source id
    sweBench
    leaderboard name
    SWE-bench Multimodal
    dimension
    coding
    metric
    Resolved tasks
    rank
    2
    evaluated at
    2025-11-17T00:00:00.000Z
    Original result ↗

    LLM Stats · GitHub archive

    0.065

    source id
    llmStatsArchive
    leaderboard name
    ARC-AGI v2 · Normalized (reported)
    dimension
    general
    metric
    ARC-AGI v2 · Normalized (reported)
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    ARC-AGI v2
    language
    en
    modality
    multimodal
    verified
    false
    max_score
    1
    categories
    • reasoning
    • vision
    • spatial_reasoning
    created_at
    2025-07-19T19:56:13.916360+00:00
    paper_link
    https://arxiv.org/abs/2505.11831
    updated_at
    2025-07-19T19:56:13.916360+00:00
    description
    ARC-AGI-2 is an upgraded benchmark for measuring abstract reasoning and problem-solving abilities in AI systems through visual grid transformation tasks. It evaluates fluid intelligence via input-output grid pairs (1x1 to 30x30) using colored cells (0-9), requiring models to identify underlying transformation rules from demonstration examples and apply them to test cases. Designed to be easy for humans but challenging for AI, focusing on core cognitive abilities like spatial reasoning, pattern recognition, and compositional generalization.
    benchmark_id
    arc-agi-v2
    multilingual
    false
    analysis_method
    accuracy
    is_self_reported
    false
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    ARC-AGI v2 · Normalized (reported)
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/openai/models/o3-2025-04-16/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://x.com/xai/status/1943158495588815072
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    openai/o3-2025-04-16
    metric_population
    3
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    o3-2025-04-16
    Original result ↗

    LLM Stats · GitHub archive

    0.065

    source id
    llmStatsArchive
    leaderboard name
    ARC-AGI v2 · Raw
    dimension
    general
    metric
    ARC-AGI v2 · Raw
    rank
    2
    aggregation scope
    status
    archived
    license
    CC-BY-4.0
    direction
    higherIsBetter
    conditions
    language
    en
    benchmark
    name
    ARC-AGI v2
    language
    en
    modality
    multimodal
    verified
    false
    max_score
    1
    categories
    • reasoning
    • vision
    • spatial_reasoning
    created_at
    2025-07-19T19:56:13.916360+00:00
    paper_link
    https://arxiv.org/abs/2505.11831
    updated_at
    2025-07-19T19:56:13.916360+00:00
    description
    ARC-AGI-2 is an upgraded benchmark for measuring abstract reasoning and problem-solving abilities in AI systems through visual grid transformation tasks. It evaluates fluid intelligence via input-output grid pairs (1x1 to 30x30) using colored cells (0-9), requiring models to identify underlying transformation rules from demonstration examples and apply them to test cases. Designed to be easy for humans but challenging for AI, focusing on core cognitive abilities like spatial reasoning, pattern recognition, and compositional generalization.
    benchmark_id
    arc-agi-v2
    multilingual
    false
    analysis_method
    accuracy
    is_self_reported
    false
    verified_by_llmstats
    false
    attribution
    Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
    license_url
    https://creativecommons.org/licenses/by/4.0/
    metric_kind
    ARC-AGI v2 · Raw
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    source_file
    data/organizations/openai/models/o3-2025-04-16/benchmarks.json
    archive_kind
    llmStats
    evidence_url
    https://x.com/xai/status/1943158495588815072
    git_revision
    872b75f63b8d853c46ed17856424029a0c47b1d2
    published_at
    2025-10-24T17:47:59Z
    snapshot_date
    2025-10-24T17:47:59Z
    source_revision
    6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
    publisher_anchor
    openai/o3-2025-04-16
    metric_population
    3
    current_catalog_url
    https://llm-stats.com/
    benchmark_evaluation
    true
    source model id
    o3-2025-04-16
    Original result ↗

    Kaggle Benchmarks

    934

    source id
    kaggleBenchmarks
    leaderboard name
    Chess Openings · Score
    dimension
    general
    metric
    Chess Openings · Score
    rank
    3
    aggregation scope
    kaggle
    true
    status
    published
    license
    Apache-2.0
    direction
    higherIsBetter
    conditions
    task
    id
    1869
    name
    Score
    version
    1
    sort_order
    DESCENDING
    display_type
    counts
    benchmark
    id
    183
    name
    Chess Openings
    type
    GAME
    owner
    kaggle
    version
    2
    categories
      version_id
      436
      maintainer_tier
      Kaggle-published
      candidate
      candidate_class
      modelVersion
      collection
      kaggleGameArena
      record_kind
      aggregate
      game_arena
      true
      attribution
      Kaggle · kaggle · Chess Openings
      license_url
      https://www.apache.org/licenses/LICENSE-2.0
      metric_kind
      Chess Openings · Score
      rank_method
      competition rank among reported evaluations in identical conditions, including unlinked observations
      source_file
      436/leaderboard.json
      source_page
      https://www.kaggle.com/benchmarks/kaggle/chess-text-openings/versions/2
      measurements
      • name
        Score
        unit
        score
        scale
        1
        metric
        Chess Openings · Score
        result
        resultCase
        numericResult
        booleanResult
        false
        numericResult
        value
        934
        standardDeviation
        0
        confidenceInterval
        0
        hasStandardDeviation
        false
        hasConfidenceInterval
        false
        unevenConfidenceInterval
        plus
        98
        minus
        95
        hasUnevenConfidenceInterval
        true
        taskVersionId
        0
        evaluationDate
        2026-01-13T00:00:00Z
        hasBooleanResult
        false
        hasNumericResult
        true
        hasTaskVersionId
        false
        hasEvaluationDate
        true
        numericResultNullable
        value
        934
        standardDeviation
        0
        confidenceInterval
        0
        hasStandardDeviation
        false
        hasConfidenceInterval
        false
        unevenConfidenceInterval
        plus
        98
        minus
        95
        hasUnevenConfidenceInterval
        true
        hasNumericResultPublic
        false
        customAdditionalResults
          hasNumericResultPrivate
          false
          task_id
          1869
          sort_order
          DESCENDING
          source_name
          Score
          display_type
          counts
          task_version
          1
          task_display_type
          COUNTS
          normalization_status
          normalized
        published_at
        2026-02-02T15:46:28.981771500Z
        candidate_class
        modelVersion
        methodology_url
        https://www.kaggle.com/game-arena-faq
        source_revision
        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
        publisher_anchor
        openai/o3-2025-04-16
        game_arena_rating
        Game-specific rating; compare within this game and version
        metric_population
        11
        benchmark_evaluation
        true
        task_metadata_source
        benchmarkVersion
        candidate_metadata_source
        benchmarkVersionMappings
        evaluated at
        2026-01-13T00:00:00.000Z
        source model id
        o3
        confidence lower
        839
        confidence upper
        1032
        Original result ↗

        Kaggle Benchmarks

        69.6%

        source id
        kaggleBenchmarks
        leaderboard name
        MultiLoKo English · EM English
        dimension
        reasoning
        metric
        MultiLoKo English · EM English
        rank
        3
        aggregation scope
        kaggle
        true
        status
        published
        license
        Apache-2.0
        direction
        higherIsBetter
        conditions
        task
        id
        27
        name
        EM English
        version
        1
        sort_order
        DESCENDING
        display_type
        percentages
        benchmark
        id
        52
        name
        MultiLoKo English
        type
        INDIVIDUAL
        owner
        metaresearch
        version
        1
        categories
        • multilingual and cross-lingual capabilities
        • general knowledge and reasoning
        version_id
        38
        maintainer_tier
        Kaggle-maintained
        maintenance_level
        KAGGLE_MAINTAINED
        candidate
        candidate_class
        modelVersion
        record_kind
        aggregate
        attribution
        Kaggle · metaresearch · MultiLoKo English
        license_url
        https://www.apache.org/licenses/LICENSE-2.0
        metric_kind
        MultiLoKo English · EM English
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        source_file
        38/leaderboard.json
        source_page
        https://www.kaggle.com/benchmarks/metaresearch/multiloko-english/versions/1
        measurements
        • name
          EM English
          unit
          percent
          scale
          1
          metric
          MultiLoKo English · EM English
          result
          resultCase
          numericResult
          booleanResult
          false
          numericResult
          value
          0.696
          standardDeviation
          0
          confidenceInterval
          0.0570189871224575
          hasStandardDeviation
          false
          hasConfidenceInterval
          true
          confidenceIntervalNullable
          0.0570189871224575
          hasUnevenConfidenceInterval
          false
          taskVersionId
          0
          evaluationDate
          2025-06-16T00:00:00Z
          hasBooleanResult
          false
          hasNumericResult
          true
          hasTaskVersionId
          false
          hasEvaluationDate
          true
          numericResultNullable
          value
          0.696
          standardDeviation
          0
          confidenceInterval
          0.0570189871224575
          hasStandardDeviation
          false
          hasConfidenceInterval
          true
          confidenceIntervalNullable
          0.0570189871224575
          hasUnevenConfidenceInterval
          false
          hasNumericResultPublic
          false
          customAdditionalResults
            hasNumericResultPrivate
            false
            task_id
            27
            sort_order
            DESCENDING
            source_name
            EM English
            display_type
            percentages
            task_version
            1
            task_display_type
            PERCENTAGES
            normalization_status
            normalized
          published_at
          2025-07-15T11:49:31.790Z
          candidate_class
          modelVersion
          source_revision
          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
          publisher_anchor
          openai/o3-2025-04-16
          metric_population
          51
          benchmark_evaluation
          true
          task_metadata_source
          benchmarkVersion
          candidate_metadata_source
          benchmarkVersionMappings
          evaluated at
          2025-06-16T00:00:00.000Z
          source model id
          o3
          confidence lower
          0.6389810128775425
          confidence upper
          0.7530189871224574
          Original result ↗

          Kaggle Benchmarks

          63.705%

          source id
          kaggleBenchmarks
          leaderboard name
          AssetOpsBench · Selection Accuracy
          dimension
          reasoning
          metric
          AssetOpsBench · Selection Accuracy
          rank
          3
          aggregation scope
          kaggle
          true
          status
          published
          license
          Apache-2.0
          direction
          higherIsBetter
          conditions
          task
          id
          1042
          name
          Selection Accuracy
          version
          1
          sort_order
          DESCENDING
          display_type
          percentages
          benchmark
          id
          243
          name
          AssetOpsBench
          type
          INDIVIDUAL
          owner
          ibm-research
          version
          1
          categories
          • evaluation
          • general knowledge and reasoning
          version_id
          222
          maintainer_tier
          Kaggle-maintained
          maintenance_level
          KAGGLE_MAINTAINED
          candidate
          candidate_class
          modelVersion
          record_kind
          aggregate
          attribution
          Kaggle · ibm-research · AssetOpsBench
          license_url
          https://www.apache.org/licenses/LICENSE-2.0
          metric_kind
          AssetOpsBench · Selection Accuracy
          rank_method
          competition rank among reported evaluations in identical conditions, including unlinked observations
          source_file
          222/leaderboard.json
          source_page
          https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
          measurements
          • name
            Selection Accuracy
            unit
            percent
            scale
            1
            metric
            AssetOpsBench · Selection Accuracy
            result
            resultCase
            numericResult
            booleanResult
            false
            numericResult
            value
            0.6370530877573131
            standardDeviation
            0
            confidenceInterval
            0
            hasStandardDeviation
            false
            hasConfidenceInterval
            false
            hasUnevenConfidenceInterval
            false
            taskVersionId
            0
            evaluationDate
            2025-11-22T14:59:47.329153Z
            hasBooleanResult
            false
            hasNumericResult
            true
            hasTaskVersionId
            false
            hasEvaluationDate
            true
            numericResultNullable
            value
            0.6370530877573131
            standardDeviation
            0
            confidenceInterval
            0
            hasStandardDeviation
            false
            hasConfidenceInterval
            false
            hasUnevenConfidenceInterval
            false
            hasNumericResultPublic
            false
            customAdditionalResults
              hasNumericResultPrivate
              false
              task_id
              1042
              task_url
              /benchmarks/tasks/benchmarkler/failuresensoriq-selectionaccuracy-benchmark-task
              sort_order
              DESCENDING
              source_name
              Selection Accuracy
              display_type
              percentages
              task_version
              1
              task_display_type
              RAW_VALUES
              normalization_status
              normalized
            published_at
            2025-12-01T22:15:26.261663300Z
            candidate_class
            modelVersion
            source_revision
            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
            publisher_anchor
            openai/o3-2025-04-16
            metric_population
            25
            benchmark_evaluation
            true
            task_metadata_source
            benchmarkVersion
            candidate_metadata_source
            benchmarkVersionMappings
            evaluated at
            2025-11-22T14:59:47.329Z
            source model id
            o3
            Original result ↗

            Kaggle Benchmarks

            52%

            source id
            kaggleBenchmarks
            leaderboard name
            MultiLoKo Tagalog · EM Tagalog
            dimension
            reasoning
            metric
            MultiLoKo Tagalog · EM Tagalog
            rank
            3
            aggregation scope
            kaggle
            true
            status
            published
            license
            Apache-2.0
            direction
            higherIsBetter
            conditions
            task
            id
            54
            name
            EM Tagalog
            version
            1
            sort_order
            DESCENDING
            display_type
            percentages
            benchmark
            id
            77
            name
            MultiLoKo Tagalog
            type
            INDIVIDUAL
            owner
            metaresearch
            version
            1
            categories
            • multilingual and cross-lingual capabilities
            • general knowledge and reasoning
            version_id
            63
            maintainer_tier
            Kaggle-maintained
            maintenance_level
            KAGGLE_MAINTAINED
            candidate
            candidate_class
            modelVersion
            record_kind
            aggregate
            attribution
            Kaggle · metaresearch · MultiLoKo Tagalog
            license_url
            https://www.apache.org/licenses/LICENSE-2.0
            metric_kind
            MultiLoKo Tagalog · EM Tagalog
            rank_method
            competition rank among reported evaluations in identical conditions, including unlinked observations
            source_file
            63/leaderboard.json
            source_page
            https://www.kaggle.com/benchmarks/metaresearch/multiloko-tagalog/versions/1
            measurements
            • name
              EM Tagalog
              unit
              percent
              scale
              1
              metric
              MultiLoKo Tagalog · EM Tagalog
              result
              resultCase
              numericResult
              booleanResult
              false
              numericResult
              value
              0.52
              standardDeviation
              0
              confidenceInterval
              0.0619298997785481
              hasStandardDeviation
              false
              hasConfidenceInterval
              true
              confidenceIntervalNullable
              0.0619298997785481
              hasUnevenConfidenceInterval
              false
              taskVersionId
              0
              evaluationDate
              2025-06-16T00:00:00Z
              hasBooleanResult
              false
              hasNumericResult
              true
              hasTaskVersionId
              false
              hasEvaluationDate
              true
              numericResultNullable
              value
              0.52
              standardDeviation
              0
              confidenceInterval
              0.0619298997785481
              hasStandardDeviation
              false
              hasConfidenceInterval
              true
              confidenceIntervalNullable
              0.0619298997785481
              hasUnevenConfidenceInterval
              false
              hasNumericResultPublic
              false
              customAdditionalResults
                hasNumericResultPrivate
                false
                task_id
                54
                sort_order
                DESCENDING
                source_name
                EM Tagalog
                display_type
                percentages
                task_version
                1
                task_display_type
                PERCENTAGES
                normalization_status
                normalized
              published_at
              2025-07-15T11:49:31.790Z
              candidate_class
              modelVersion
              source_revision
              b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
              publisher_anchor
              openai/o3-2025-04-16
              metric_population
              51
              benchmark_evaluation
              true
              task_metadata_source
              benchmarkVersion
              candidate_metadata_source
              benchmarkVersionMappings
              evaluated at
              2025-06-16T00:00:00.000Z
              source model id
              o3
              confidence lower
              0.4580701002214519
              confidence upper
              0.5819298997785481
              Original result ↗

              Kaggle Benchmarks

              67.579%

              source id
              kaggleBenchmarks
              leaderboard name
              AssetOpsBench · Score
              dimension
              reasoning
              metric
              AssetOpsBench · Score
              rank
              4
              aggregation scope
              kaggle
              true
              status
              published
              license
              Apache-2.0
              direction
              higherIsBetter
              conditions
              task
              id
              487619
              name
              Score
              version
              5
              sort_order
              DESCENDING
              display_type
              percentages
              benchmark
              id
              243
              name
              AssetOpsBench
              type
              INDIVIDUAL
              owner
              ibm-research
              version
              1
              categories
              • evaluation
              • general knowledge and reasoning
              version_id
              222
              maintainer_tier
              Kaggle-maintained
              maintenance_level
              KAGGLE_MAINTAINED
              candidate
              candidate_class
              modelVersion
              record_kind
              aggregate
              attribution
              Kaggle · ibm-research · AssetOpsBench
              license_url
              https://www.apache.org/licenses/LICENSE-2.0
              metric_kind
              AssetOpsBench · Score
              rank_method
              competition rank among reported evaluations in identical conditions, including unlinked observations
              source_file
              222/leaderboard.json
              source_page
              https://www.kaggle.com/benchmarks/ibm-research/asset-ops-bench/versions/1
              measurements
              • name
                Score
                unit
                percent
                scale
                1
                metric
                AssetOpsBench · Score
                result
                resultCase
                numericResult
                booleanResult
                false
                numericResult
                value
                0.675790526184227
                standardDeviation
                0
                confidenceInterval
                0
                hasStandardDeviation
                false
                hasConfidenceInterval
                false
                hasUnevenConfidenceInterval
                false
                taskVersionId
                0
                evaluationDate
                2025-11-22T04:59:59.816908Z
                hasBooleanResult
                false
                hasNumericResult
                true
                hasTaskVersionId
                false
                hasEvaluationDate
                true
                numericResultNullable
                value
                0.675790526184227
                standardDeviation
                0
                confidenceInterval
                0
                hasStandardDeviation
                false
                hasConfidenceInterval
                false
                hasUnevenConfidenceInterval
                false
                hasNumericResultPublic
                false
                customAdditionalResults
                  hasNumericResultPrivate
                  false
                  task_id
                  487619
                  task_url
                  /benchmarks/tasks/benchmarkler/failuresensoriq-main-benchmark-task
                  sort_order
                  DESCENDING
                  source_name
                  Score
                  display_type
                  percentages
                  task_version
                  5
                  task_display_type
                  PERCENTAGES
                  normalization_status
                  normalized
                published_at
                2025-12-01T22:15:26.261663300Z
                candidate_class
                modelVersion
                source_revision
                b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                publisher_anchor
                openai/o3-2025-04-16
                metric_population
                25
                benchmark_evaluation
                true
                task_metadata_source
                benchmarkVersion
                candidate_metadata_source
                benchmarkVersionMappings
                evaluated at
                2025-11-22T04:59:59.816Z
                source model id
                o3
                Original result ↗

                Kaggle Benchmarks

                54.8%

                source id
                kaggleBenchmarks
                leaderboard name
                MultiLoKo French · EM French
                dimension
                reasoning
                metric
                MultiLoKo French · EM French
                rank
                4
                aggregation scope
                kaggle
                true
                status
                published
                license
                Apache-2.0
                direction
                higherIsBetter
                conditions
                task
                id
                36
                name
                EM French
                version
                1
                sort_order
                DESCENDING
                display_type
                percentages
                benchmark
                id
                59
                name
                MultiLoKo French
                type
                INDIVIDUAL
                owner
                metaresearch
                version
                1
                categories
                • multilingual and cross-lingual capabilities
                • general knowledge and reasoning
                version_id
                45
                maintainer_tier
                Kaggle-maintained
                maintenance_level
                KAGGLE_MAINTAINED
                candidate
                candidate_class
                modelVersion
                record_kind
                aggregate
                attribution
                Kaggle · metaresearch · MultiLoKo French
                license_url
                https://www.apache.org/licenses/LICENSE-2.0
                metric_kind
                MultiLoKo French · EM French
                rank_method
                competition rank among reported evaluations in identical conditions, including unlinked observations
                source_file
                45/leaderboard.json
                source_page
                https://www.kaggle.com/benchmarks/metaresearch/multiloko-french/versions/1
                measurements
                • name
                  EM French
                  unit
                  percent
                  scale
                  1
                  metric
                  MultiLoKo French · EM French
                  result
                  resultCase
                  numericResult
                  booleanResult
                  false
                  numericResult
                  value
                  0.548
                  standardDeviation
                  0
                  confidenceInterval
                  0.0616932406038344
                  hasStandardDeviation
                  false
                  hasConfidenceInterval
                  true
                  confidenceIntervalNullable
                  0.0616932406038344
                  hasUnevenConfidenceInterval
                  false
                  taskVersionId
                  0
                  evaluationDate
                  2025-06-16T00:00:00Z
                  hasBooleanResult
                  false
                  hasNumericResult
                  true
                  hasTaskVersionId
                  false
                  hasEvaluationDate
                  true
                  numericResultNullable
                  value
                  0.548
                  standardDeviation
                  0
                  confidenceInterval
                  0.0616932406038344
                  hasStandardDeviation
                  false
                  hasConfidenceInterval
                  true
                  confidenceIntervalNullable
                  0.0616932406038344
                  hasUnevenConfidenceInterval
                  false
                  hasNumericResultPublic
                  false
                  customAdditionalResults
                    hasNumericResultPrivate
                    false
                    task_id
                    36
                    sort_order
                    DESCENDING
                    source_name
                    EM French
                    display_type
                    percentages
                    task_version
                    1
                    task_display_type
                    PERCENTAGES
                    normalization_status
                    normalized
                  published_at
                  2025-07-15T11:49:31.790Z
                  candidate_class
                  modelVersion
                  source_revision
                  b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                  publisher_anchor
                  openai/o3-2025-04-16
                  metric_population
                  51
                  benchmark_evaluation
                  true
                  task_metadata_source
                  benchmarkVersion
                  candidate_metadata_source
                  benchmarkVersionMappings
                  evaluated at
                  2025-06-16T00:00:00.000Z
                  source model id
                  o3
                  confidence lower
                  0.48630675939616563
                  confidence upper
                  0.6096932406038345
                  Original result ↗

                  Kaggle Benchmarks

                  47.6%

                  source id
                  kaggleBenchmarks
                  leaderboard name
                  MultiLoKo Dutch · EM Dutch
                  dimension
                  reasoning
                  metric
                  MultiLoKo Dutch · EM Dutch
                  rank
                  4
                  aggregation scope
                  kaggle
                  true
                  status
                  published
                  license
                  Apache-2.0
                  direction
                  higherIsBetter
                  conditions
                  task
                  id
                  34
                  name
                  EM Dutch
                  version
                  1
                  sort_order
                  DESCENDING
                  display_type
                  percentages
                  benchmark
                  id
                  57
                  name
                  MultiLoKo Dutch
                  type
                  INDIVIDUAL
                  owner
                  metaresearch
                  version
                  1
                  categories
                  • multilingual and cross-lingual capabilities
                  • general knowledge and reasoning
                  version_id
                  43
                  maintainer_tier
                  Kaggle-maintained
                  maintenance_level
                  KAGGLE_MAINTAINED
                  candidate
                  candidate_class
                  modelVersion
                  record_kind
                  aggregate
                  attribution
                  Kaggle · metaresearch · MultiLoKo Dutch
                  license_url
                  https://www.apache.org/licenses/LICENSE-2.0
                  metric_kind
                  MultiLoKo Dutch · EM Dutch
                  rank_method
                  competition rank among reported evaluations in identical conditions, including unlinked observations
                  source_file
                  43/leaderboard.json
                  source_page
                  https://www.kaggle.com/benchmarks/metaresearch/multiloko-dutch/versions/1
                  measurements
                  • name
                    EM Dutch
                    unit
                    percent
                    scale
                    1
                    metric
                    MultiLoKo Dutch · EM Dutch
                    result
                    resultCase
                    numericResult
                    booleanResult
                    false
                    numericResult
                    value
                    0.476
                    standardDeviation
                    0
                    confidenceInterval
                    0.0619080616686651
                    hasStandardDeviation
                    false
                    hasConfidenceInterval
                    true
                    confidenceIntervalNullable
                    0.0619080616686651
                    hasUnevenConfidenceInterval
                    false
                    taskVersionId
                    0
                    evaluationDate
                    2025-06-16T00:00:00Z
                    hasBooleanResult
                    false
                    hasNumericResult
                    true
                    hasTaskVersionId
                    false
                    hasEvaluationDate
                    true
                    numericResultNullable
                    value
                    0.476
                    standardDeviation
                    0
                    confidenceInterval
                    0.0619080616686651
                    hasStandardDeviation
                    false
                    hasConfidenceInterval
                    true
                    confidenceIntervalNullable
                    0.0619080616686651
                    hasUnevenConfidenceInterval
                    false
                    hasNumericResultPublic
                    false
                    customAdditionalResults
                      hasNumericResultPrivate
                      false
                      task_id
                      34
                      sort_order
                      DESCENDING
                      source_name
                      EM Dutch
                      display_type
                      percentages
                      task_version
                      1
                      task_display_type
                      PERCENTAGES
                      normalization_status
                      normalized
                    published_at
                    2025-07-15T11:49:31.790Z
                    candidate_class
                    modelVersion
                    source_revision
                    b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                    publisher_anchor
                    openai/o3-2025-04-16
                    metric_population
                    51
                    benchmark_evaluation
                    true
                    task_metadata_source
                    benchmarkVersion
                    candidate_metadata_source
                    benchmarkVersionMappings
                    evaluated at
                    2025-06-16T00:00:00.000Z
                    source model id
                    o3
                    confidence lower
                    0.4140919383313349
                    confidence upper
                    0.5379080616686651
                    Original result ↗

                    Kaggle Benchmarks

                    41.2%

                    source id
                    kaggleBenchmarks
                    leaderboard name
                    MultiLoKo Hebrew · EM Hebrew
                    dimension
                    reasoning
                    metric
                    MultiLoKo Hebrew · EM Hebrew
                    rank
                    4
                    aggregation scope
                    kaggle
                    true
                    status
                    published
                    license
                    Apache-2.0
                    direction
                    higherIsBetter
                    conditions
                    task
                    id
                    38
                    name
                    EM Hebrew
                    version
                    1
                    sort_order
                    DESCENDING
                    display_type
                    percentages
                    benchmark
                    id
                    61
                    name
                    MultiLoKo Hebrew
                    type
                    INDIVIDUAL
                    owner
                    metaresearch
                    version
                    1
                    categories
                    • multilingual and cross-lingual capabilities
                    • general knowledge and reasoning
                    version_id
                    47
                    maintainer_tier
                    Kaggle-maintained
                    maintenance_level
                    KAGGLE_MAINTAINED
                    candidate
                    candidate_class
                    modelVersion
                    record_kind
                    aggregate
                    attribution
                    Kaggle · metaresearch · MultiLoKo Hebrew
                    license_url
                    https://www.apache.org/licenses/LICENSE-2.0
                    metric_kind
                    MultiLoKo Hebrew · EM Hebrew
                    rank_method
                    competition rank among reported evaluations in identical conditions, including unlinked observations
                    source_file
                    47/leaderboard.json
                    source_page
                    https://www.kaggle.com/benchmarks/metaresearch/multiloko-hebrew/versions/1
                    measurements
                    • name
                      EM Hebrew
                      unit
                      percent
                      scale
                      1
                      metric
                      MultiLoKo Hebrew · EM Hebrew
                      result
                      resultCase
                      numericResult
                      booleanResult
                      false
                      numericResult
                      value
                      0.412
                      standardDeviation
                      0
                      confidenceInterval
                      0.0610120135077044
                      hasStandardDeviation
                      false
                      hasConfidenceInterval
                      true
                      confidenceIntervalNullable
                      0.0610120135077044
                      hasUnevenConfidenceInterval
                      false
                      taskVersionId
                      0
                      evaluationDate
                      2025-06-16T00:00:00Z
                      hasBooleanResult
                      false
                      hasNumericResult
                      true
                      hasTaskVersionId
                      false
                      hasEvaluationDate
                      true
                      numericResultNullable
                      value
                      0.412
                      standardDeviation
                      0
                      confidenceInterval
                      0.0610120135077044
                      hasStandardDeviation
                      false
                      hasConfidenceInterval
                      true
                      confidenceIntervalNullable
                      0.0610120135077044
                      hasUnevenConfidenceInterval
                      false
                      hasNumericResultPublic
                      false
                      customAdditionalResults
                        hasNumericResultPrivate
                        false
                        task_id
                        38
                        sort_order
                        DESCENDING
                        source_name
                        EM Hebrew
                        display_type
                        percentages
                        task_version
                        1
                        task_display_type
                        PERCENTAGES
                        normalization_status
                        normalized
                      published_at
                      2025-07-15T11:49:31.790Z
                      candidate_class
                      modelVersion
                      source_revision
                      b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                      publisher_anchor
                      openai/o3-2025-04-16
                      metric_population
                      51
                      benchmark_evaluation
                      true
                      task_metadata_source
                      benchmarkVersion
                      candidate_metadata_source
                      benchmarkVersionMappings
                      evaluated at
                      2025-06-16T00:00:00.000Z
                      source model id
                      o3
                      confidence lower
                      0.35098798649229557
                      confidence upper
                      0.4730120135077044
                      Original result ↗

                      Kaggle Benchmarks

                      40.8%

                      source id
                      kaggleBenchmarks
                      leaderboard name
                      MultiLoKo Japanese · EM Japanese
                      dimension
                      reasoning
                      metric
                      MultiLoKo Japanese · EM Japanese
                      rank
                      4
                      aggregation scope
                      kaggle
                      true
                      status
                      published
                      license
                      Apache-2.0
                      direction
                      higherIsBetter
                      conditions
                      task
                      id
                      42
                      name
                      EM Japanese
                      version
                      1
                      sort_order
                      DESCENDING
                      display_type
                      percentages
                      benchmark
                      id
                      65
                      name
                      MultiLoKo Japanese
                      type
                      INDIVIDUAL
                      owner
                      metaresearch
                      version
                      1
                      categories
                      • multilingual and cross-lingual capabilities
                      • general knowledge and reasoning
                      version_id
                      51
                      maintainer_tier
                      Kaggle-maintained
                      maintenance_level
                      KAGGLE_MAINTAINED
                      candidate
                      candidate_class
                      modelVersion
                      record_kind
                      aggregate
                      attribution
                      Kaggle · metaresearch · MultiLoKo Japanese
                      license_url
                      https://www.apache.org/licenses/LICENSE-2.0
                      metric_kind
                      MultiLoKo Japanese · EM Japanese
                      rank_method
                      competition rank among reported evaluations in identical conditions, including unlinked observations
                      source_file
                      51/leaderboard.json
                      source_page
                      https://www.kaggle.com/benchmarks/metaresearch/multiloko-japanese/versions/1
                      measurements
                      • name
                        EM Japanese
                        unit
                        percent
                        scale
                        1
                        metric
                        MultiLoKo Japanese · EM Japanese
                        result
                        resultCase
                        numericResult
                        booleanResult
                        false
                        numericResult
                        value
                        0.408
                        standardDeviation
                        0
                        confidenceInterval
                        0.0609212802792317
                        hasStandardDeviation
                        false
                        hasConfidenceInterval
                        true
                        confidenceIntervalNullable
                        0.0609212802792317
                        hasUnevenConfidenceInterval
                        false
                        taskVersionId
                        0
                        evaluationDate
                        2025-06-16T00:00:00Z
                        hasBooleanResult
                        false
                        hasNumericResult
                        true
                        hasTaskVersionId
                        false
                        hasEvaluationDate
                        true
                        numericResultNullable
                        value
                        0.408
                        standardDeviation
                        0
                        confidenceInterval
                        0.0609212802792317
                        hasStandardDeviation
                        false
                        hasConfidenceInterval
                        true
                        confidenceIntervalNullable
                        0.0609212802792317
                        hasUnevenConfidenceInterval
                        false
                        hasNumericResultPublic
                        false
                        customAdditionalResults
                          hasNumericResultPrivate
                          false
                          task_id
                          42
                          sort_order
                          DESCENDING
                          source_name
                          EM Japanese
                          display_type
                          percentages
                          task_version
                          1
                          task_display_type
                          PERCENTAGES
                          normalization_status
                          normalized
                        published_at
                        2025-07-15T11:49:31.790Z
                        candidate_class
                        modelVersion
                        source_revision
                        b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                        publisher_anchor
                        openai/o3-2025-04-16
                        metric_population
                        51
                        benchmark_evaluation
                        true
                        task_metadata_source
                        benchmarkVersion
                        candidate_metadata_source
                        benchmarkVersionMappings
                        evaluated at
                        2025-06-16T00:00:00.000Z
                        source model id
                        o3
                        confidence lower
                        0.34707871972076826
                        confidence upper
                        0.4689212802792317
                        Original result ↗

                        Kaggle Benchmarks

                        38.8%

                        source id
                        kaggleBenchmarks
                        leaderboard name
                        MultiLoKo Arabic · EM Arabic
                        dimension
                        reasoning
                        metric
                        MultiLoKo Arabic · EM Arabic
                        rank
                        4
                        aggregation scope
                        kaggle
                        true
                        status
                        published
                        license
                        Apache-2.0
                        direction
                        higherIsBetter
                        conditions
                        task
                        id
                        30
                        name
                        EM Arabic
                        version
                        1
                        sort_order
                        DESCENDING
                        display_type
                        percentages
                        benchmark
                        id
                        53
                        name
                        MultiLoKo Arabic
                        type
                        INDIVIDUAL
                        owner
                        metaresearch
                        version
                        1
                        categories
                        • multilingual and cross-lingual capabilities
                        • general knowledge and reasoning
                        version_id
                        39
                        maintainer_tier
                        Kaggle-maintained
                        maintenance_level
                        KAGGLE_MAINTAINED
                        candidate
                        candidate_class
                        modelVersion
                        record_kind
                        aggregate
                        attribution
                        Kaggle · metaresearch · MultiLoKo Arabic
                        license_url
                        https://www.apache.org/licenses/LICENSE-2.0
                        metric_kind
                        MultiLoKo Arabic · EM Arabic
                        rank_method
                        competition rank among reported evaluations in identical conditions, including unlinked observations
                        source_file
                        39/leaderboard.json
                        source_page
                        https://www.kaggle.com/benchmarks/metaresearch/multiloko-arabic/versions/1
                        measurements
                        • name
                          EM Arabic
                          unit
                          percent
                          scale
                          1
                          metric
                          MultiLoKo Arabic · EM Arabic
                          result
                          resultCase
                          numericResult
                          booleanResult
                          false
                          numericResult
                          value
                          0.388
                          standardDeviation
                          0
                          confidenceInterval
                          0.0604045510115502
                          hasStandardDeviation
                          false
                          hasConfidenceInterval
                          true
                          confidenceIntervalNullable
                          0.0604045510115502
                          hasUnevenConfidenceInterval
                          false
                          taskVersionId
                          0
                          evaluationDate
                          2025-06-16T00:00:00Z
                          hasBooleanResult
                          false
                          hasNumericResult
                          true
                          hasTaskVersionId
                          false
                          hasEvaluationDate
                          true
                          numericResultNullable
                          value
                          0.388
                          standardDeviation
                          0
                          confidenceInterval
                          0.0604045510115502
                          hasStandardDeviation
                          false
                          hasConfidenceInterval
                          true
                          confidenceIntervalNullable
                          0.0604045510115502
                          hasUnevenConfidenceInterval
                          false
                          hasNumericResultPublic
                          false
                          customAdditionalResults
                            hasNumericResultPrivate
                            false
                            task_id
                            30
                            sort_order
                            DESCENDING
                            source_name
                            EM Arabic
                            display_type
                            percentages
                            task_version
                            1
                            task_display_type
                            PERCENTAGES
                            normalization_status
                            normalized
                          published_at
                          2025-07-15T11:49:31.790Z
                          candidate_class
                          modelVersion
                          source_revision
                          b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                          publisher_anchor
                          openai/o3-2025-04-16
                          metric_population
                          51
                          benchmark_evaluation
                          true
                          task_metadata_source
                          benchmarkVersion
                          candidate_metadata_source
                          benchmarkVersionMappings
                          evaluated at
                          2025-06-16T00:00:00.000Z
                          source model id
                          o3
                          confidence lower
                          0.32759544898844983
                          confidence upper
                          0.4484045510115502
                          Original result ↗

                          Kaggle Benchmarks

                          36.8%

                          source id
                          kaggleBenchmarks
                          leaderboard name
                          MultiLoKo Bengali · EM Bengali
                          dimension
                          reasoning
                          metric
                          MultiLoKo Bengali · EM Bengali
                          rank
                          4
                          aggregation scope
                          kaggle
                          true
                          status
                          published
                          license
                          Apache-2.0
                          direction
                          higherIsBetter
                          conditions
                          task
                          id
                          31
                          name
                          EM Bengali
                          version
                          1
                          sort_order
                          DESCENDING
                          display_type
                          percentages
                          benchmark
                          id
                          54
                          name
                          MultiLoKo Bengali
                          type
                          INDIVIDUAL
                          owner
                          metaresearch
                          version
                          1
                          categories
                          • multilingual and cross-lingual capabilities
                          • general knowledge and reasoning
                          version_id
                          40
                          maintainer_tier
                          Kaggle-maintained
                          maintenance_level
                          KAGGLE_MAINTAINED
                          candidate
                          candidate_class
                          modelVersion
                          record_kind
                          aggregate
                          attribution
                          Kaggle · metaresearch · MultiLoKo Bengali
                          license_url
                          https://www.apache.org/licenses/LICENSE-2.0
                          metric_kind
                          MultiLoKo Bengali · EM Bengali
                          rank_method
                          competition rank among reported evaluations in identical conditions, including unlinked observations
                          source_file
                          40/leaderboard.json
                          source_page
                          https://www.kaggle.com/benchmarks/metaresearch/multiloko-bengali/versions/1
                          measurements
                          • name
                            EM Bengali
                            unit
                            percent
                            scale
                            1
                            metric
                            MultiLoKo Bengali · EM Bengali
                            result
                            resultCase
                            numericResult
                            booleanResult
                            false
                            numericResult
                            value
                            0.368
                            standardDeviation
                            0
                            confidenceInterval
                            0.0597806365533776
                            hasStandardDeviation
                            false
                            hasConfidenceInterval
                            true
                            confidenceIntervalNullable
                            0.0597806365533776
                            hasUnevenConfidenceInterval
                            false
                            taskVersionId
                            0
                            evaluationDate
                            2025-06-16T00:00:00Z
                            hasBooleanResult
                            false
                            hasNumericResult
                            true
                            hasTaskVersionId
                            false
                            hasEvaluationDate
                            true
                            numericResultNullable
                            value
                            0.368
                            standardDeviation
                            0
                            confidenceInterval
                            0.0597806365533776
                            hasStandardDeviation
                            false
                            hasConfidenceInterval
                            true
                            confidenceIntervalNullable
                            0.0597806365533776
                            hasUnevenConfidenceInterval
                            false
                            hasNumericResultPublic
                            false
                            customAdditionalResults
                              hasNumericResultPrivate
                              false
                              task_id
                              31
                              sort_order
                              DESCENDING
                              source_name
                              EM Bengali
                              display_type
                              percentages
                              task_version
                              1
                              task_display_type
                              PERCENTAGES
                              normalization_status
                              normalized
                            published_at
                            2025-07-15T11:49:31.790Z
                            candidate_class
                            modelVersion
                            source_revision
                            b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                            publisher_anchor
                            openai/o3-2025-04-16
                            metric_population
                            51
                            benchmark_evaluation
                            true
                            task_metadata_source
                            benchmarkVersion
                            candidate_metadata_source
                            benchmarkVersionMappings
                            evaluated at
                            2025-06-16T00:00:00.000Z
                            source model id
                            o3
                            confidence lower
                            0.3082193634466224
                            confidence upper
                            0.4277806365533776
                            Original result ↗

                            Kaggle Benchmarks

                            33.2%

                            source id
                            kaggleBenchmarks
                            leaderboard name
                            MultiLoKo Polish · EM Polish
                            dimension
                            reasoning
                            metric
                            MultiLoKo Polish · EM Polish
                            rank
                            4
                            aggregation scope
                            kaggle
                            true
                            status
                            published
                            license
                            Apache-2.0
                            direction
                            higherIsBetter
                            conditions
                            task
                            id
                            47
                            name
                            EM Polish
                            version
                            1
                            sort_order
                            DESCENDING
                            display_type
                            percentages
                            benchmark
                            id
                            70
                            name
                            MultiLoKo Polish
                            type
                            INDIVIDUAL
                            owner
                            metaresearch
                            version
                            1
                            categories
                            • multilingual and cross-lingual capabilities
                            • general knowledge and reasoning
                            version_id
                            56
                            maintainer_tier
                            Kaggle-maintained
                            maintenance_level
                            KAGGLE_MAINTAINED
                            candidate
                            candidate_class
                            modelVersion
                            record_kind
                            aggregate
                            attribution
                            Kaggle · metaresearch · MultiLoKo Polish
                            license_url
                            https://www.apache.org/licenses/LICENSE-2.0
                            metric_kind
                            MultiLoKo Polish · EM Polish
                            rank_method
                            competition rank among reported evaluations in identical conditions, including unlinked observations
                            source_file
                            56/leaderboard.json
                            source_page
                            https://www.kaggle.com/benchmarks/metaresearch/multiloko-polish/versions/1
                            measurements
                            • name
                              EM Polish
                              unit
                              percent
                              scale
                              1
                              metric
                              MultiLoKo Polish · EM Polish
                              result
                              resultCase
                              numericResult
                              booleanResult
                              false
                              numericResult
                              value
                              0.332
                              standardDeviation
                              0
                              confidenceInterval
                              0.0583761379818247
                              hasStandardDeviation
                              false
                              hasConfidenceInterval
                              true
                              confidenceIntervalNullable
                              0.0583761379818247
                              hasUnevenConfidenceInterval
                              false
                              taskVersionId
                              0
                              evaluationDate
                              2025-06-16T00:00:00Z
                              hasBooleanResult
                              false
                              hasNumericResult
                              true
                              hasTaskVersionId
                              false
                              hasEvaluationDate
                              true
                              numericResultNullable
                              value
                              0.332
                              standardDeviation
                              0
                              confidenceInterval
                              0.0583761379818247
                              hasStandardDeviation
                              false
                              hasConfidenceInterval
                              true
                              confidenceIntervalNullable
                              0.0583761379818247
                              hasUnevenConfidenceInterval
                              false
                              hasNumericResultPublic
                              false
                              customAdditionalResults
                                hasNumericResultPrivate
                                false
                                task_id
                                47
                                sort_order
                                DESCENDING
                                source_name
                                EM Polish
                                display_type
                                percentages
                                task_version
                                1
                                task_display_type
                                PERCENTAGES
                                normalization_status
                                normalized
                              published_at
                              2025-07-15T11:49:31.790Z
                              candidate_class
                              modelVersion
                              source_revision
                              b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                              publisher_anchor
                              openai/o3-2025-04-16
                              metric_population
                              51
                              benchmark_evaluation
                              true
                              task_metadata_source
                              benchmarkVersion
                              candidate_metadata_source
                              benchmarkVersionMappings
                              evaluated at
                              2025-06-16T00:00:00.000Z
                              source model id
                              o3
                              confidence lower
                              0.27362386201817535
                              confidence upper
                              0.3903761379818247
                              Original result ↗

                              Kaggle Benchmarks

                              31.6%

                              source id
                              kaggleBenchmarks
                              leaderboard name
                              MultiLoKo Malay · EM Malay
                              dimension
                              reasoning
                              metric
                              MultiLoKo Malay · EM Malay
                              rank
                              4
                              aggregation scope
                              kaggle
                              true
                              status
                              published
                              license
                              Apache-2.0
                              direction
                              higherIsBetter
                              conditions
                              task
                              id
                              45
                              name
                              EM Malay
                              version
                              1
                              sort_order
                              DESCENDING
                              display_type
                              percentages
                              benchmark
                              id
                              68
                              name
                              MultiLoKo Malay
                              type
                              INDIVIDUAL
                              owner
                              metaresearch
                              version
                              1
                              categories
                              • multilingual and cross-lingual capabilities
                              • general knowledge and reasoning
                              version_id
                              54
                              maintainer_tier
                              Kaggle-maintained
                              maintenance_level
                              KAGGLE_MAINTAINED
                              candidate
                              candidate_class
                              modelVersion
                              record_kind
                              aggregate
                              attribution
                              Kaggle · metaresearch · MultiLoKo Malay
                              license_url
                              https://www.apache.org/licenses/LICENSE-2.0
                              metric_kind
                              MultiLoKo Malay · EM Malay
                              rank_method
                              competition rank among reported evaluations in identical conditions, including unlinked observations
                              source_file
                              54/leaderboard.json
                              source_page
                              https://www.kaggle.com/benchmarks/metaresearch/multiloko-malay/versions/1
                              measurements
                              • name
                                EM Malay
                                unit
                                percent
                                scale
                                1
                                metric
                                MultiLoKo Malay · EM Malay
                                result
                                resultCase
                                numericResult
                                booleanResult
                                false
                                numericResult
                                value
                                0.316
                                standardDeviation
                                0
                                confidenceInterval
                                0.0576301405634277
                                hasStandardDeviation
                                false
                                hasConfidenceInterval
                                true
                                confidenceIntervalNullable
                                0.0576301405634277
                                hasUnevenConfidenceInterval
                                false
                                taskVersionId
                                0
                                evaluationDate
                                2025-06-16T00:00:00Z
                                hasBooleanResult
                                false
                                hasNumericResult
                                true
                                hasTaskVersionId
                                false
                                hasEvaluationDate
                                true
                                numericResultNullable
                                value
                                0.316
                                standardDeviation
                                0
                                confidenceInterval
                                0.0576301405634277
                                hasStandardDeviation
                                false
                                hasConfidenceInterval
                                true
                                confidenceIntervalNullable
                                0.0576301405634277
                                hasUnevenConfidenceInterval
                                false
                                hasNumericResultPublic
                                false
                                customAdditionalResults
                                  hasNumericResultPrivate
                                  false
                                  task_id
                                  45
                                  sort_order
                                  DESCENDING
                                  source_name
                                  EM Malay
                                  display_type
                                  percentages
                                  task_version
                                  1
                                  task_display_type
                                  PERCENTAGES
                                  normalization_status
                                  normalized
                                published_at
                                2025-07-15T11:49:31.790Z
                                candidate_class
                                modelVersion
                                source_revision
                                b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                publisher_anchor
                                openai/o3-2025-04-16
                                metric_population
                                51
                                benchmark_evaluation
                                true
                                task_metadata_source
                                benchmarkVersion
                                candidate_metadata_source
                                benchmarkVersionMappings
                                evaluated at
                                2025-06-16T00:00:00.000Z
                                source model id
                                o3
                                confidence lower
                                0.2583698594365723
                                confidence upper
                                0.3736301405634277
                                Original result ↗

                                Kaggle Benchmarks

                                30.4%

                                source id
                                kaggleBenchmarks
                                leaderboard name
                                MultiLoKo Thai · EM Thai
                                dimension
                                reasoning
                                metric
                                MultiLoKo Thai · EM Thai
                                rank
                                4
                                aggregation scope
                                kaggle
                                true
                                status
                                published
                                license
                                Apache-2.0
                                direction
                                higherIsBetter
                                conditions
                                task
                                id
                                55
                                name
                                EM Thai
                                version
                                1
                                sort_order
                                DESCENDING
                                display_type
                                percentages
                                benchmark
                                id
                                78
                                name
                                MultiLoKo Thai
                                type
                                INDIVIDUAL
                                owner
                                metaresearch
                                version
                                1
                                categories
                                • multilingual and cross-lingual capabilities
                                • general knowledge and reasoning
                                version_id
                                64
                                maintainer_tier
                                Kaggle-maintained
                                maintenance_level
                                KAGGLE_MAINTAINED
                                candidate
                                candidate_class
                                modelVersion
                                record_kind
                                aggregate
                                attribution
                                Kaggle · metaresearch · MultiLoKo Thai
                                license_url
                                https://www.apache.org/licenses/LICENSE-2.0
                                metric_kind
                                MultiLoKo Thai · EM Thai
                                rank_method
                                competition rank among reported evaluations in identical conditions, including unlinked observations
                                source_file
                                64/leaderboard.json
                                source_page
                                https://www.kaggle.com/benchmarks/metaresearch/multiloko-thai/versions/1
                                measurements
                                • name
                                  EM Thai
                                  unit
                                  percent
                                  scale
                                  1
                                  metric
                                  MultiLoKo Thai · EM Thai
                                  result
                                  resultCase
                                  numericResult
                                  booleanResult
                                  false
                                  numericResult
                                  value
                                  0.304
                                  standardDeviation
                                  0
                                  confidenceInterval
                                  0.0570189871224575
                                  hasStandardDeviation
                                  false
                                  hasConfidenceInterval
                                  true
                                  confidenceIntervalNullable
                                  0.0570189871224575
                                  hasUnevenConfidenceInterval
                                  false
                                  taskVersionId
                                  0
                                  evaluationDate
                                  2025-06-16T00:00:00Z
                                  hasBooleanResult
                                  false
                                  hasNumericResult
                                  true
                                  hasTaskVersionId
                                  false
                                  hasEvaluationDate
                                  true
                                  numericResultNullable
                                  value
                                  0.304
                                  standardDeviation
                                  0
                                  confidenceInterval
                                  0.0570189871224575
                                  hasStandardDeviation
                                  false
                                  hasConfidenceInterval
                                  true
                                  confidenceIntervalNullable
                                  0.0570189871224575
                                  hasUnevenConfidenceInterval
                                  false
                                  hasNumericResultPublic
                                  false
                                  customAdditionalResults
                                    hasNumericResultPrivate
                                    false
                                    task_id
                                    55
                                    sort_order
                                    DESCENDING
                                    source_name
                                    EM Thai
                                    display_type
                                    percentages
                                    task_version
                                    1
                                    task_display_type
                                    PERCENTAGES
                                    normalization_status
                                    normalized
                                  published_at
                                  2025-07-15T11:49:31.790Z
                                  candidate_class
                                  modelVersion
                                  source_revision
                                  b32d007df5acbddafca23c7dd2beba8221c519cf7dd3445a1d7bc28740ffd0d8
                                  publisher_anchor
                                  openai/o3-2025-04-16
                                  metric_population
                                  51
                                  benchmark_evaluation
                                  true
                                  task_metadata_source
                                  benchmarkVersion
                                  candidate_metadata_source
                                  benchmarkVersionMappings
                                  evaluated at
                                  2025-06-16T00:00:00.000Z
                                  source model id
                                  o3
                                  confidence lower
                                  0.24698101287754248
                                  confidence upper
                                  0.3610189871224575
                                  Original result ↗

                                  Published prices

                                  1–8 / 8

                                  openai/o3 ↗

                                  $0.50

                                  Inference provider
                                  openRouter
                                  Platform
                                  OpenRouter
                                  Unit
                                  cache_read_tokens_per_million
                                  Currency
                                  USD
                                  Billing conditions

                                  openai/o3 ↗

                                  $2.00

                                  Inference provider
                                  openRouter
                                  Platform
                                  OpenRouter
                                  Unit
                                  input_tokens_per_million
                                  Currency
                                  USD
                                  Billing conditions

                                  openai/o3 ↗

                                  $8.00

                                  Inference provider
                                  openRouter
                                  Platform
                                  OpenRouter
                                  Unit
                                  output_tokens_per_million
                                  Currency
                                  USD
                                  Billing conditions

                                  openai/o3 ↗

                                  $0.01

                                  Inference provider
                                  openRouter
                                  Platform
                                  OpenRouter
                                  Unit
                                  web_search
                                  Currency
                                  USD
                                  Billing conditions

                                  openai/o3:batch ↗

                                  $0.25

                                  Inference provider
                                  openRouter
                                  Platform
                                  OpenRouter
                                  Unit
                                  cache_read_tokens_per_million
                                  Currency
                                  USD
                                  Billing conditions
                                  mode
                                  batch

                                  openai/o3:batch ↗

                                  $1.00

                                  Inference provider
                                  openRouter
                                  Platform
                                  OpenRouter
                                  Unit
                                  input_tokens_per_million
                                  Currency
                                  USD
                                  Billing conditions
                                  mode
                                  batch

                                  openai/o3:batch ↗

                                  $4.00

                                  Inference provider
                                  openRouter
                                  Platform
                                  OpenRouter
                                  Unit
                                  output_tokens_per_million
                                  Currency
                                  USD
                                  Billing conditions
                                  mode
                                  batch

                                  openai/o3:batch ↗

                                  $0.01

                                  Inference provider
                                  openRouter
                                  Platform
                                  OpenRouter
                                  Unit
                                  web_search
                                  Currency
                                  USD
                                  Billing conditions
                                  mode
                                  batch

                                  Provider specifications & access

                                  1–1 / 1

                                  OpenAI o3 · azure-openai v2025-04-16 ↗

                                  source model id
                                  azure-openai/o3@2025-04-16
                                  hosting
                                  foundry-managed-compute
                                  platforms
                                  • Microsoft Foundry
                                  license
                                  custom
                                  context length
                                  200000
                                  max output tokens
                                  100000
                                  architecture
                                  input_modalities
                                  • text
                                  • image
                                  output_modalities
                                  • text
                                  capabilities
                                  • reasoning
                                  • toolCalling

                                  Safety assessments

                                  1–1 / 1

                                  Foundation Model Transparency Index 2025 ↗

                                  35 / 100

                                  source id
                                  stanfordFMTI
                                  canonical model id
                                  openai/o3-2025-04-16
                                  taxonomy
                                  transparency
                                  hazard category
                                  transparency
                                  hazard name
                                  Organization transparency
                                  policy area
                                  OpenAI
                                  assessment type
                                  organization-level disclosure
                                  metric
                                  disclosure_score
                                  result label
                                  35 / 100
                                  sample size
                                  100
                                  updated at
                                  2026-11-09T18:08:05.227Z

                                  Source specification observations

                                  1–9 / 9

                                  azureFoundry ↗

                                  field
                                  context_length
                                  value
                                  200000
                                  normalized value
                                  200000
                                  scope
                                  source-model
                                  source id
                                  azureFoundry
                                  confidence
                                  1

                                  openRouter ↗

                                  field
                                  context_length
                                  value
                                  200000
                                  normalized value
                                  200000
                                  scope
                                  source-model
                                  source id
                                  openRouter
                                  confidence
                                  1

                                  openRouter ↗

                                  field
                                  context_length
                                  value
                                  200000
                                  normalized value
                                  200000
                                  scope
                                  source-model
                                  source id
                                  openRouter
                                  confidence
                                  1

                                  openRouter ↗

                                  field
                                  knowledge_cutoff
                                  reported field
                                  knowledge_cutoff
                                  value
                                  2024-06-30
                                  normalized value
                                  2024-06-30
                                  precision
                                  day
                                  scope
                                  source-model
                                  source id
                                  openRouter
                                  confidence
                                  1

                                  openRouter ↗

                                  field
                                  knowledge_cutoff
                                  reported field
                                  knowledge_cutoff
                                  value
                                  2024-06-30
                                  normalized value
                                  2024-06-30
                                  precision
                                  day
                                  scope
                                  source-model
                                  source id
                                  openRouter
                                  confidence
                                  1

                                  azureFoundry ↗

                                  field
                                  max_output_tokens
                                  value
                                  100000
                                  normalized value
                                  100000
                                  scope
                                  source-model
                                  source id
                                  azureFoundry
                                  confidence
                                  1

                                  openRouter ↗

                                  field
                                  max_output_tokens
                                  value
                                  100000
                                  normalized value
                                  100000
                                  scope
                                  source-model
                                  source id
                                  openRouter
                                  confidence
                                  1

                                  openRouter ↗

                                  field
                                  max_output_tokens
                                  value
                                  100000
                                  normalized value
                                  100000
                                  scope
                                  source-model
                                  source id
                                  openRouter
                                  confidence
                                  1

                                  azureFoundry ↗

                                  field
                                  knowledge_cutoff
                                  reported field
                                  training_data_cutoff
                                  value
                                  May 2024
                                  normalized value
                                  2024-05
                                  precision
                                  month
                                  scope
                                  source-model
                                  source id
                                  azureFoundry
                                  confidence
                                  1

                                  Original sources

                                  1–13 / 13

                                  OpenAI · Model reference ↗

                                  Microsoft Foundry · Model page ↗

                                  OpenRouter · Model page ↗

                                  OpenRouter · Model page ↗

                                  Microsoft Foundry · Model documentation ↗

                                  GitHub · Catalog ↗

                                  GitHub · Catalog ↗

                                  Kaggle · Catalog ↗

                                  LMArena · Catalog ↗

                                  Microsoft Foundry · Catalog ↗

                                  OpenRouter · Catalog ↗

                                  gorilla.cs.berkeley.edu · Catalog ↗

                                  www.swebench.com · Catalog ↗

                                  Source links

                                  1–10 / 10

                                  Explore more models

                                  Models

                                  Loading
                                  EVIDENCE / COMPARISON

                                  Benchmarks

                                  Explore every benchmark

                                  Loading

                                  These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                                  Install TensorTrace