Meta: Llama 3.3 70B Instruct

The Meta Llama 3.3 multilingual large language model (LLM) is a pretrained and instruction tuned generative model in 70B (text in/text out). The Llama 3.3 instruction tuned text only model is optimiz…

meta-llama/llama-3.3-70b-instruct

The Meta Llama 3.3 multilingual large language model (LLM) is a pretrained and instruction tuned generative model in 70B (text in/text out). The Llama 3.3 instruction tuned text only model is optimized for multilingual dialogue use cases and outperforms many of the available open source and closed chat models on common industry benchmarks.

Supported languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.

[Model Card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)

Specifications & capabilities

Input
text
Output
text
Context window · tokens
131072
Knowledge cutoff
2023-12-31
Published model ID
Llama-3.3-70B-Instruct
Capabilities
promptCaching, structuredOutputs, toolCalling
Description source ↗

Evaluations

21–40 / 41

OpenRouter Benchmarks

48.437%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · CoreWeave
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
26
sample size
1386
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
CoreWeave
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:04:29.124Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1386
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.48436714285714283
accuracy_std_dev
0.012361779684964115
cost_per_task_usd
0.0026505512265512263
output_tokens_per_task
2925.426406926407
latency_seconds_per_task
49.856584415584415
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
27
benchmark_evaluation
true
openrouter_permaslug
meta-llama/llama-3.3-70b-instruct
source model id
meta-llama/llama-3.3-70b-instruct
Original result ↗

OpenRouter Benchmarks

47.715%

source id
openRouterBenchmarks
leaderboard name
GPQA Diamond · Accuracy · Parasail
dimension
knowledge
metric
GPQA Diamond · Accuracy
rank
33
sample size
1386
aggregation scope
status
published
harness
url
https://github.com/OpenRouterTeam/benchmark-harness
name
OpenRouter benchmark harness
direction
higherIsBetter
run_count
7
conditions
window
newest 90 days; successful visible nonzero runs
dataset
benchmark-leaderboard
benchmark
id
gpqa_diamond
name
GPQA Diamond
record_kind
aggregate
provider_name
Parasail
attribution
Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
last_run_at
2026-10-31T16:36:11.519Z
license_url
https://creativecommons.org/licenses/by/4.0/
methodology
Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
metric_kind
GPQA Diamond · Accuracy
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
sample_size
1386
source_file
benchmark-leaderboard.json
source_page
https://openrouter.ai/benchmarks/gpqa-diamond
measurements
accuracy
0.47715228571428564
accuracy_std_dev
0.014837295958055795
cost_per_task_usd
0.0013270916305916304
output_tokens_per_task
2383.0901875901877
latency_seconds_per_task
44.547831168831166
published_at
2026-11-16T01:46:02.653Z
snapshot_date
2026-11-16
source_revision
7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
transformations
TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
metric_population
36
benchmark_evaluation
true
openrouter_permaslug
meta-llama/llama-3.3-70b-instruct
source model id
meta-llama/llama-3.3-70b-instruct
Original result ↗

Open LLM Leaderboard v2

44.847%

source id
openLLMLeaderboardV2
leaderboard name
Open LLM v2 Average
dimension
general
metric
Normalized six-task average
rank
40
aggregation scope
suite
Open LLM v2
co2_kg
76.55907410745293
status
archived
precision
bfloat16
task_count
6
metric_kind
Normalized six-task average
rank_method
Derived competition rank over published evaluations with reported scores, including variants and source flags
weight_type
Original
published_at
2025-03-20T12:17:27Z
chat_template
true
suite_version
2
source_flagged
false
missing_metrics
    dataset_revision
    9c09a7cae43334062a82cb164f2ef255013dafa2
    metric_population
    4576
    ranking_population
    4576
    source_flag_anomaly
    false
    source model id
    meta-llama/Llama-3.3-70B-Instruct
    Original result ↗

    OpenRouter Benchmarks

    38.148%

    source id
    openRouterBenchmarks
    leaderboard name
    tau2-bench Verified Airline · Accuracy · Novita
    dimension
    agentic
    metric
    tau2-bench Verified Airline · Accuracy
    rank
    50
    sample size
    45
    aggregation scope
    status
    published
    harness
    url
    https://github.com/OpenRouterTeam/benchmark-harness
    name
    OpenRouter benchmark harness
    direction
    higherIsBetter
    run_count
    1
    conditions
    window
    newest 90 days; successful visible nonzero runs
    dataset
    benchmark-leaderboard
    benchmark
    id
    tau_bench_verified_airline
    name
    tau2-bench Verified Airline
    record_kind
    aggregate
    provider_name
    Novita
    attribution
    Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
    last_run_at
    2026-10-03T16:15:51.302Z
    license_url
    https://creativecommons.org/licenses/by/4.0/
    methodology
    Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
    metric_kind
    tau2-bench Verified Airline · Accuracy
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    sample_size
    45
    source_file
    benchmark-leaderboard.json
    source_page
    https://openrouter.ai/benchmarks/tau2-bench-airline
    measurements
    accuracy
    0.38148099999999996
    cost_per_task_usd
    0.008997444444444445
    output_tokens_per_task
    3561.511111111111
    latency_seconds_per_task
    232.9572888888889
    published_at
    2026-11-16T01:46:02.653Z
    snapshot_date
    2026-11-16
    source_revision
    7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
    transformations
    TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
    metric_population
    52
    benchmark_evaluation
    true
    openrouter_permaslug
    meta-llama/llama-3.3-70b-instruct
    source model id
    meta-llama/llama-3.3-70b-instruct
    Original result ↗

    OpenRouter Benchmarks

    47.234%

    source id
    openRouterBenchmarks
    leaderboard name
    GPQA Diamond · Accuracy · Novita
    dimension
    knowledge
    metric
    GPQA Diamond · Accuracy
    rank
    55
    sample size
    1386
    aggregation scope
    status
    published
    harness
    url
    https://github.com/OpenRouterTeam/benchmark-harness
    name
    OpenRouter benchmark harness
    direction
    higherIsBetter
    run_count
    7
    conditions
    window
    newest 90 days; successful visible nonzero runs
    dataset
    benchmark-leaderboard
    benchmark
    id
    gpqa_diamond
    name
    GPQA Diamond
    record_kind
    aggregate
    provider_name
    Novita
    attribution
    Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
    last_run_at
    2026-10-31T16:31:54.106Z
    license_url
    https://creativecommons.org/licenses/by/4.0/
    methodology
    Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
    metric_kind
    GPQA Diamond · Accuracy
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    sample_size
    1386
    source_file
    benchmark-leaderboard.json
    source_page
    https://openrouter.ai/benchmarks/gpqa-diamond
    measurements
    accuracy
    0.47234228571428577
    accuracy_std_dev
    0.009696421014466937
    cost_per_task_usd
    0.0010906248196248197
    output_tokens_per_task
    2454.1998556998556
    latency_seconds_per_task
    63.82189610389611
    published_at
    2026-11-16T01:46:02.653Z
    snapshot_date
    2026-11-16
    source_revision
    7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
    transformations
    TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
    metric_population
    59
    benchmark_evaluation
    true
    openrouter_permaslug
    meta-llama/llama-3.3-70b-instruct
    source model id
    meta-llama/llama-3.3-70b-instruct
    Original result ↗

    OpenRouter Benchmarks

    30.556%

    source id
    openRouterBenchmarks
    leaderboard name
    tau2-bench Verified Airline · Accuracy · DeepInfra
    dimension
    agentic
    metric
    tau2-bench Verified Airline · Accuracy
    rank
    62
    sample size
    96
    aggregation scope
    status
    published
    harness
    url
    https://github.com/OpenRouterTeam/benchmark-harness
    name
    OpenRouter benchmark harness
    direction
    higherIsBetter
    run_count
    2
    conditions
    window
    newest 90 days; successful visible nonzero runs
    dataset
    benchmark-leaderboard
    benchmark
    id
    tau_bench_verified_airline
    name
    tau2-bench Verified Airline
    record_kind
    aggregate
    provider_name
    DeepInfra
    attribution
    Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
    last_run_at
    2026-08-29T16:43:42.449Z
    license_url
    https://creativecommons.org/licenses/by/4.0/
    methodology
    Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
    metric_kind
    tau2-bench Verified Airline · Accuracy
    rank_method
    competition rank among reported evaluations in identical conditions, including unlinked observations
    sample_size
    96
    source_file
    benchmark-leaderboard.json
    source_page
    https://openrouter.ai/benchmarks/tau2-bench-airline
    measurements
    accuracy
    0.3055554583333333
    accuracy_std_dev
    0.054493000000000014
    cost_per_task_usd
    0.024076770833333334
    output_tokens_per_task
    2060.5833333333335
    latency_seconds_per_task
    191.48885416666667
    published_at
    2026-11-16T01:46:02.653Z
    snapshot_date
    2026-11-16
    source_revision
    7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
    transformations
    TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
    metric_population
    63
    benchmark_evaluation
    true
    openrouter_permaslug
    meta-llama/llama-3.3-70b-instruct
    source model id
    meta-llama/llama-3.3-70b-instruct
    Original result ↗

    Open LLM Leaderboard v2

    56.561%

    source id
    openLLMLeaderboardV2
    leaderboard name
    Open LLM v2 BBH · Normalized
    dimension
    reasoning
    metric
    Normalized score
    rank
    67
    aggregation scope
    suite
    Open LLM v2
    co2_kg
    76.55907410745293
    status
    archived
    precision
    bfloat16
    task_count
    6
    metric_kind
    Normalized score
    rank_method
    Derived competition rank over published evaluations with reported scores, including variants and source flags
    weight_type
    Original
    published_at
    2025-03-20T12:17:27Z
    chat_template
    true
    suite_version
    2
    source_flagged
    false
    missing_metrics
      dataset_revision
      9c09a7cae43334062a82cb164f2ef255013dafa2
      metric_population
      4576
      ranking_population
      4576
      source_flag_anomaly
      false
      source model id
      meta-llama/Llama-3.3-70B-Instruct
      Original result ↗

      OpenRouter Benchmarks

      46.412%

      source id
      openRouterBenchmarks
      leaderboard name
      GPQA Diamond · Accuracy · DeepInfra
      dimension
      knowledge
      metric
      GPQA Diamond · Accuracy
      rank
      67
      sample size
      1584
      aggregation scope
      status
      published
      harness
      url
      https://github.com/OpenRouterTeam/benchmark-harness
      name
      OpenRouter benchmark harness
      direction
      higherIsBetter
      run_count
      8
      conditions
      window
      newest 90 days; successful visible nonzero runs
      dataset
      benchmark-leaderboard
      benchmark
      id
      gpqa_diamond
      name
      GPQA Diamond
      record_kind
      aggregate
      provider_name
      DeepInfra
      attribution
      Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
      last_run_at
      2026-10-31T16:02:14.361Z
      license_url
      https://creativecommons.org/licenses/by/4.0/
      methodology
      Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
      metric_kind
      GPQA Diamond · Accuracy
      rank_method
      competition rank among reported evaluations in identical conditions, including unlinked observations
      sample_size
      1584
      source_file
      benchmark-leaderboard.json
      source_page
      https://openrouter.ai/benchmarks/gpqa-diamond
      measurements
      accuracy
      0.46412025
      accuracy_std_dev
      0.011095032793439591
      cost_per_task_usd
      0.0007977228535353536
      output_tokens_per_task
      2242.354797979798
      latency_seconds_per_task
      144.4113554292929
      published_at
      2026-11-16T01:46:02.653Z
      snapshot_date
      2026-11-16
      source_revision
      7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
      transformations
      TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
      metric_population
      71
      benchmark_evaluation
      true
      openrouter_permaslug
      meta-llama/llama-3.3-70b-instruct
      source model id
      meta-llama/llama-3.3-70b-instruct
      Original result ↗

      Open LLM Leaderboard v2

      69.193%

      source id
      openLLMLeaderboardV2
      leaderboard name
      Open LLM v2 BBH · Raw
      dimension
      reasoning
      metric
      Raw accuracy
      rank
      96
      aggregation scope
      suite
      Open LLM v2
      co2_kg
      76.55907410745293
      status
      archived
      precision
      bfloat16
      task_count
      6
      metric_kind
      Raw accuracy
      rank_method
      Derived competition rank over published evaluations with reported scores, including variants and source flags
      weight_type
      Original
      published_at
      2025-03-20T12:17:27Z
      chat_template
      true
      suite_version
      2
      source_flagged
      false
      missing_metrics
        dataset_revision
        9c09a7cae43334062a82cb164f2ef255013dafa2
        metric_population
        4576
        ranking_population
        4576
        source_flag_anomaly
        false
        source model id
        meta-llama/Llama-3.3-70B-Instruct
        Original result ↗

        OpenRouter Benchmarks

        33.622%

        source id
        openRouterBenchmarks
        leaderboard name
        tau2-bench Verified Airline · Accuracy · auto-routing
        dimension
        agentic
        metric
        tau2-bench Verified Airline · Accuracy
        rank
        124
        sample size
        346
        aggregation scope
        status
        published
        harness
        url
        https://github.com/OpenRouterTeam/benchmark-harness
        name
        OpenRouter benchmark harness
        direction
        higherIsBetter
        run_count
        7
        conditions
        window
        newest 90 days; successful visible nonzero runs
        dataset
        benchmark-leaderboard
        benchmark
        id
        tau_bench_verified_airline
        name
        tau2-bench Verified Airline
        record_kind
        aggregate
        provider_name
        auto-routing
        attribution
        Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
        last_run_at
        2026-10-31T19:07:35.559Z
        license_url
        https://creativecommons.org/licenses/by/4.0/
        methodology
        Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
        metric_kind
        tau2-bench Verified Airline · Accuracy
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        sample_size
        346
        source_file
        benchmark-leaderboard.json
        source_page
        https://openrouter.ai/benchmarks/tau2-bench-airline
        measurements
        accuracy
        0.33622356069364157
        accuracy_std_dev
        0.046445988932944285
        cost_per_task_usd
        0.057168858381502895
        output_tokens_per_task
        1325.586705202312
        latency_seconds_per_task
        73.26867919075144
        published_at
        2026-11-16T01:46:02.653Z
        snapshot_date
        2026-11-16
        source_revision
        7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
        transformations
        TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
        metric_population
        128
        benchmark_evaluation
        true
        openrouter_permaslug
        meta-llama/llama-3.3-70b-instruct
        source model id
        meta-llama/llama-3.3-70b-instruct
        Original result ↗

        OpenRouter Benchmarks

        47.547%

        source id
        openRouterBenchmarks
        leaderboard name
        GPQA Diamond · Accuracy · auto-routing
        dimension
        knowledge
        metric
        GPQA Diamond · Accuracy
        rank
        126
        sample size
        1386
        aggregation scope
        status
        published
        harness
        url
        https://github.com/OpenRouterTeam/benchmark-harness
        name
        OpenRouter benchmark harness
        direction
        higherIsBetter
        run_count
        7
        conditions
        window
        newest 90 days; successful visible nonzero runs
        dataset
        benchmark-leaderboard
        benchmark
        id
        gpqa_diamond
        name
        GPQA Diamond
        record_kind
        aggregate
        provider_name
        auto-routing
        attribution
        Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0: reuse and republish with attribution to OpenRouter.
        last_run_at
        2026-10-31T16:23:41.139Z
        license_url
        https://creativecommons.org/licenses/by/4.0/
        methodology
        Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.
        metric_kind
        GPQA Diamond · Accuracy
        rank_method
        competition rank among reported evaluations in identical conditions, including unlinked observations
        sample_size
        1386
        source_file
        benchmark-leaderboard.json
        source_page
        https://openrouter.ai/benchmarks/gpqa-diamond
        measurements
        accuracy
        0.4754687142857143
        accuracy_std_dev
        0.010214854935188212
        cost_per_task_usd
        0.0013394170274170275
        output_tokens_per_task
        2360.8838383838383
        latency_seconds_per_task
        85.51639105339105
        published_at
        2026-11-16T01:46:02.653Z
        snapshot_date
        2026-11-16
        source_revision
        7ccaf17664252a3053dcb7df8ad0a46e72954a3b4524ddc1c8a7ef8a41b15568
        transformations
        TensorTrace preserves source results, converts latency to seconds and derives ranks within matching conditions. Individual runs have no derived rank.
        metric_population
        132
        benchmark_evaluation
        true
        openrouter_permaslug
        meta-llama/llama-3.3-70b-instruct
        source model id
        meta-llama/llama-3.3-70b-instruct
        Original result ↗

        Open LLM Leaderboard v2

        48.338%

        source id
        openLLMLeaderboardV2
        leaderboard name
        Open LLM v2 MATH Lvl 5 · Normalized
        dimension
        math
        metric
        Normalized score
        rank
        190
        aggregation scope
        suite
        Open LLM v2
        co2_kg
        76.55907410745293
        status
        archived
        precision
        bfloat16
        task_count
        6
        metric_kind
        Normalized score
        rank_method
        Derived competition rank over published evaluations with reported scores, including variants and source flags
        weight_type
        Original
        published_at
        2025-03-20T12:17:27Z
        chat_template
        true
        suite_version
        2
        source_flagged
        false
        missing_metrics
          dataset_revision
          9c09a7cae43334062a82cb164f2ef255013dafa2
          metric_population
          4576
          ranking_population
          4576
          source_flag_anomaly
          false
          source model id
          meta-llama/Llama-3.3-70B-Instruct
          Original result ↗

          Open LLM Leaderboard v2

          48.338%

          source id
          openLLMLeaderboardV2
          leaderboard name
          Open LLM v2 MATH Lvl 5 · Raw
          dimension
          math
          metric
          Raw accuracy
          rank
          190
          aggregation scope
          suite
          Open LLM v2
          co2_kg
          76.55907410745293
          status
          archived
          precision
          bfloat16
          task_count
          6
          metric_kind
          Raw accuracy
          rank_method
          Derived competition rank over published evaluations with reported scores, including variants and source flags
          weight_type
          Original
          published_at
          2025-03-20T12:17:27Z
          chat_template
          true
          suite_version
          2
          source_flagged
          false
          missing_metrics
            dataset_revision
            9c09a7cae43334062a82cb164f2ef255013dafa2
            metric_population
            4576
            ranking_population
            4576
            source_flag_anomaly
            false
            source model id
            meta-llama/Llama-3.3-70B-Instruct
            Original result ↗

            Open LLM Leaderboard v2

            53.316%

            source id
            openLLMLeaderboardV2
            leaderboard name
            Open LLM v2 MMLU-PRO · Raw
            dimension
            knowledge
            metric
            Raw accuracy
            rank
            299
            aggregation scope
            suite
            Open LLM v2
            co2_kg
            76.55907410745293
            status
            archived
            precision
            bfloat16
            task_count
            6
            metric_kind
            Raw accuracy
            rank_method
            Derived competition rank over published evaluations with reported scores, including variants and source flags
            weight_type
            Original
            published_at
            2025-03-20T12:17:27Z
            chat_template
            true
            suite_version
            2
            source_flagged
            false
            missing_metrics
              dataset_revision
              9c09a7cae43334062a82cb164f2ef255013dafa2
              metric_population
              4576
              ranking_population
              4576
              source_flag_anomaly
              false
              source model id
              meta-llama/Llama-3.3-70B-Instruct
              Original result ↗

              Open LLM Leaderboard v2

              48.129%

              source id
              openLLMLeaderboardV2
              leaderboard name
              Open LLM v2 MMLU-PRO · Normalized
              dimension
              knowledge
              metric
              Normalized score
              rank
              299
              aggregation scope
              suite
              Open LLM v2
              co2_kg
              76.55907410745293
              status
              archived
              precision
              bfloat16
              task_count
              6
              metric_kind
              Normalized score
              rank_method
              Derived competition rank over published evaluations with reported scores, including variants and source flags
              weight_type
              Original
              published_at
              2025-03-20T12:17:27Z
              chat_template
              true
              suite_version
              2
              source_flagged
              false
              missing_metrics
                dataset_revision
                9c09a7cae43334062a82cb164f2ef255013dafa2
                metric_population
                4576
                ranking_population
                4576
                source_flag_anomaly
                false
                source model id
                meta-llama/Llama-3.3-70B-Instruct
                Original result ↗

                Open LLM Leaderboard v2

                15.566%

                source id
                openLLMLeaderboardV2
                leaderboard name
                Open LLM v2 MUSR · Normalized
                dimension
                reasoning
                metric
                Normalized score
                rank
                754
                aggregation scope
                suite
                Open LLM v2
                co2_kg
                76.55907410745293
                status
                archived
                precision
                bfloat16
                task_count
                6
                metric_kind
                Normalized score
                rank_method
                Derived competition rank over published evaluations with reported scores, including variants and source flags
                weight_type
                Original
                published_at
                2025-03-20T12:17:27Z
                chat_template
                true
                suite_version
                2
                source_flagged
                false
                missing_metrics
                  dataset_revision
                  9c09a7cae43334062a82cb164f2ef255013dafa2
                  metric_population
                  4576
                  ranking_population
                  4576
                  source_flag_anomaly
                  false
                  source model id
                  meta-llama/Llama-3.3-70B-Instruct
                  Original result ↗

                  Open LLM Leaderboard v2

                  44.613%

                  source id
                  openLLMLeaderboardV2
                  leaderboard name
                  Open LLM v2 MUSR · Raw
                  dimension
                  reasoning
                  metric
                  Raw accuracy
                  rank
                  837
                  aggregation scope
                  suite
                  Open LLM v2
                  co2_kg
                  76.55907410745293
                  status
                  archived
                  precision
                  bfloat16
                  task_count
                  6
                  metric_kind
                  Raw accuracy
                  rank_method
                  Derived competition rank over published evaluations with reported scores, including variants and source flags
                  weight_type
                  Original
                  published_at
                  2025-03-20T12:17:27Z
                  chat_template
                  true
                  suite_version
                  2
                  source_flagged
                  false
                  missing_metrics
                    dataset_revision
                    9c09a7cae43334062a82cb164f2ef255013dafa2
                    metric_population
                    4576
                    ranking_population
                    4576
                    source_flag_anomaly
                    false
                    source model id
                    meta-llama/Llama-3.3-70B-Instruct
                    Original result ↗

                    Open LLM Leaderboard v2

                    32.886%

                    source id
                    openLLMLeaderboardV2
                    leaderboard name
                    Open LLM v2 GPQA · Raw
                    dimension
                    knowledge
                    metric
                    Raw accuracy
                    rank
                    899
                    aggregation scope
                    suite
                    Open LLM v2
                    co2_kg
                    76.55907410745293
                    status
                    archived
                    precision
                    bfloat16
                    task_count
                    6
                    metric_kind
                    Raw accuracy
                    rank_method
                    Derived competition rank over published evaluations with reported scores, including variants and source flags
                    weight_type
                    Original
                    published_at
                    2025-03-20T12:17:27Z
                    chat_template
                    true
                    suite_version
                    2
                    source_flagged
                    false
                    missing_metrics
                      dataset_revision
                      9c09a7cae43334062a82cb164f2ef255013dafa2
                      metric_population
                      4576
                      ranking_population
                      4576
                      source_flag_anomaly
                      false
                      source model id
                      meta-llama/Llama-3.3-70B-Instruct
                      Original result ↗

                      Open LLM Leaderboard v2

                      10.515%

                      source id
                      openLLMLeaderboardV2
                      leaderboard name
                      Open LLM v2 GPQA · Normalized
                      dimension
                      knowledge
                      metric
                      Normalized score
                      rank
                      899
                      aggregation scope
                      suite
                      Open LLM v2
                      co2_kg
                      76.55907410745293
                      status
                      archived
                      precision
                      bfloat16
                      task_count
                      6
                      metric_kind
                      Normalized score
                      rank_method
                      Derived competition rank over published evaluations with reported scores, including variants and source flags
                      weight_type
                      Original
                      published_at
                      2025-03-20T12:17:27Z
                      chat_template
                      true
                      suite_version
                      2
                      source_flagged
                      false
                      missing_metrics
                        dataset_revision
                        9c09a7cae43334062a82cb164f2ef255013dafa2
                        metric_population
                        4576
                        ranking_population
                        4576
                        source_flag_anomaly
                        false
                        source model id
                        meta-llama/Llama-3.3-70B-Instruct
                        Original result ↗

                        Published prices

                        1–5 / 5

                        Provider specifications & access

                        1–3 / 3

                        Llama-3.3-70B-Instruct ↗

                        source model id
                        meta-llama/Llama-3.3-70B-Instruct
                        hosting
                        siliconflow-hosted
                        platforms
                        • SiliconFlow
                        architecture
                        input_modalities
                        • text
                        output_modalities
                        • text
                        capabilities
                        • structuredOutputs
                        • toolCalling

                        Llama-3-3-70B-Instruct ↗

                        source model id
                        publishers/hf-meta-llama/models/llama-3.3-70b-instruct@001
                        hosting
                        vertex-self-deployed
                        platforms
                        • Google Cloud Vertex AI
                        architecture
                        input_modalities
                        • text
                        output_modalities
                        • text
                        capabilities

                          Llama-3-3-70B-Instruct ↗

                          source model id
                          publishers/meta/models/llama3-3@llama-3.3-70b-instruct
                          hosting
                          vertex-self-deployed
                          platforms
                          • Google Cloud Vertex AI
                          architecture
                          input_modalities
                          • text
                          output_modalities
                          • text
                          capabilities

                            Safety assessments

                            1–1 / 1

                            Foundation Model Transparency Index 2025 ↗

                            31 / 100

                            source id
                            stanfordFMTI
                            canonical model id
                            meta-llama/llama-3.3-70b-instruct
                            taxonomy
                            transparency
                            hazard category
                            transparency
                            hazard name
                            Organization transparency
                            policy area
                            Meta
                            assessment type
                            organization-level disclosure
                            metric
                            disclosure_score
                            result label
                            31 / 100
                            sample size
                            100
                            updated at
                            2026-11-09T18:08:05.227Z

                            Source specification observations

                            1–4 / 4

                            openRouter ↗

                            field
                            context_length
                            value
                            131072
                            normalized value
                            131072
                            scope
                            source-model
                            source id
                            openRouter
                            confidence
                            1

                            siliconFlow ↗

                            field
                            context_length
                            value
                            33K
                            scope
                            source-model
                            source id
                            siliconFlow
                            confidence
                            1

                            openRouter ↗

                            field
                            knowledge_cutoff
                            reported field
                            knowledge_cutoff
                            value
                            2023-12-31
                            normalized value
                            2023-12-31
                            precision
                            day
                            scope
                            source-model
                            source id
                            openRouter
                            confidence
                            1

                            openRouter ↗

                            field
                            max_output_tokens
                            value
                            115200
                            normalized value
                            115200
                            scope
                            source-model
                            source id
                            openRouter
                            confidence
                            1

                            Original sources

                            1–18 / 18

                            SiliconFlow · Model page ↗

                            Hugging Face · Model repository ↗

                            DeepInfra · Model page ↗

                            Google Cloud · Model page ↗

                            Google Cloud · Model page ↗

                            OpenRouter · Model page ↗

                            SiliconFlow · Playground ↗

                            SiliconFlow · API documentation ↗

                            DeepInfra · Catalog ↗

                            GitHub · Catalog ↗

                            GitHub · Catalog ↗

                            Google Cloud · Catalog ↗

                            Hugging Face · Catalog ↗

                            Hugging Face · Catalog ↗

                            IBM · Catalog ↗

                            OpenRouter · Catalog ↗

                            OpenRouter · Catalog ↗

                            SiliconFlow · Catalog ↗

                            Source links

                            1–12 / 12

                            Explore more models

                            Models

                            Loading
                            EVIDENCE / COMPARISON

                            Benchmarks

                            Explore every benchmark

                            Loading

                            These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

                            Install TensorTrace