codellama/CodeLlama-13b-Python-hf

# Key capabilities ## About this model This model is designed for general code synthesis and understanding. This is a static model trained on an offline dataset. Future versions of Code Llama - Instr…

azureFoundry/azureml-meta/CodeLlama-13b-Python-hf@10

Key capabilities

About this model

This model is designed for general code synthesis and understanding. This is a static model trained on an offline dataset. Future versions of Code Llama - Instruct will be released as we improve model safety with community feedback.

Key model capabilities

  • Code completion.
  • Infilling.
  • Instructions / chat.
  • Python specialist.

Use cases

See Responsible AI for additional considerations for responsible use.

Key use cases

Code Llama and its variants is intended for commercial and research use in English and relevant programming languages. The base model Code Llama can be adapted for a variety of code synthesis and understanding tasks, Code Llama - Python is designed specifically to handle the Python programming language, and Code Llama - Instruct is intended to be safer to use for code assistant and generation applications.

Out of scope use cases

Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in languages other than English. Use in any other way that is prohibited by the Acceptable Use Policy and Licensing Agreement for Code Llama and its variants.

Pricing

Pricing is based on a number of factors, including deployment type and tokens used. See pricing details here.

Technical specs

Code Llama is an auto-regressive language model that uses an optimized transformer architecture. Code Llama comes in three model sizes, and three variants:

  • Code Llama: base models designed for general code synthesis and understanding
  • Code Llama - Python: designed specifically for Python
  • Code Llama - Instruct: for instruction following and safer deployment

All variants are available in sizes of 7B, 13B and 34B parameters.

This repository contains the base version of the 34B parameters model.

Training cut-off date

Code Llama and its variants have been trained between January 2023 and July 2023.

Training time

In aggregate, training all 9 Code Llama models required 400K GPU hours of computation on hardware of type A100-80GB (TDP of 350-400W). We used custom training libraries. The training and fine-tuning of the released models have been performed Meta's Research Super Cluster.

Input formats

Models input text only.

Output formats

Models generate text only.

Supported languages

Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in languages other than English.

Sample JSON response

{
    "input_data": {
        "input_string": [
            "def fibonacci("
        ],
        "parameters": {
            "top_p": 0.9,
            "temperature": 0.2,
            "do_sample": true,
            "max_new_tokens": 100
        }
    }
}

Sample output:

[
  {
    "0": "def fibonacci(n):\n    if n == 0:\n        return 0\n    elif n == 1:\n        return 1\n    else:\n        return fibonacci(n-1) + fibonacci(n-2)\n\n\ndef main():\n    n = int(input(\"Enter a number: \"))\n    print(fibonacci(n))\n\n\nif __name__ == \"__main__\":\n    main()"
  }
]

Model architecture

Code Llama is an auto-regressive language model that uses an optimized transformer architecture.

Long context

The provider has not supplied this information.

Optimizing model performance

The provider has not supplied this information.

Additional assets

More information can be found in the paper "Code Llama: Open Foundation Models for Code" or its arXiv page.

Training disclosure

Training, testing and validation

All experiments reported here and the released models have been trained and fine-tuned using the same data as Llama 2 with different weights (see Section 2 and Table 1 in the research paper for details). Estimated total emissions were 65.3 tCO2eq, 100% of which were offset by Meta's sustainability program.

Distribution

Distribution channels

A custom commercial license is available at: https://ai.meta.com/resources/models-and-libraries/llama-downloads/

More information

Code Llama and its variants are a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Code Llama's potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate or objectionable responses to user prompts. Therefore, before deploying any applications of Code Llama, developers should perform safety testing and tuning tailored to their specific applications of the model.

Please see the Responsible Use Guide available available at https://ai.meta.com/llama/responsible-user-guide.

See evaluations for the main models and detailed ablations in Section 3 and safety evaluations in Section 4 of the research paper.

Specifications & capabilities

Input
text
Output
text
Published model ID
CodeLlama-13b-Python-hf

Evaluations

1–20 / 34

SambaNova ToolBench

92.4%

source id
toolBench
leaderboard name
The Cat API
dimension
toolUse
metric
The Cat API
rank
4
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
The Cat API
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

BigCode Models Leaderboard

0

source id
bigCode
leaderboard name
Throughput (tokens/s) bs=50
dimension
runtime
metric
Throughput (tokens/s) bs=50
rank
5
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
Throughput (tokens/s) bs=50
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
15
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

SambaNova ToolBench

93%

source id
toolBench
leaderboard name
Open Weather
dimension
toolUse
metric
Open Weather
rank
6
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
Open Weather
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

SambaNova ToolBench

0

source id
toolBench
leaderboard name
WebShop Long
dimension
toolUse
metric
WebShop Long
rank
6
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
WebShop Long
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

SambaNova ToolBench

86%

source id
toolBench
leaderboard name
Home Search
dimension
toolUse
metric
Home Search
rank
7
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
Home Search
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

SambaNova ToolBench

41.5%

source id
toolBench
leaderboard name
Tabletop
dimension
toolUse
metric
Tabletop
rank
7
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
Tabletop
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

BigCode Models Leaderboard

28,568

source id
bigCode
leaderboard name
Peak Memory (MB)
dimension
runtime
metric
Peak Memory (MB)
rank
8
aggregation scope
status
published
direction
lowerIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
Peak Memory (MB)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
21
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

BigCode Models Leaderboard

25.3

source id
bigCode
leaderboard name
Throughput (tokens/s)
dimension
runtime
metric
Throughput (tokens/s)
rank
8
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
Throughput (tokens/s)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
21
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

SambaNova ToolBench

50.8%

source id
toolBench
leaderboard name
Google Sheets
dimension
toolUse
metric
Google Sheets
rank
9
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
Google Sheets
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

SambaNova ToolBench

21.9

source id
toolBench
leaderboard name
VirtualHome
dimension
toolUse
metric
VirtualHome
rank
10
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
VirtualHome
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

SambaNova ToolBench

56.3%

source id
toolBench
leaderboard name
Avg.
dimension
toolUse
metric
Avg.
rank
10
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
Avg.
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

SambaNova ToolBench

62.5%

source id
toolBench
leaderboard name
Trip Booking
dimension
toolUse
metric
Trip Booking
rank
14
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
Trip Booking
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

SambaNova ToolBench

2.4

source id
toolBench
leaderboard name
WebShop Short
dimension
toolUse
metric
WebShop Short
rank
16
aggregation scope
status
published
direction
higherIsBetter
conditions
shots
3
suite
SambaNova ToolBench
tuned_on_toolbench
False
metric_kind
WebShop Short
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
results.json
source_revision
ec63fbf3e95dc80fdddb584b650e52889dbd61140b97ba8ef7b6f1dd4dc7b615
metric_population
41
benchmark_evaluation
true
source model id
CodeLlama-13b-Python-hf
Original result ↗

BigCode Models Leaderboard

34.55%

source id
bigCode
leaderboard name
php
dimension
coding
metric
php
rank
19
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
php
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
29
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

BigCode Models Leaderboard

30.4%

source id
bigCode
leaderboard name
julia
dimension
coding
metric
julia
rank
19
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
julia
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
29
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

BigCode Models Leaderboard

40.66%

source id
bigCode
leaderboard name
javascript
dimension
coding
metric
javascript
rank
20
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
javascript
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
29
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

BigCode Models Leaderboard

36.21%

source id
bigCode
leaderboard name
cpp
dimension
coding
metric
cpp
rank
20
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
cpp
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
29
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

BigCode Models Leaderboard

33.56%

source id
bigCode
leaderboard name
java
dimension
coding
metric
java
rank
20
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
java
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
29
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

BigCode Models Leaderboard

28.67%

source id
bigCode
leaderboard name
Average score
dimension
coding
metric
Average score
rank
20
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
Average score
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
29
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

BigCode Models Leaderboard

12.51%

source id
bigCode
leaderboard name
racket
dimension
coding
metric
racket
rank
20
aggregation scope
status
published
direction
higherIsBetter
conditions
suite
BigCode
evaluation_sequence_length
16384
metric_kind
racket
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
bigcode/bigcode-models-leaderboard/data/code_eval_board.csv
published_at
2024-11-11T20:36:38Z
source_revision
e9d7f58791a40b01c9477337f81d5d77602db07768042d96cbbb42e11a50ee09
metric_population
29
benchmark_evaluation
true
source model id
CodeLlama-13b-Python
Original result ↗

Provider specifications & access

1–3 / 3

codellama/CodeLlama-13b-Python-hf ↗

source model id
azureml-meta/CodeLlama-13b-Python-hf@10
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
llama2
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    codellama/CodeLlama-13b-Python-hf ↗

    source model id
    azureml-meta/CodeLlama-13b-Python-hf@11
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    license
    llama2
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      codellama/CodeLlama-13b-Python-hf ↗

      source model id
      HuggingFace/codellama-codellama-13b-python-hf@1
      hosting
      foundry-managed-compute
      platforms
      • Microsoft Foundry
      license
      llama2
      architecture
      input_modalities
      • text
      output_modalities
      • text
      capabilities

        Safety assessments

        1–1 / 1

        Foundation Model Transparency Index 2025 ↗

        31 / 100

        source id
        stanfordFMTI
        canonical model id
        azureFoundry/azureml-meta/CodeLlama-13b-Python-hf@10
        taxonomy
        transparency
        hazard category
        transparency
        hazard name
        Organization transparency
        policy area
        Meta
        assessment type
        organization-level disclosure
        metric
        disclosure_score
        result label
        31 / 100
        sample size
        100
        updated at
        2026-11-09T18:08:05.227Z

        Original sources

        1–10 / 10

        Hugging Face · Model repository ↗

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model documentation ↗

        GitHub · Catalog ↗

        Hugging Face · Catalog ↗

        Hugging Face · Catalog ↗

        Hugging Face · Catalog ↗

        Microsoft Foundry · Catalog ↗

        Source links

        1–5 / 5

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace