camembert-base · azureml v14

CamemBERT is a state-of-the-art language model for French based on the RoBERTa model. It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretrain…

azureFoundry/azureml/camembert-base@14

CamemBERT is a state-of-the-art language model for French based on the RoBERTa model.

It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains.

Training Details

Training Data

OSCAR or Open Super-large Crawled Aggregated coRpus is a multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the Ungoliant architecture.

Training Procedure

Model#paramsArch.Training data
camembert-base110MBaseOSCAR (138 GB of text)
camembert/camembert-large335MLargeCCNet (135 GB of text)
camembert/camembert-base-ccnet110MBaseCCNet (135 GB of text)
camembert/camembert-base-wikipedia-4gb110MBaseWikipedia (4 GB of text)
camembert/camembert-base-oscar-4gb110MBaseSubsample of OSCAR (4 GB of text)
camembert/camembert-base-ccnet-4gb110MBaseSubsample of CCNet (4 GB of text)

Evaluation Results

The model developers evaluated CamemBERT using four different downstream tasks for French: part-of-speech (POS) tagging, dependency parsing, named entity recognition (NER) and natural language inference (NLI).

Limitations and Biases

CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)).

This model was pretrained on a subcorpus of OSCAR multilingual corpus. Some of the limitations and risks associated with the OSCAR dataset, which are further detailed in the OSCAR dataset card, include the following:

The quality of some OSCAR sub-corpora might be lower than expected, specifically for the lowest-resource languages.

Constructed from Common Crawl, Personal and sensitive information might be present.

Model Evaluation samples

TaskUse caseDatasetPython sample (Notebook)CLI with YAML
Fill MaskFill Maskrcds/wikipedia-for-mask-fillingevaluate-model-fill-mask.ipynbevaluate-model-fill-mask.yml

Inference samples

Inference typePython sample (Notebook)
Real timesdk-example.ipynb
Real timefill-mask-online-endpoint.ipynb

Sample inputs and outputs

Sample input

{
    "input_data": [
        "Paris est la <mask> de la France.",
        "Aujourd’hui, c’est un <mask> jour day!"
    ]
}

Sample output

[
  "capitale",
  "nouveau"
]

Specifications & capabilities

Provider specifications & access

1–1 / 1

camembert-base · azureml v14 ↗

source model id
azureml/camembert-base@14
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
    output_modalities
      capabilities

        Original sources

        1–3 / 3

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model documentation ↗

        Microsoft Foundry · Catalog ↗

        Source links

        1–1 / 1

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace