distilroberta-base · azureml v15

**distilroberta-base** is a distilled version of the [RoBERTa-base model](https://huggingface.co/roberta-base). It follows the same training procedure as [DistilBERT](https://huggingface.co/distilber…

azureFoundry/azureml/distilroberta-base@15

distilroberta-base is a distilled version of the RoBERTa-base model. It follows the same training procedure as DistilBERT. The code for the distillation process can be found here. This model is case-sensitive: it makes a difference between english and English.

The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base). On average DistilRoBERTa is twice as fast as Roberta-base.

Training Details

DistilRoBERTa was pre-trained on OpenWebTextCorpus, a reproduction of OpenAI's WebText dataset (it is ~4 times less training data than the teacher RoBERTa). See the roberta-base model card for further details on training.

Evaluation Results

Whe

Specifications & capabilities

Provider specifications & access

1–1 / 1

distilroberta-base · azureml v15 ↗

source model id
azureml/distilroberta-base@15
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
    output_modalities
      capabilities

        Original sources

        1–3 / 3

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model documentation ↗

        Microsoft Foundry · Catalog ↗

        Source links

        1–1 / 1

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace