distilroberta-base · azureml v16 ↗
- source model id
- azureml/distilroberta-base@16
- hosting
- foundry-managed-compute
- platforms
- Microsoft Foundry
- license
- apache-2.0
- architecture
- input_modalities
- output_modalities
- capabilities
**distilroberta-base** is a distilled version of the [RoBERTa-base model](https://huggingface.co/roberta-base). It follows the same training procedure as [DistilBERT](https://huggingface.co/distilber…
azureFoundry/azureml/distilroberta-base@16
distilroberta-base is a distilled version of the RoBERTa-base model. It follows the same training procedure as DistilBERT. The code for the distillation process can be found here. This model is case-sensitive: it makes a difference between english and English.
The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base). On average DistilRoBERTa is twice as fast as Roberta-base.
DistilRoBERTa was pre-trained on OpenWebTextCorpus, a reproduction of OpenAI's WebText dataset (it is ~4 times less training data than the teacher RoBERTa). See the roberta-base model card for further details on training.
Whe
1–1 / 1
1–3 / 3
Microsoft Foundry · Model page ↗
1–1 / 1
These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.