roberta-large · azureml v7 ↗
- source model id
- azureml/roberta-large@7
- hosting
- foundry-managed-compute
- platforms
- Microsoft Foundry
- license
- mit
- architecture
- input_modalities
- output_modalities
- capabilities
The RoBERTa Large model is a pretrained language model developed by the Hugging Face team, based on the transformer architecture. It was trained on a large corpus of English data in a self-supervised…
azureFoundry/azureml/roberta-large@7
The RoBERTa Large model is a pretrained language model developed by the Hugging Face team, based on the transformer architecture. It was trained on a large corpus of English data in a self-supervised manner using the masked language modeling (MLM) objective. The model is case-sensitive and primarily intended for use in fine-tuning downstream tasks such as sequence classification, token classification, or question answering. It was trained on a combination of five datasets weighing 160GB of text, and uses a vocabulary size of 50,000 for tokenization. The model was trained for 500K steps on 1024 V100 GPUs with a batch size of 8K and a sequence length of 512. The optimizer used was Adam with a learning rate of 4e-4, β1=0.9, β2=0.98, and ϵ=1e-6, with a weight decay of 0.01 and learning rate warmup for 30,000 steps.
Please Note: This model accepts masks in <mask> format. See Sample input for reference.
The above summary was generated using ChatGPT. Review the <a href="https://huggi
1–1 / 1
1–3 / 3
Microsoft Foundry · Model page ↗
1–1 / 1
These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.