roberta-large · azureml v1

The RoBERTa Large model is a pretrained language model developed by the Hugging Face team, based on the transformer architecture. It was trained on a large corpus of English data in a self-supervised…

azureFoundry/azureml/roberta-large@1

The RoBERTa Large model is a pretrained language model developed by the Hugging Face team, based on the transformer architecture. It was trained on a large corpus of English data in a self-supervised manner using the masked language modeling (MLM) objective. The model is case-sensitive and primarily intended for use in fine-tuning downstream tasks such as sequence classification, token classification, or question answering. It was trained on a combination of five datasets weighing 160GB of text, and uses a vocabulary size of 50,000 for tokenization. The model was trained for 500K steps on 1024 V100 GPUs with a batch size of 8K and a sequence length of 512. The optimizer used was Adam with a learning rate of 4e-4, β1=0.9, β2=0.98, and ϵ=1e-6, with a weight decay of 0.01 and learning rate warmup for 30,000 steps.

The above summary was generated using ChatGPT. Review the original model card to understand the data used to train the model, evaluati

Specifications & capabilities

Provider specifications & access

1–1 / 1

roberta-large · azureml v1 ↗

source model id
azureml/roberta-large@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
mit
architecture
input_modalities
    output_modalities
      capabilities

        Original sources

        1–3 / 3

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model documentation ↗

        Microsoft Foundry · Catalog ↗

        Source links

        1–1 / 1

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace