microsoft-deberta-base · azureml v14 ↗
- source model id
- azureml/microsoft-deberta-base@14
- hosting
- foundry-managed-compute
- platforms
- Microsoft Foundry
- license
- mit
- architecture
- input_modalities
- output_modalities
- capabilities
[DeBERTa](https://arxiv.org/abs/2006.03654) (Decoding-enhanced BERT with Disentangled Attention) improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. With thos…
azureFoundry/azureml/microsoft-deberta-base@14
DeBERTa (Decoding-enhanced BERT with Disentangled Attention) improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. With those two improvements, DeBERTa out perform RoBERTa on a majority of NLU tasks with 80GB training data.
Please check the official repository for more details and updates.
This the DeBERTa XLarge model with 48 layers, 1024 hidden size. Total parameters 750M.
We present the dev results on SQuAD 1.1/2.0 and several GLUE benchmark tasks.
| Model | SQuAD 1.1 | SQuAD 2.0 | MNLI-m/mm | SST-2 | QNLI | CoLA | RTE | MRPC | QQP | STS-B | | ------------------------------------------------------------------------------------------- | ------------- | ------------- | ------------- | -------- | -------- |
1–1 / 1
1–3 / 3
Microsoft Foundry · Model page ↗
1–1 / 1
These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.