microsoft/deberta-base-mnli

## DeBERTa: Decoding-enhanced BERT with Disentangled Attention [DeBERTa](https://arxiv.org/abs/2006.03654) improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder.…

azureFoundry/HuggingFace/microsoft-deberta-base-mnli@5

DeBERTa: Decoding-enhanced BERT with Disentangled Attention

DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. It outperforms BERT and RoBERTa on majority of NLU tasks with 80GB training data.

Please check the official repository for more details and updates.

This model is the base DeBERTa model fine-tuned with MNLI task

Fine-tuning on NLU tasks

We present the dev results on SQuAD 1.1/2.0 and MNLI tasks.

ModelSQuAD 1.1SQuAD 2.0MNLI-m
RoBERTa-base91.5/84.683.7/80.587.6
XLNet-Large-/--/80.286.8
DeBERTa-base93.1/87.286.2/83.188.8

Citation

If you find DeBERTa useful for your work, please cite the following paper:

@inproceedings{
he2021deberta,
title={DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION},
author={Pengcheng He and Xiaodong Liu and Jianfeng Gao and Weizhu Chen},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=XPZIaotutsD}
}

microsoft/deberta-base-mnli is a pre-trained language model available on the Hugging Face Hub. It's specifically designed for the text-classification task in the transformers library. If you want to learn more about the model's architecture, hyperparameters, limitations, and biases, you can find this information on the model's dedicated Model Card on the Hugging Face Hub.

Here's an example API request payload that you can use to obtain predictions from the model:

{
  "inputs": "[CLS] I love you. [SEP] I like you. [SEP]"
}

Specifications & capabilities

Published model ID
deberta-base-mnli

Provider specifications & access

1–1 / 1

microsoft/deberta-base-mnli ↗

source model id
HuggingFace/microsoft-deberta-base-mnli@5
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
mit
architecture
input_modalities
    output_modalities
      capabilities

        Original sources

        1–4 / 4

        Hugging Face · microsoft/deberta-base-mnli ↗

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model documentation ↗

        Microsoft Foundry · Catalog ↗

        Source links

        1–1 / 1

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace