microsoft/prophetnet-large-uncased

## prophetnet-large-uncased Pretrained weights for [ProphetNet](https://arxiv.org/abs/2001.04063). ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-s…

azureFoundry/HuggingFace/microsoft-prophetnet-large-uncased@5

prophetnet-large-uncased

Pretrained weights for ProphetNet.
ProphetNet is a new pre-trained language model for sequence-to-sequence learning with a novel self-supervised objective called future n-gram prediction.
ProphetNet is able to predict more future tokens with a n-stream decoder. The original implementation is Fairseq version at github repo.

Usage

This pre-trained model can be fine-tuned on sequence-to-sequence tasks. The model could e.g. be trained on headline generation as follows:

from transformers import ProphetNetForConditionalGeneration, ProphetNetTokenizer

model = ProphetNetForConditionalGeneration.from_pretrained("microsoft/prophetnet-large-uncased")
tokenizer = ProphetNetTokenizer.from_pretrained("microsoft/prophetnet-large-uncased")

input_str = "the us state department said wednesday it had received no formal word from bolivia that it was expelling the us ambassador there but said the charges made against him are `` baseless ."
target_str = "us rejects charges against its ambassador in bolivia"

input_ids = tokenizer(input_str, return_tensors="pt").input_ids
labels = tokenizer(target_str, return_tensors="pt").input_ids

loss = model(input_ids, labels=labels).loss

Citation

@article{yan2020prophetnet,
  title={Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training},
  author={Yan, Yu and Qi, Weizhen and Gong, Yeyun and Liu, Dayiheng and Duan, Nan and Chen, Jiusheng and Zhang, Ruofei and Zhou, Ming},
  journal={arXiv preprint arXiv:2001.04063},
  year={2020}
}

microsoft/prophetnet-large-uncased is a pre-trained language model available on the Hugging Face Hub. It's specifically designed for the text2text-generation task in the transformers library. If you want to learn more about the model's architecture, hyperparameters, limitations, and biases, you can find this information on the model's dedicated Model Card on the Hugging Face Hub.

Specifications & capabilities

Input
text
Output
text
Published model ID
prophetnet-large-uncased

Provider specifications & access

1–1 / 1

microsoft/prophetnet-large-uncased ↗

source model id
HuggingFace/microsoft-prophetnet-large-uncased@5
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    Original sources

    1–4 / 4

    Hugging Face · microsoft/prophetnet-large-uncased ↗

    Microsoft Foundry · Model page ↗

    Microsoft Foundry · Model documentation ↗

    Microsoft Foundry · Catalog ↗

    Source links

    1–1 / 1

    Explore more models

    Models

    Loading
    EVIDENCE / COMPARISON

    Benchmarks

    Explore every benchmark

    Loading

    These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

    Install TensorTrace