t5-base · azureml v18

The developers of the Text-To-Text Transfer Transformer (T5) [write](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html): > With T5, we propose reframing all NLP tasks into a …

azureFoundry/azureml/t5-base@18

The developers of the Text-To-Text Transfer Transformer (T5) write:

With T5, we propose reframing all NLP tasks into a unified text-to-text-format where the input and output are always text strings, in contrast to BERT-style models that can only output either a class label or a span of the input. Our text-to-text framework allows us to use the same model, loss function, and hyperparameters on any NLP task.

T5-Base is the checkpoint with 220 million parameters.

Training Details

Training Data

The model is pre-trained on the Colossal Clean Crawled Corpus (C4), which was developed and released in the context of the same research paper as T5.

The model was pre-trained on a on a multi-task mixture of unsupervised and supervised tasks. Thereby, the following datasets were being used for:

Datasets used for Unsupervised denoising objective:

Datasets used for Supervised text-to-text language modeling objective

Training Procedure

In their abstract, the model developers write:

In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks.

The framework introduced, the T5 framework, involves a training procedure that brings together the approaches studied in the paper. See the research paper for further details.

Evaluation Results

For full results for T5-Base, see the research paper, Table 14.

Testing Data, Factors & Metrics

The developers evaluated the model on 24 tasks, see the research paper for full details.

Model Evaluation samples

TaskUse caseDatasetPython sample (Notebook)CLI with YAML
TranslationTranslationwmt16/ro-enevaluate-model-translation.ipynbevaluate-model-translation.yml

Inference samples

Inference typePython sample (Notebook)
Real timesdk-example.ipynb
Real timetext-translation-online-endpoint.ipynb

Sample inputs and outputs

Sample input

{
    "input_data": [
        "translate English to French: Life is so beautiful, once you learn how to live with it",
        "translate English to German: Berlin is the capital of Germany"
    ]
}

Sample output

[
  "La vie est si belle, une fois que vous apprenez à la vivre",
  "Berlin ist die Hauptstadt Deutschlands"
]

Specifications & capabilities

Provider specifications & access

1–1 / 1

t5-base · azureml v18 ↗

source model id
azureml/t5-base@18
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
    output_modalities
      capabilities

        Original sources

        1–3 / 3

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model documentation ↗

        Microsoft Foundry · Catalog ↗

        Source links

        1–1 / 1

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace