nvidia/EGM-8B

`nvidia/EGM-8B` powered by vLLM - [Original Model Card](https://huggingface.co/nvidia/EGM-8B) - [vLLM Documentation](https://docs.vllm.ai/) > **⚠️ Warning**: `nvidia/EGM-8B` is a base model hence it …

azureFoundry/HuggingFace/nvidia-egm-8b@1

nvidia/EGM-8B powered by vLLM

⚠️ Warning: nvidia/EGM-8B is a base model hence it doesn't have a pre-defined chat template as it's not fine-tuned for chat applications but rahter just pre-trained, meaning that it's likely not optimal for consumer facing inference or downstream applications.

Completions API

⚠️ Warning: note that the OpenAI Completions API is a legacy API, despite it being supported in vLLM; but it's the only way to get completions for base models as those don't have a chat template.

Send Request

You can use cURL or any REST Client to send a request to the Azure ML endpoint with your Azure ML token.

curl <AZUREML_ENDPOINT_URL> \
    -X POST \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{"model":"nvidia/EGM-8B","prompt":"What is Deep Learning?"}'

Supported Parameters

The following are the only mandatory parameters to send in the HTTP POST request to /v1/completions.

  • model (string): Model ID used to generate the response, in this case since only a single model is deployed within the same endpoint you can either set it to nvidia/EGM-8B or leave it blank instead.
  • prompt (string or array): The prompt(s) to generate completions for, encoded as a string, array of strings, array of tokens, or array of token arrays. Note that <|endoftext|> is the document separator that the model sees during training, so if a prompt is not specified the model will generate as if from the beginning of a new document.

A list of messages comprising the conversation so far. Depending on the model you use, different message types (modalities) are supported, like text, images, and audio.

The rest of the parameters are optional, and since this model is powered by vLLM with an OpenAI compatible interface on top for the Completions API for text generation, then the I/O interfaces for both generation and streaming are the same as in OpenAI Completions API.

You can find the whole specification of the allowed parameters in the OpenAI Completions API Specification, or alternatively in the endpoint /openapi.json for the current Azure ML Endpoint.

Example payload

{
  "model": "nvidia/EGM-8B",
  "prompt": "What is Deep Learning?",
  "max_tokens": 256,
  "temperature": 0.6
}

Specifications & capabilities

Input
text
Output
text
Published model ID
EGM-8B

Provider specifications & access

1–2 / 2

nvidia/EGM-8B ↗

source model id
HuggingFace/nvidia-egm-8b@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
  • text
output_modalities
  • text
capabilities

    nvidia/EGM-8B ↗

    source model id
    HuggingFace/nvidia-egm-8b@2
    hosting
    foundry-managed-compute
    platforms
    • Microsoft Foundry
    license
    apache-2.0
    architecture
    input_modalities
    • text
    output_modalities
    • text
    capabilities

      Original sources

      1–5 / 5

      Hugging Face · nvidia/EGM-8B ↗

      Microsoft Foundry · Model page ↗

      Microsoft Foundry · Model page ↗

      Microsoft Foundry · Model documentation ↗

      Microsoft Foundry · Catalog ↗

      Source links

      1–1 / 1

      Explore more models

      Models

      Loading
      EVIDENCE / COMPARISON

      Benchmarks

      Explore every benchmark

      Loading

      These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

      Install TensorTrace