ibm-granite/granite-4.1-8b-GGUF ↗
- source model id
- HuggingFace/ibm-granite-granite-4.1-8b-gguf@1
- hosting
- foundry-managed-compute
- platforms
- Microsoft Foundry
- license
- apache-2.0
- architecture
- input_modalities
- text
- output_modalities
- text
- capabilities
`ibm-granite/granite-4.1-8b-GGUF` with Q8_0 quantization, powered by llama.cpp - [Original Model Card](https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF) - [llama.cpp Documentation](https://gith…
azureFoundry/HuggingFace/ibm-granite-granite-4.1-8b-gguf@1
ibm-granite/granite-4.1-8b-GGUF with Q8_0 quantization, powered by llama.cpp
GGUF is a binary file format optimized for quick loading and saving of models weights, making it highly efficient for inference purposes. It supports multiple quantization schemes, allowing users to choose the best trade-off between performance and resource usage. llama.cpp has native support for GGUF checkpoints, enabling seamless integration and utilization of models stored in this format.
This model's weights come in GGUF format with Q8_0 quantization.
For more information on GGUF format, you can check the Hugging Face docs and the official GGUF documentation.
You can use cURL or any REST Client to send a request to the Azure ML endpoint with your Azure ML token.
curl <AZUREML_ENDPOINT_URL> \
-X POST \
-H "Authorization: Bearer <AZUREML_TOKEN>" \
-H "Content-Type: application/json" \
-d '{"model":"ibm-granite/granite-4.1-8b-GGUF","messages":[{"role":"user","content":"What is Deep Learning?"}]}'
The following are the only mandatory parameters to send in the HTTP POST request to v1/chat/completions.
The rest of the parameters are optional, and since this model is powered by llama.cpp with an OpenAI compatible interface on top for the Chat Completions API for text generation, then the I/O interfaces for both generation and streaming are the same as in OpenAI Chat Completions API.
You can find the whole specification of the allowed parameters in the OpenAI Chat Completion API Specification, or alternatively in the endpoint /openapi.json for the current Azure ML Endpoint.
{
"model": "ibm-granite/granite-4.1-8b-GGUF",
"messages": [
{"role":"user","content":"What is Deep Learning?"}
],
"max_completion_tokens": 256,
"temperature": 0.6
}
1–2 / 2
1–1 / 1
95 / 100
1–6 / 6
Hugging Face · ibm-granite/granite-4.1-8b-GGUF ↗
Microsoft Foundry · Model page ↗
Microsoft Foundry · Model page ↗
1–2 / 2
These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.