Ollama · llama3-gradient
<img src="https://ollama.com/assets/library/llama3-gradient/11962db3-2750-4346-b89c-b9b5612cc76a" width="360" /> This model extends LLama-3 8B's context length from 8k to > 1040K, developed by Gradie…
ollama/llama3-gradient
<img src="https://ollama.com/assets/library/llama3-gradient/11962db3-2750-4346-b89c-b9b5612cc76a" width="360" />
This model extends LLama-3 8B's context length from 8k to > 1040K, developed by Gradient, sponsored by compute from Crusoe Energy. It demonstrates that SOTA LLMs can learn to operate on long context with minimal training by appropriately adjusting RoPE theta. We trained on 830M tokens for this stage, and 1.4B tokens total for all stages, which is < 0.01% of Llama-3's original pre-training data.
## Large Context Window
> Note: using a 256k context window requires at least 64GB of memory. Using a 1M+ context window requires significantly more (100GB+).
To extend the context window (to 256k for example) use:
<sub>API</sub>
```
curl http://localhost:11434/api/generate -d '{
"model": "llama3-gradient",
"prompt": "Why is the sky blue?",
"options": {
"num_ctx": 256000
}
}'
```
<sub>CLI</sub>
```
ollama run llama3-gradient
>>> /set parameter num_ctx 256000
```
## References
[Website](https://gradient.ai/)
[Hugging Face](https://huggingface.co/gradientai/Llama-3-8B-Instruct-Gradient-1048k)
Specifications & capabilities
- Published model ID
- llama3-gradient
Deployments & artifacts
1–20 / 35
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 8B
- parameter count
- 8000000000
- parameter count is approximate
- true
- quantization level
- Q4_0
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 4661224159
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q4_0
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 39969744863
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- F16
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 141117925311
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q2_K
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 26375120863
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q3_K_L
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 37140604895
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q3_K_M
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 34267506655
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q3_K_S
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 30912063455
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q4_0
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 39969744863
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q4_1
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 44313602015
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q4_K_M
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 42520405983
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q4_K_S
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 40347232223
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q5_0
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 48657459167
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q5_1
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 53001316319
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q5_K_M
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 49949829087
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q5_K_S
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 48657459167
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q6_K
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 57888155615
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 71B
- parameter count
- 71000000000
- parameter count is approximate
- true
- quantization level
- Q8_0
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 74975061983
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 8B
- parameter count
- 8000000000
- parameter count is approximate
- true
- quantization level
- Q4_0
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 4661224159
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 8B
- parameter count
- 8000000000
- parameter count is approximate
- true
- quantization level
- F16
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 16068903583
- updated at
- 2026-11-16T08:10:06.079865+00:00
ollama
- source id
- ollama
- execution mode
- local
- parameter size
- 8B
- parameter count
- 8000000000
- parameter count is approximate
- true
- quantization level
- Q2_K
- context window
- 1000000
- context window is approximate
- true
- artifact size bytes
- 3179143903
- updated at
- 2026-11-16T08:10:06.079865+00:00
Explore more models