Ollama · llama3-gradient

<img src="https://ollama.com/assets/library/llama3-gradient/11962db3-2750-4346-b89c-b9b5612cc76a" width="360" /> This model extends LLama-3 8B's context length from 8k to > 1040K, developed by Gradie…

ollama/llama3-gradient

<img src="https://ollama.com/assets/library/llama3-gradient/11962db3-2750-4346-b89c-b9b5612cc76a" width="360" />

This model extends LLama-3 8B's context length from 8k to > 1040K, developed by Gradient, sponsored by compute from Crusoe Energy. It demonstrates that SOTA LLMs can learn to operate on long context with minimal training by appropriately adjusting RoPE theta. We trained on 830M tokens for this stage, and 1.4B tokens total for all stages, which is < 0.01% of Llama-3's original pre-training data.

## Large Context Window

> Note: using a 256k context window requires at least 64GB of memory. Using a 1M+ context window requires significantly more (100GB+).

To extend the context window (to 256k for example) use:

<sub>API</sub> ``` curl http://localhost:11434/api/generate -d '{ "model": "llama3-gradient", "prompt": "Why is the sky blue?", "options": { "num_ctx": 256000 } }' ```

<sub>CLI</sub> ``` ollama run llama3-gradient >>> /set parameter num_ctx 256000 ```

## References

[Website](https://gradient.ai/)

[Hugging Face](https://huggingface.co/gradientai/Llama-3-8B-Instruct-Gradient-1048k)

Specifications & capabilities

Published model ID
llama3-gradient

Deployments & artifacts

1–20 / 35

ollama

source id
ollama
execution mode
local
parameter size
8B
parameter count
8000000000
parameter count is approximate
true
quantization level
Q4_0
context window
1000000
context window is approximate
true
artifact size bytes
4661224159
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q4_0
context window
1000000
context window is approximate
true
artifact size bytes
39969744863
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
F16
context window
1000000
context window is approximate
true
artifact size bytes
141117925311
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q2_K
context window
1000000
context window is approximate
true
artifact size bytes
26375120863
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q3_K_L
context window
1000000
context window is approximate
true
artifact size bytes
37140604895
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q3_K_M
context window
1000000
context window is approximate
true
artifact size bytes
34267506655
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q3_K_S
context window
1000000
context window is approximate
true
artifact size bytes
30912063455
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q4_0
context window
1000000
context window is approximate
true
artifact size bytes
39969744863
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q4_1
context window
1000000
context window is approximate
true
artifact size bytes
44313602015
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q4_K_M
context window
1000000
context window is approximate
true
artifact size bytes
42520405983
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q4_K_S
context window
1000000
context window is approximate
true
artifact size bytes
40347232223
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q5_0
context window
1000000
context window is approximate
true
artifact size bytes
48657459167
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q5_1
context window
1000000
context window is approximate
true
artifact size bytes
53001316319
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q5_K_M
context window
1000000
context window is approximate
true
artifact size bytes
49949829087
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q5_K_S
context window
1000000
context window is approximate
true
artifact size bytes
48657459167
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q6_K
context window
1000000
context window is approximate
true
artifact size bytes
57888155615
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
71B
parameter count
71000000000
parameter count is approximate
true
quantization level
Q8_0
context window
1000000
context window is approximate
true
artifact size bytes
74975061983
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
8B
parameter count
8000000000
parameter count is approximate
true
quantization level
Q4_0
context window
1000000
context window is approximate
true
artifact size bytes
4661224159
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
8B
parameter count
8000000000
parameter count is approximate
true
quantization level
F16
context window
1000000
context window is approximate
true
artifact size bytes
16068903583
updated at
2026-11-16T08:10:06.079865+00:00

ollama

source id
ollama
execution mode
local
parameter size
8B
parameter count
8000000000
parameter count is approximate
true
quantization level
Q2_K
context window
1000000
context window is approximate
true
artifact size bytes
3179143903
updated at
2026-11-16T08:10:06.079865+00:00

Original sources

1–2 / 2

Ollama · Model page ↗

Ollama · Catalog ↗

Source links

1–1 / 1

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace