~/ / AI articles / Local LLMs on a VPS / How much memory does a local LLM need on a VPS? Model file plus KV cache

How much memory does a local LLM need on a VPS? Model file plus KV cache

Before you rent a server to run a local model, you need to know how much memory it takes. The answer has two parts: the model itself, and a cache that grows with the length of the conversation and the number of requests handled at once. This article puts together what the Ollama library, the Ollama FAQ and the official model configuration files say, calculates the cache for six popular models, and lets you try your own numbers.

Published: 2026-10-26 · 5 min read · Model data checked against the official sources on 2026-10-26. Sizes and defaults change, so confirm them before you choose a plan.

Two things fill the memory

The first is the model weights, which take about as much memory as the download file. The second is the KV cache, which holds the keys and values that the model has already computed for the text in its context. The Ollama FAQ states that the required RAM scales by OLLAMA_NUM_PARALLEL multiplied by OLLAMA_CONTEXT_LENGTH.

The size of the cache can be worked out from the model's configuration: 2 × layers × KV heads × head dimension × bytes per value gives the bytes for one token, and you multiply by the context length and the number of parallel requests. In f16 a value takes 2 bytes. For Qwen3 8B, which has 36 layers, 8 KV heads and a head dimension of 128, one token takes 2 × 36 × 8 × 128 × 2 = 147,456 bytes. At 4,096 tokens that is 576 MiB, and at 32,768 tokens it is 4,608 MiB. This formula is our calculation from the configuration values, not a figure published by Ollama.

What the official sources say about each model

The download size and the quantization are the ones the Ollama library shows for each tag, and all six tags below are shown with the Q4_K_M quantization. The layers, KV heads and head dimension come from each model's config.json on Hugging Face.

Model	Download size (GB)	Layers	KV heads	Head dimension
Qwen3 0.6B	0.523	28	8	128
Qwen3 1.7B	1.4	28	8	128
Qwen3 4B	2.5	36	8	128
Qwen3 8B	5.2	36	8	128
Qwen3 14B	9.3	40	8	128
Mistral 7B (v0.3)	4.4	32	8	128

The Mistral 7B configuration gives no head dimension, so it is the hidden size divided by the number of attention heads, 4096 ÷ 32 = 128. The Qwen3 4B tag in the Ollama library lists a 256K context window. The 2507 releases of Qwen3 4B on Hugging Face have the same layers, KV heads and head dimension as the original.

What that means for memory

Using the formula above, this table shows the cache for each model at two context lengths, and the estimated total at 4,096 tokens. The total is the weights, the cache in f16 for one request at a time, and a reserve of 1 GiB for the operating system and other services. The last column is the smallest common memory size that holds it.

Model	KV cache at 4,096 tokens (MiB)	KV cache at 32,768 tokens (MiB)	Estimated total at 4,096 tokens (GiB)	Smallest common memory size (GiB)
Qwen3 0.6B	448	3584	1.92	2
Qwen3 1.7B	448	3584	2.74	4
Qwen3 4B	576	4608	3.89	4
Qwen3 8B	576	4608	6.41	8
Qwen3 14B	640	5120	10.29	12
Mistral 7B (v0.3)	512	4096	5.60	8

These are estimates, not measurements. A model that fits with only a little to spare at 4,096 tokens will not fit when you raise the context length: at 32,768 tokens the cache of Qwen3 8B alone is 4,608 MiB.

Longer context, more parallel requests

The Ollama FAQ says that required RAM scales by OLLAMA_NUM_PARALLEL multiplied by OLLAMA_CONTEXT_LENGTH, and gives an example: a 2K context with 4 parallel requests results in an 8K context and additional memory.
OLLAMA_NUM_PARALLEL is documented with a default of 1, so a single-user server starts at one request at a time.
The FAQ also says that quantizing the key-value cache with q8_0 uses approximately half the memory of f16, and q4_0 approximately a quarter. The cache then shrinks to about 288 MiB for Qwen3 8B at 4,096 tokens with q8_0, using the ratio from the FAQ.
The context-length page of the Ollama documentation says that a larger context length increases the memory a model needs, and that you can check the PROCESSOR column of ollama ps to see whether a model has been offloaded to the CPU.

Try your own numbers

The calculator below uses the same formula and the same data. Choose a model or enter your own, set the context length, the parallel requests, the cache type and a reserve, and type the memory of the plan you are looking at to see whether the estimate fits.

Try it: Local LLM Memory Estimator

Model

Ollama tag qwen3:8b, quantization Q4_K_M, download size 5.2 GB.

File size (GB)LayersKV headsHead dimension

Context length (tokens)Parallel requestsKV cache typeReserve for OS and other services (GiB)Memory of the plan (GiB, optional)

Estimate

Model weights: 4.84 GiB
KV cache: 0.56 GiB
Reserve: 1.00 GiB
Total: 6.41 GiB (6.88 GB)
Smallest common memory size that holds it: 8 GiB

This is an estimate. It does not include the runtime's compute buffers, and it assumes that every layer has the same KV shape. Measure a real run before you choose a plan.

Model data checked on 2026-10-26.

Sources: Ollama library · Model configuration (config.json) · Ollama FAQ

Everything runs in your browser. Nothing is uploaded.

Before you choose a plan

The estimate leaves out the runtime's own working memory for its compute buffers. Our suggestion: run the model on a test server, watch the memory use with the context length you intend to use, and keep some margin.
A GB is 1,000,000,000 bytes and a GiB is 1,073,741,824 bytes. A provider's "GB" may mean either, so check how your provider counts before you compare.
Memory is not the only limit. The model files in the table need disk space too, and the documents we checked give no speed figures for CPU-only operation.
If a model only just fits, choose the next size up or a smaller model instead of lowering the context length until it starts.