~/ / AI articles / Local LLMs on a VPS / Running AI on your own VPS: how much memory to plan for, and what not to expose
Running AI on your own VPS: how much memory to plan for, and what not to expose
Running a language model or an AI app builder on a rented VPS is attractive: a fixed monthly fee and your data stay on a server you control. But the plan you pick decides whether it runs at all, and a default install can leave a service reachable from the internet. This article collects what the official Ollama and Dify documentation says about memory, CPU-only operation and network exposure, so that you can choose a plan before you pay for it.
Published: · 4 min read · Checked against the official documentation on 2026-10-26. Versions and defaults change, so confirm in the current documents before you install.
What has to fit in memory
A model's download size is the smallest amount of memory it needs, not the amount you should plan for. The Ollama library lists the download size of each tag. For example, Llama 3.2 is listed at 1.3 GB for the 1b tag and 2.0 GB for the 3b tag, both with a 128K context window.
On top of the model, memory grows with the context. The Ollama FAQ states that the required RAM scales by OLLAMA_NUM_PARALLEL multiplied by OLLAMA_CONTEXT_LENGTH, and gives an example: a 2K context with 4 parallel requests becomes an 8K context and needs additional memory. The FAQ also says that quantizing the key-value cache with q8_0 uses approximately half the memory of f16, and q4_0 about a quarter.
| Model tag | Download size | Context window |
|---|---|---|
| llama3.2:1b | 1.3 GB | 128K |
| llama3.2:3b | 2.0 GB | 128K |
- The Ollama FAQ gives 4096 tokens as the default context window, and the context length page says the default depends on the VRAM of your GPU (4k below 24 GiB). Check the value your install actually uses.
- The Ollama README that we checked does not list a RAM requirement per model size. Measure instead of guessing: load the model with the context length you intend to use, send a prompt, and watch the server's memory use.
- Leave room for the operating system and every other service on the same server.
Download sizes were read from the Ollama library page on the date above. They are file sizes, not measured memory use.
No GPU: what CPU-only means
Most VPS plans have no GPU, so a model runs on the CPU and in system memory. The Ollama FAQ describes this case: ollama ps shows 100% CPU in the PROCESSOR column when a model was loaded entirely in system memory, and the default for OLLAMA_MAX_LOADED_MODELS is 3 for CPU inference.
The documents give no speed figures for CPU-only operation, and we did not measure any. Start with the smallest model that does your task, and test it for a few days before you commit to a longer contract.
Running Dify on a VPS
Dify can be self-hosted with Docker Compose. Its documentation lists minimum system requirements, which are minimums rather than recommendations:
- CPU: 2 cores or more
- RAM: 4 GiB or more
- Linux: Docker 19.03 or later and Docker Compose 2.24.0 or later
| Step | What you do |
|---|---|
| 1 | Clone the Dify repository and go to the dify/docker directory |
| 2 | Copy .env.example to .env |
| 3 | Generate a token with openssl rand -base64 32 and set it as DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN in .env |
| 4 | Run docker compose up -d |
| 5 | Check docker compose ps: containers should show Up or healthy |
| 6 | Open /install on your server to create the admin account, then log in |
The Dify page we checked does not mention Ollama or local models. If one server runs both Dify and a local model, plan for the memory of both (our reading of the two documents, not a figure from either).
What not to expose to the internet
The Ollama FAQ says that Ollama binds 127.0.0.1 port 11434 by default and that you change the bind address with the OLLAMA_HOST environment variable. The FAQ page that we checked does not describe any authentication for the Ollama server. What follows is our recommendation, not a statement from the documents:
- Leave Ollama on its default address. If another machine needs it, reach it through an SSH tunnel instead of changing the bind address.
- If you do bind it to a network address, restrict the port in the firewall to the addresses you use, and never open it to everyone.
- Dify is served on the server address, and you create the admin account from /install. Do this immediately after the containers start, and put HTTPS and a firewall rule in front of it before you share the address.
- Keep API keys for model providers in the .env file or a secret store, and out of anything you commit.
Self-hosting or an API?
A VPS costs the same each month whether you use it or not, while an API charges for the tokens you send and receive. If your volume is low, an API can be cheaper than keeping a server running. If it is high and a small model is enough, a server can win. Put in your own volume and compare.
Try it: LLM API Cost Calculator
Estimated cost
Enter the prices and token counts to see the cost.
Official pricing pages
This only multiplies the numbers you enter. It does not know the vendor's prices, discounts, caching, batch rates, tiered pricing or tax.
To find the input tokens of a prompt, use the Token Counter.
Sources
Read next
- Calling a local model from your code: structured outputs and the OpenAI-compatible API in OllamaConstrain a local model's output with a JSON Schema and call Ollama from OpenAI-style clients: what the official docs say, and what is not supported.
- What can a local model do? Tools, thinking, vision and embeddings in Ollama, and what fits on a VPSTools, thinking, vision, embeddings: what the labels in the Ollama library mean, which models carry them, and how big the downloads are.
- Install Ollama on a VPS and keep it running: install, systemd service, settings and updatesInstall Ollama on a Linux VPS and keep it running: the install command, the systemd service, environment variables, model storage and updates, from the official docs.
- How much memory does a local LLM need on a VPS? Model file plus KV cacheModel file plus KV cache: what the Ollama library, the Ollama FAQ and the official model configurations say about the memory a local LLM needs on a VPS.