~/ / AI articles / Local LLMs on a VPS / Install Ollama on a VPS and keep it running: install, systemd service, settings and updates
Install Ollama on a VPS and keep it running: install, systemd service, settings and updates
Once you know that a model fits in the memory of your VPS, installing Ollama takes a few commands. What takes more care is keeping it running: starting it at boot, setting its environment variables, choosing where large model files go, and updating it later. This article walks through the official Ollama documentation for Linux and the FAQ, step by step.
Published: · 4 min read · Checked against the official Ollama documentation on 2026-10-26. Commands and defaults change, so confirm them in the current documents before you run anything on a server.
Check the memory first
A model that does not fit in the memory of your server will not run well, and downloading it first wastes time and disk space. Estimate the memory before you install anything: the model file, plus the KV cache for the context length you plan to use, plus a reserve for the operating system.
Try it: Local LLM Memory Estimator
Estimate
- Model weights
- 4.84 GiB
- KV cache
- 0.56 GiB
- Reserve
- 1.00 GiB
- Total
- 6.41 GiB (6.88 GB)
- Smallest common memory size that holds it
- 8 GiB
This is an estimate. It does not include the runtime's compute buffers, and it assumes that every layer has the same KV shape. Measure a real run before you choose a plan.
Model data checked on 2026-10-26.
Sources: Ollama library · Model configuration (config.json) · Ollama FAQ
Everything runs in your browser. Nothing is uploaded.
Install
The Ollama documentation for Linux gives a one-line install command and a manual install. Read a script before you pipe it into a shell on a server that matters; the script is at the address in the command.
| Way | Command or step |
|---|---|
| Install script | curl -fsSL https://ollama.com/install.sh | sh |
| Manual: remove old libraries when upgrading | sudo rm -rf /usr/lib/ollama |
| Manual: download and extract | curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst | sudo tar x -C /usr |
| Manual: start the server | ollama serve |
| Check that it is running (in another terminal) | ollama -v |
For an ARM64 server the documentation uses ollama-linux-arm64.tar.zst instead of the amd64 package. GPU drivers are optional: the documentation lists CUDA for NVIDIA GPUs and ROCm for AMD GPUs, and a typical VPS has neither.
Run it as a service
To start Ollama at boot and restart it after a failure, the documentation describes a systemd service. First create a dedicated user and group, then create the unit file /etc/systemd/system/ollama.service.
| Section | Lines in the documentation's unit file |
|---|---|
| [Unit] | a description, and After=network-online.target |
| [Service] | ExecStart=/usr/bin/ollama serve |
| [Service] | User=ollama and Group=ollama |
| [Service] | Restart=always and RestartSec=3 |
| [Service] | Environment="PATH=$PATH" |
| [Install] | WantedBy=multi-user.target |
- Create the user and group: sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama, then sudo usermod -a -G ollama $(whoami)
- Load and enable the unit: sudo systemctl daemon-reload, then sudo systemctl enable ollama
- Start it and check it: sudo systemctl start ollama, then sudo systemctl status ollama
- Read the logs: journalctl -e -u ollama
The documentation does not say whether the install script sets up this service itself. After you install, check with systemctl status ollama instead of assuming.
Settings that matter on a server
Environment variables go into a systemd override. The FAQ says to edit the service with systemctl edit ollama.service, add the variable under [Service], then run systemctl daemon-reload and systemctl restart ollama. For example: Environment="OLLAMA_HOST=0.0.0.0:11434".
| Variable | What the documentation says |
|---|---|
| OLLAMA_HOST | Ollama binds 127.0.0.1 port 11434 by default; this variable changes the bind address |
| OLLAMA_KEEP_ALIVE | By default models are kept in memory for 5 minutes before being unloaded; a keep_alive value in an API request overrides this setting |
| OLLAMA_MAX_LOADED_MODELS | The maximum number of models loaded at once if they fit in memory; the default is 3 for CPU inference |
| OLLAMA_MODELS | Models are stored in /usr/share/ollama/.ollama/models on Linux; set this variable to use another directory, and give the ollama user read and write access to it, for example with sudo chown -R ollama:ollama <directory> |
Model files are large (the Qwen3 8B download is 5.2 GB, and the 14B is 9.3 GB), so check the free disk space of the VPS before you download several models.
Reaching it from outside
The FAQ says that Ollama runs an HTTP server and can be exposed using a proxy server such as Nginx. Its example listens on port 80, sets a server_name, forwards requests with proxy_pass http://localhost:11434 and sets the Host header to localhost:11434.
The FAQ page that we checked does not describe any authentication for the Ollama server. Our recommendation, not a statement from the documents: keep the default address and reach the server through an SSH tunnel, or, if you put a proxy in front, add authentication and HTTPS there and restrict the port in the firewall to the addresses you use.
Update, check and remove
- Update by running the install script again, or by downloading the tarball again with the manual steps.
- To install a specific version, set OLLAMA_VERSION in front of the script, for example curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.5.7 sh. The version numbers are on the GitHub releases page.
- To run a model from the library, use the command shown on its page, for example ollama run qwen3:4b.
- To remove the service, run sudo systemctl stop ollama and sudo systemctl disable ollama, then delete the unit file. The documentation also lists removing the libraries, the binary, and the models, user and group in /usr/share/ollama.
Sources
Read next
- Calling a local model from your code: structured outputs and the OpenAI-compatible API in OllamaConstrain a local model's output with a JSON Schema and call Ollama from OpenAI-style clients: what the official docs say, and what is not supported.
- What can a local model do? Tools, thinking, vision and embeddings in Ollama, and what fits on a VPSTools, thinking, vision, embeddings: what the labels in the Ollama library mean, which models carry them, and how big the downloads are.
- How much memory does a local LLM need on a VPS? Model file plus KV cacheModel file plus KV cache: what the Ollama library, the Ollama FAQ and the official model configurations say about the memory a local LLM needs on a VPS.
- Running AI on your own VPS: how much memory to plan for, and what not to exposePlanning to run Ollama or Dify on a VPS? What the official docs say about memory, CPU-only inference and the ports to keep closed.