Watch the memory of Ollama on a small VPS
See what Ollama holds in memory, how long it stays there and which settings change it: ollama ps, keep_alive and the limits, from the official FAQ.
Published: Tools you can use: Local LLM Memory Estimator
~/ / Run a local LLM on your server
Run a local LLM on your server
Tools for running Ollama on your own computer or a VPS, in the order you need them. Estimate the memory a model needs, keep the server running with systemd, publish it safely behind nginx, and call its API. When something does not work, read its responses, the model's details and the server log.
Tools, in the order you use them
- Estimate the memory a local LLM needs: model file plus KV cacheLocal LLM Memory EstimatorEstimate the memory a local LLM needs on a VPS: model file plus KV cache
- Build Ollama's systemd override: OLLAMA_HOST, OLLAMA_MODELS, KEEP_ALIVE and moreOllama systemd Override GeneratorBuild the systemd override for Ollama's environment variables, quoted the way systemd reads them
- Build an nginx config for Ollama: HTTPS, basic auth, allow list, streamingOllama nginx Config GeneratorBuild an nginx server block for Ollama with HTTPS, a password and streaming
- Build a curl request to Ollama's /api/chat: format, tools, think, optionsOllama API Request BuilderBuild a curl request to Ollama's /api/chat with format, tools, think and options
- Convert chat requests and responses between Ollama /api/chat and /v1/chat/completionsOllama OpenAI ConverterConvert chat requests and responses between Ollama /api/chat and /v1/chat/completions
- Join a streamed Ollama response and see its speed and stop reasonOllama Stream ReaderJoin a streamed Ollama response back together and see why it stopped and how fast it ran
- Read ollama show: context, layers, capabilities, license, KV cacheOllama Model Info ReaderRead ollama show or /api/show: context, layers, capabilities, license and KV cache size
- Read an Ollama server log: memory, GPU layers, context, speed, warningsOllama Log ReaderRead an Ollama server log: memory per model load, GPU layers, context, speed and warnings
Articles on this topic
Connect Dify to Ollama on your own VPS
Run Dify on your own VPS and use Ollama as its model: install, the plugin fields, the Docker address problem and embeddings, from the official documentation.
Published: Tools you can use: Local LLM Memory Estimator
Start a local LLM without an expensive PC: a beginner's guide to using a VPS
No powerful PC? Rent a VPS and run a small local LLM from your browser: six steps from memory estimate to a protected chat screen, with the limits stated honestly.
Published: Tools you can use: Local LLM Memory Estimator, LLM API Cost Calculator
Open WebUI on a VPS: run it in Docker and connect it to Ollama
Run Open WebUI with Docker and connect it to Ollama on your own server: the commands, the first account, the two connection types and how to update.
Published: Tools you can use: Local LLM Memory Estimator
Calling a local model from your code: structured outputs and the OpenAI-compatible API in Ollama
Constrain a local model's output with a JSON Schema and call Ollama from OpenAI-style clients: what the official docs say, and what is not supported.
Published: Tools you can use: JSON Schema Validator, LLM JSON Repair, Ollama OpenAI Converter
What can a local model do? Tools, thinking, vision and embeddings in Ollama, and what fits on a VPS
Tools, thinking, vision, embeddings: what the labels in the Ollama library mean, which models carry them, and how big the downloads are.
Published: Tools you can use: Local LLM Memory Estimator