~/ / AI articles / Local LLMs on a VPS / What can a local model do? Tools, thinking, vision and embeddings in Ollama, and what fits on a VPS
What can a local model do? Tools, thinking, vision and embeddings in Ollama, and what fits on a VPS
A local model is more than a chat box. The Ollama library labels models with the things they can do, and the documentation describes each of them: calling tools, showing their reasoning, reading images, and turning text into vectors for search. This article explains what each label means, lists ten models with their labels and download sizes, and shows how to match what you want to do with the memory of your server.
Published: · 5 min read · Labels, sizes and context windows checked against the Ollama library and documentation on 2026-10-26. The library changes often, so check the page of the tag you plan to use.
The four labels
The Ollama library marks each model with the capabilities it supports. Four of them matter for what you build on a server. The documentation describes them as follows.
| Label | What the documentation says | How you use it |
|---|---|---|
| tools | Tool calling, also called function calling, lets a model invoke tools and incorporate their results into its replies. It supports a single tool call, parallel tool calls and a multi-turn agent loop. | You list the available tools in the tools field of a request to /api/chat. When the model calls a tool you run it and send the output back as a message with the role tool. |
| thinking | Thinking-capable models output their reasoning separately from their final answer, so you can audit the model's steps or hide the trace. | You set the think parameter on a request. The reasoning comes back in message.thinking and the answer in message.content. |
| vision | Vision models accept images alongside text, so the model can describe, classify and answer questions about what it sees. | You pass an images array in the message. The REST API expects base64-encoded image data. |
| embedding | Embeddings are numeric vectors that you can store in a vector database, search with cosine similarity, or use in RAG pipelines. | You call /api/embed. The documentation recommends cosine similarity for most semantic search, and the same embedding model for indexing and for querying. |
Ten models and their labels
This table lists the labels that the Ollama library shows for each model family, together with one tag, the size shown on that tag's row of the tags list, and its context window. The size is a download size, so it is the least memory the model needs, not the amount to plan for.
| Tag | Labels | Download size | Context window | What the library says it is |
|---|---|---|---|---|
| qwen3:4b | tools, thinking | 2.5 GB | 256K | Qwen3 is the latest generation of large language models in the Qwen series |
| llama3.2:3b | tools | 2.0 GB | 128K | Meta's Llama 3.2 goes small with 1B and 3B models |
| gemma3:4b | vision | 3.4 GB | 128K | The most capable model that runs on a single GPU |
| qwen3-vl:4b | vision, tools, thinking | 3.3 GB | 256K | The most powerful vision-language model in the Qwen family to date |
| deepseek-r1:7b | tools, thinking | 4.7 GB | 128K | A family of open reasoning models |
| qwen2.5-coder:7b | tools | 4.7 GB | 32K | The latest series of code-specific Qwen models |
| mistral:7b | tools | 4.4 GB | 32K | The 7B model released by Mistral AI, updated to version 0.3 |
| gpt-oss:20b | tools, thinking | 14 GB | 128K | OpenAI's open-weight models for reasoning and agentic tasks |
| nomic-embed-text | embedding | 274 MB | 2K | An open embedding model with a large token context window |
| embeddinggemma | embedding | 622 MB | 2K | A 300M parameter embedding model from Google |
The labels belong to the model family, and a family has several sizes. Open the page of the exact tag you plan to use to see its input types and size before you download it.
Matching a capability to a task
The documentation names the uses of each capability, and this is our reading of what they mean for a server you run yourself, not a claim about quality:
- Tools: an assistant that can look something up or trigger an action in your own system, because the model asks for a tool and your code runs it. The tool definition you send has to follow the shape of the API, which the Tool Definition Checker can check.
- Thinking: answers where you want to see or log the model's reasoning separately, for example while you test a prompt. The reasoning arrives before the answer, so a response takes more tokens.
- Vision: describing or classifying an image, or answering a question about it, with a model that accepts images alongside text.
- Embedding: search by meaning and retrieval for RAG. The two embedding models in the table are small downloads, 274 MB and 622 MB, which is far less than a chat model.
What fits on a VPS
Memory decides what you can run. The download sizes above are a floor, and the context length adds a KV cache on top. For the Qwen3 and Mistral 7B models the calculator below uses the official configuration values, so you can see the total for your context length and for the plan you are considering.
For the other models in the table the calculator has no preset, because it needs the number of layers, KV heads and the head dimension from each model's configuration file. You can type them in as a custom model.
Try it: Local LLM Memory Estimator
Estimate
- Model weights
- 4.84 GiB
- KV cache
- 0.56 GiB
- Reserve
- 1.00 GiB
- Total
- 6.41 GiB (6.88 GB)
- Smallest common memory size that holds it
- 8 GiB
This is an estimate. It does not include the runtime's compute buffers, and it assumes that every layer has the same KV shape. Measure a real run before you choose a plan.
Model data checked on 2026-10-26.
Sources: Ollama library · Model configuration (config.json) · Ollama FAQ
Everything runs in your browser. Nothing is uploaded.
Choosing
- Start from the capability you need, then pick the smallest model that carries its label, and measure memory and speed on a test server before you commit to a plan.
- Check the context window in the table against your use: the two embedding models list 2K, and mistral:7b and qwen2.5-coder:7b list 32K.
- Several capabilities can sit on one model. Qwen3 carries tools and thinking, and Qwen3-VL adds vision.
- The library lists new models often. Treat this table as a snapshot of the date above.
Sources
Read next
- Calling a local model from your code: structured outputs and the OpenAI-compatible API in OllamaConstrain a local model's output with a JSON Schema and call Ollama from OpenAI-style clients: what the official docs say, and what is not supported.
- Install Ollama on a VPS and keep it running: install, systemd service, settings and updatesInstall Ollama on a Linux VPS and keep it running: the install command, the systemd service, environment variables, model storage and updates, from the official docs.
- How much memory does a local LLM need on a VPS? Model file plus KV cacheModel file plus KV cache: what the Ollama library, the Ollama FAQ and the official model configurations say about the memory a local LLM needs on a VPS.
- Running AI on your own VPS: how much memory to plan for, and what not to exposePlanning to run Ollama or Dify on a VPS? What the official docs say about memory, CPU-only inference and the ports to keep closed.