~/ / AI articles / Local LLMs on a VPS / Connect Dify to Ollama on your own VPS
Connect Dify to Ollama on your own VPS
Dify is a platform for building AI apps that you can run on your own server. Ollama runs the model. This article connects the two on one VPS: how big the server has to be, how to start Dify, which fields the Ollama plugin asks for, and why the address that works on a laptop can fail inside Docker. It also says where the documentation stops, so that you know which parts to test yourself.
Published: · 6 min read · Facts checked against the official documentation and repositories on 2026-10-27. Dify changes quickly, so read the release notes of the version you install.
What you need before you start
The Dify self-hosting guide lists the minimum for Dify itself. The model needs its own memory on top of that, so the two numbers have to be added up. The estimator below gives a figure for the model. Adding it to Dify's minimum is our rule of thumb, not a statement from the Dify documentation.
| Item | What the Dify guide says |
|---|---|
| CPU | CPU >= 2 Core |
| Memory | RAM >= 4 GiB |
| Docker Compose | Dify requires Docker Compose 2.24.0 or later. |
| Git, curl, jq | The clone command below uses git, curl and jq. |
Try it: Local LLM Memory Estimator
Estimate
- Model weights
- 4.84 GiB
- KV cache
- 0.56 GiB
- Reserve
- 1.00 GiB
- Total
- 6.41 GiB (6.88 GB)
- Smallest common memory size that holds it
- 8 GiB
This is an estimate. It does not include the runtime's compute buffers, and it assumes that every layer has the same KV shape. Measure a real run before you choose a plan.
Model data checked on 2026-10-26.
Sources: Ollama library · Model configuration (config.json) · Ollama FAQ
Everything runs in your browser. Nothing is uploaded.
Step 1: start Dify with Docker Compose
These are the commands of the Dify self-hosting guide. The first one downloads the latest release tag from GitHub, so you start from a release and not from an unfinished branch. At the time of our check the latest release was 1.18.0 (published 2026-10-23).
| Step | Command |
|---|---|
| Clone the latest release | git clone --branch "$(curl -s https://api.github.com/repos/langgenius/dify/releases/latest | jq -r .tag_name)" https://github.com/langgenius/dify.git |
| Go to the Docker folder | cd dify/docker |
| Copy the settings file | cp .env.example .env |
| Create a token for the agent backend | openssl rand -base64 32 |
| Start Dify | docker compose up -d |
Put the token that openssl prints into .env as DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN before you start. The guide says that the agent backend and its sandbox do not start without it.
Step 2: create the administrator account
After the containers are up, the guide says to open the administrator initialization page first: http://your_server_ip/install on a server, or http://localhost/install on your own computer. Create the account there, then log in at http://your_server_ip.
The documentation does not say who can reach that page before the account exists. Our recommendation, not a statement from the documents: create the account right after the start, and read the article about protecting a server before you leave Dify open to the internet.
Step 3: add the Ollama plugin and the model
In the current Dify documentation, model providers are installed from Integrations > Model Provider or from the Marketplace, and then you click Setup on the provider's card. Menu names change between versions, so look for the provider list in your version. The Marketplace page of the Ollama plugin (version 1.0.3 at the time of our check) says to install Ollama, run a model and confirm that its local API works before you add it to Dify.
These are the fields of the plugin, taken from its README and its Marketplace page.
| Field | What to enter |
|---|---|
| Model Name | The Ollama model, for example gemma3, qwen3, deepseek-r1 or gpt-oss. |
| Base URL | The Ollama host root, http://<your-ollama-host>:11434 for a local Ollama. Do not include /api, because the plugin adds /api/chat, /api/generate and /api/embed itself. |
| API Key | Optional for a local Ollama. Required only for Ollama cloud. |
| Model Type | Chat for chat, tool calling and vision. The Marketplace page also lists text embeddings and rerank. |
| Model Context Length | Match the context window of the model. |
| Upper bound for max tokens | The largest value of max tokens (num_predict) that Dify allows. |
| Vision support / Function call support | Choose Yes only for a model that supports it. |
| Attach Dify app_id as request headers | Off by default. When on, Dify adds X-Dify-App-Id and X-Dify-Source headers to its calls to Ollama. Ollama does not use them. |
To know which models support vision and tool calling, see the article about what the models in the Ollama library can do.
Step 4: the address that works from inside Docker
Dify runs in containers, and Ollama runs on the server itself. In a container, localhost is the container, not the server. The plugin README says: use a host address that the Dify container can reach, such as http://host.docker.internal:11434 or a LAN IP address.
The Docker documentation explains the name. The --add-host option supports a special value, host-gateway, that resolves to the internal IP address of the host, and it is conventional to use host.docker.internal for that name. It also says that Docker Desktop resolves the name automatically. The example in the documentation is --add-host host.docker.internal=host-gateway.
| What we checked | Result |
|---|---|
| Does Dify's docker-compose.yaml define host.docker.internal? | No. We found no host.docker.internal and no extra_hosts entry in the file of the main branch. |
| Does the Docker Compose reference describe extra_hosts? | Yes, as hostname mappings in the form HOSTNAME=IP that go into /etc/hosts. The page we read does not mention host-gateway for extra_hosts. |
| Does Ollama listen on addresses other than 127.0.0.1? | Not by default. The Ollama FAQ says it binds 127.0.0.1 port 11434, and that the OLLAMA_HOST environment variable changes the bind address. |
Our reading, not a statement from the documents: on a Linux server you add the host name yourself, or use the address of the server, and you may also have to change the address that Ollama listens on. We did not test the combination. If you change OLLAMA_HOST, port 11434 can be reached from other machines, and Ollama has no password of its own, so close that port to the outside in the firewall. The articles about installing Ollama and about putting nginx in front of it describe the settings.
Step 5: embeddings, and what the plugin does not do
The Marketplace page of the plugin says that it supports streaming, thinking, vision, tool calling and text embeddings, and that the embedding vectors are L2-normalized. For embeddings you add a second model of the type text embeddings, with an embedding model from the Ollama library.
It also lists what is not included. Ollama has no native rerank endpoint, so a rerank model needs an OpenAI-compatible rerank service. Web search is not bundled either. It needs a separate Dify tool plugin or an external workflow.
If the connection fails
- Run ollama ps and check that the model is loaded, and open http://127.0.0.1:11434 on the server to see that Ollama answers.
- Check the Base URL. It is the host root, with the port and without /api.
- Check that the Dify containers can reach the address. The Dify documentation says that its outbound requests are routed through a proxy that blocks internal and private IP ranges. The page names HTTP nodes and image downloads, and it does not say whether requests of model providers take the same path. If you use a private address and it is refused, this is one place to look.
- Read the logs of the containers with docker compose logs. We did not find a list of error messages in the documentation.
Keeping Dify up to date
The guide says that upgrade steps vary between releases, so follow the upgrade guide for the version you move to, which is on the Dify Releases page. After an upgrade, compare .env.example with your .env for new or changed variables. If you changed docker-compose.yaml, apply your changes again to the new release's file, because a release can add services that are required.
Sources
- Dify documentation: Deploy with Docker Compose
- Dify documentation: Model providers
- Dify documentation: Environment variables
- Dify official plugins: Ollama plugin README
- Dify Marketplace: Ollama plugin
- Dify repository: docker-compose.yaml
- Docker documentation: docker container run (--add-host)
- Docker documentation: Compose services (extra_hosts)
- Ollama documentation: FAQ
Read next
- Start a local LLM without an expensive PC: a beginner's guide to using a VPSNo powerful PC? Rent a VPS and run a small local LLM from your browser: six steps from memory estimate to a protected chat screen, with the limits stated honestly.
- Put nginx in front of Ollama: basic authentication, HTTPS and streaming settingsReach Ollama from outside safely: nginx as a proxy, basic authentication, HTTPS, streaming and timeout settings, and a firewall rule, from the official docs.
- Open WebUI on a VPS: run it in Docker and connect it to OllamaRun Open WebUI with Docker and connect it to Ollama on your own server: the commands, the first account, the two connection types and how to update.
- Calling a local model from your code: structured outputs and the OpenAI-compatible API in OllamaConstrain a local model's output with a JSON Schema and call Ollama from OpenAI-style clients: what the official docs say, and what is not supported.