~/ / AI articles / Local LLMs on a VPS / Embeddings with Ollama: the basics for search and RAG

Embeddings with Ollama: the basics for search and RAG

A search that finds passages by meaning, and RAG, where a chat model answers from your own documents, both start with embeddings. Ollama can make them on the same server that runs your chat model. This article covers the request, the models the Ollama library lists, the settings that change the result, and how to compare the vectors, using only what Ollama's documentation and library pages say. Two tools on this page let you try the steps in your browser.

Published: 2026-11-04 · 5 min read · Facts checked against Ollama's documentation and library pages on 2026-11-04. Models and fields change, so check the current pages before you rely on them.

What an embedding is

Ollama's embeddings page says that embeddings turn text into numeric vectors that you can store in a vector database, search with cosine similarity, or use in RAG pipelines. The vector length depends on the model; the page gives typically 384 to 1024 dimensions.

A search embeds the question with the same model and looks for the stored vectors that are most similar to it.

Make embeddings with /api/embed

The endpoint is POST /api/embed. The request needs model and input, and input can be a single string or an array of strings. With an array, the response holds one vector per string in the embeddings field, in the same order, so a batch of passages needs only one request.

The documentation also shows the command line: ollama run embeddinggemma "Hello world" prints the vector as a JSON array, and text can be piped in with echo.

Field	What the API reference says
model	Required. The model name.
input	Required. A text or an array of texts.
truncate	Default true: inputs that exceed the context window are truncated. If false, an error is returned.
dimensions	The number of dimensions to generate embeddings for.
keep_alive	How long the model stays loaded.
options	Runtime options, such as num_ctx for the context length.

Ollama also has an OpenAI-compatible endpoint, /v1/embeddings. Its documentation lists model, input as a string or an array of strings, encoding format and dimensions as supported, and arrays of tokens and user as not supported.

Embedding models in the Ollama library

The Ollama library has an embedding category. The table shows six of the models listed there, with the size and context window that each model's tags page gives for the latest tag. Four of the six are under 1GB.

Model	Size of the latest tag	Context window
all-minilm	46MB	512
nomic-embed-text	274MB	2K
embeddinggemma	622MB	2K
mxbai-embed-large	670MB	512
bge-m3	1.2GB	8K
qwen3-embedding	4.7GB	40K

Other tags change the size: for example, the library lists qwen3-embedding:0.6b at 639MB with a 32K context window, and embeddinggemma:300m-qat-q4_0 at 239MB. We did not compare the quality of these models; the library pages do not rank them. When we tried all-minilm on Ollama 0.40.11, ollama show listed a context length of 512 and a num_ctx parameter of 256, and a long input was cut at 256 tokens. Check ollama show for the model you use.

Long input is cut off without an error

The context window is counted in tokens. Because truncate defaults to true, a text longer than the model's context window is shortened and still returns a vector, with no error. A long document sent as one input can lose its end this way.

Two things help. Split long documents into smaller chunks and embed each chunk, which also lets a search return the part that answers the question. And send truncate set to false, so that Ollama returns an error instead of shortening a chunk that is still too long.

The chunker below splits text at paragraphs, lines, sentences and spaces, with an overlap, and builds one /api/embed request for all the chunks. It counts characters, not tokens, because each model has its own tokenizer, so leave a margin below the context window.

Try it: RAG Text Chunker

Text to splitSeparate paragraphs with a blank line

Maximum chunk length (characters)From 50 to 20000Overlap (characters)0 for none. Must be smaller than the maximum lengthEmbedding modelThe same model you will use for the searchOllama addresshttp://localhost:11434

Fail instead of truncating ("truncate": false)By default, Ollama truncates input that is longer than the model's context window.

Lengths are in characters. The model's context window is in tokens, and how many tokens a text uses depends on the model, so leave a margin.

curl command

Run it on a machine that can reach Ollama. The response has one vector per chunk, in this order.

curl http://localhost:11434/api/embed -d '{
  "model": "embeddinggemma",
  "input": [
    "Embeddings turn text into numeric vectors. You can store them in a vector database and search them with cosine similarity, or use them in a RAG pipeline.\n\nThe length of each vector depends on the embedding model. Use the same model when you index your documents and when you embed a question.\n\n",
    "Ollama'\''s /api/embed endpoint takes a single string or an array of strings. By default, input that is longer than the model'\''s context window is truncated. Set truncate to false to get an error instead.\n\n",
    "Split long documents into smaller chunks before you embed them, so that each chunk fits in the context window and a search can return the part that answers the question."
  ],
  "truncate": false
}'

Chunks

3 chunks. The longest has 294 characters.

#0 · 294 characters · starts at offset 0
Embeddings turn text into numeric vectors. You can store them in a vector database and search them with cosine similarity, or use them in a RAG pipeline. The length of each vector depends on the embedding model. Use the same model when you index your documents and when you embed a question.
#1 · 202 characters · starts at offset 294
Ollama's /api/embed endpoint takes a single string or an array of strings. By default, input that is longer than the model's context window is truncated. Set truncate to false to get an error instead.
#2 · 169 characters · starts at offset 496
Split long documents into smaller chunks before you embed them, so that each chunk fits in the context window and a search can return the part that answers the question.

JSON body for /api/embed

{
  "model": "embeddinggemma",
  "input": [
    "Embeddings turn text into numeric vectors. You can store them in a vector database and search them with cosine similarity, or use them in a RAG pipeline.\n\nThe length of each vector depends on the embedding model. Use the same model when you index your documents and when you embed a question.\n\n",
    "Ollama's /api/embed endpoint takes a single string or an array of strings. By default, input that is longer than the model's context window is truncated. Set truncate to false to get an error instead.\n\n",
    "Split long documents into smaller chunks before you embed them, so that each chunk fits in the context window and a search can return the part that answers the question."
  ],
  "truncate": false
}

Request fields as given in the Ollama API reference for /api/embed, checked on 2026-11-04. Ollama API: Generate embeddings

Everything is done in your browser. The tool only writes the command and sends nothing.

Use one model for indexing and searching

The embeddings page says to use the same embedding model for both indexing and querying. The vector length depends on the model, so vectors from two different models may not even have the same number of dimensions.

The same goes for dimensions: if you set it when you index your documents, set the same value when you embed the questions. If you change the model or this setting, embed the stored documents again.

Compare the vectors

The documentation recommends cosine similarity for most semantic search use cases, and says that /api/embed returns L2-normalized, unit-length vectors. For unit-length vectors the cosine similarity and the dot product are the same number, so a vector database set to either will rank the results the same way.

To check a model on your own texts before you build anything, embed a question and a few passages in one request, with the question first. Paste the response below: the passages are ranked by cosine similarity to the question, and the tool shows whether every vector is unit length and has the same dimensions.

Try it: Embedding Similarity Calculator

Embedding response (JSON)From /api/embed, /v1/embeddings, or an array of vectorsLabels, one per line (optional)Paste the input texts in the same order

The example uses made-up, unit-length vectors with 4 dimensions so that you can read them. Real embeddings have hundreds or thousands of dimensions.

Read as an Ollama /api/embed response: 4 vectors with 4 dimensions. All vectors have a length of 1, so cosine similarity and dot product are equal.

Compare against

Ranking by cosine similarity

Compared with #0 How do I keep a model loaded in Ollama?. Higher is more similar.

Rank	Text	Cosine	Dot product	Distance
1	#1 keep_alive controls how long the model stays in memory	0.9962	0.9962	0.0876
2	#2 Protect Ollama with nginx and a password	0.4037	0.4037	1.0921
3	#3 Tomorrow's weather forecast	0.0100	0.0100	1.4071

Cosine similarity matrix

Each cell compares the vector in the row with the vector in the column.

	#0	#1	#2	#3
#0	1.000	0.996	0.404	0.010
#1	0.996	1.000	0.475	-0.005
#2	0.404	0.475	1.000	-0.377
#3	0.010	-0.005	-0.377	1.000

Vectors

Text	Dimensions	Length (L2 norm)	Unit length
#0 How do I keep a model loaded in Ollama?	4	1.000000	Yes
#1 keep_alive controls how long the model stays in memory	4	1.000000	Yes
#2 Protect Ollama with nginx and a password	4	1.000001	Yes
#3 Tomorrow's weather forecast	4	1.000000	Yes

Based on Ollama's embeddings documentation and API reference, checked on 2026-11-04. Ollama: Embeddings / Ollama API: Generate embeddings

Everything is calculated in your browser. Nothing is sent anywhere.

Memory on a small server

An embedding model is loaded like any other model. The Ollama FAQ says that OLLAMA_MAX_LOADED_MODELS, the number of models that can be loaded at the same time if they fit in memory, defaults to 3 times the number of GPUs, or 3 for CPU inference. So a chat model and an embedding model can stay loaded together when the memory allows it.

Our article on watching Ollama's memory explains ollama ps, keep_alive and the other settings that decide what stays in memory.