Embeddings and Similarity for LLMs: Searching Text by Meaning

Version note: The examples in this post were tested with Ollama 0.34 on a local embedding model (nomic-embed-text), using .NET 10 (Microsoft.Extensions.AI 10.11.0, OllamaSharp 5.5.2) and Node 22.

Guess First

Think of three sentences: "Your order has shipped.", "Your package is on its way." and "Your order has not shipped." Which one is closer in meaning to the first? If we asked an embedding model, which would get the higher similarity score?

For a human the answer is obvious: the second sentence says the same thing in other words, the third says the opposite. We will see the model's answer with real scores in the middle of this post; the result is a good starting point for understanding what an embedding is and what it isn't.

What Is an Embedding?

An embedding is the output of a model that turns a text into a fixed-length array of numbers, a vector. Chat models generate text; embedding models don't generate text, they give you the text's "coordinates of meaning". Texts with similar meaning land on nearby points in that space.

A map analogy works. You describe a city with two numbers, latitude and longitude, and nearby cities have nearby numbers. An embedding does the same with hundreds of dimensions instead of two. The individual dimensions have no meaning of their own; you won't find one labelled "space" or "food". What matters is the angle between two vectors.

In this post we use the nomic-embed-text model in Ollama. According to ollama show it is a small model with 137M parameters and F16 weights, very light next to chat models. Its only job is producing vectors: its capabilities list contains only embedding.

The First Vector

The embed method of the ollama package takes a string or an array of strings and returns a vector for each:

import ollama from 'ollama';

const response = await ollama.embed({
  model: 'nomic-embed-text',
  // nomic-embed-text expects a task prefix: search_query / search_document
  input: ['search_query: a book about surviving on another planet'],
});

const vector = response.embeddings[0];
// Length of the vector (L2 norm)
const norm = Math.sqrt(vector.reduce((sum, x) => sum + x * x, 0));

console.log('dimensions:', vector.length);
console.log('first 5:', vector.slice(0, 5).map((x) => x.toFixed(4)).join(', '));
console.log('norm:', norm.toFixed(4));
console.log('prompt tokens:', response.prompt_eval_count);

$ npx tsx embed-basics.ts
dimensions: 768
first 5: 0.0021, 0.0249, -0.2053, -0.0557, 0.0372
norm: 1.0000
prompt tokens: 13

Each of the four lines tells you something. The vector has 768 dimensions; a short or long query doesn't change that. You can't read any meaning from the first five numbers, which is expected. The norm is 1: the vectors Ollama returns are unit length, and we will see the practical benefit shortly. The last line reminds you that embedding costs tokens too; nothing is generated, but the text is read.

The search_query: at the start of the text isn't decoration. The nomic-embed-text model card says the text must start with a task instruction prefix: search_document: for documents to be searched, search_query: for search queries, clustering: for clustering and classification: for classification. The model places the same text slightly differently depending on the prefix.

Writing Cosine Similarity by Hand

The standard way to measure how similar two vectors are is cosine similarity: the cosine of the angle between them. The value is between -1 and 1; 1 means the same direction, 0 unrelated, -1 the opposite direction. The formula is short: the dot product divided by the product of the two vectors' lengths. Let's do a semantic search over a book catalog. The data is in books.ts, a title and a one-sentence description per book:

export interface Book {
  title: string;
  description: string;
}

export const books: Book[] = [
  { title: 'Dune', description: 'A noble family takes control of a desert planet that holds the most valuable resource in the galaxy.' },
  { title: 'The Martian', description: 'An astronaut stranded alone on Mars must grow food and fix his equipment to survive until rescue.' },
  { title: 'Project Hail Mary', description: 'A science teacher wakes up on an interstellar ship with no memory and must save Earth from a dying sun.' },
  { title: 'The Hobbit', description: 'A quiet homebody joins a band of dwarves on a quest to win back their mountain from a dragon.' },
  { title: 'Gone Girl', description: 'On their fifth anniversary a wife disappears, and the husband becomes the main suspect.' },
  { title: 'The Joy of Cooking', description: 'A classic kitchen reference with thousands of recipes, from simple breakfasts to holiday dinners.' },
  { title: 'Clean Code', description: 'Practical advice for software developers on writing readable, maintainable programs.' },
  { title: 'Sapiens', description: 'A short history of humankind, from hunter-gatherers to the modern economy.' },
];

import ollama from 'ollama';
import { books } from './books.js';

// cos(a, b) = (a . b) / (|a| * |b|)
function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0;
  let normA = 0;
  let normB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    normA += a[i] * a[i];
    normB += b[i] * b[i];
  }
  return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

async function embed(texts: string[]): Promise<number[][]> {
  const response = await ollama.embed({ model: 'nomic-embed-text', input: texts });
  return response.embeddings;
}

// Index once at startup: one vector per book
const index = await embed(books.map((b) => `search_document: ${b.title}. ${b.description}`));

async function search(query: string, topK = 3): Promise<void> {
  const [queryVector] = await embed([`search_query: ${query}`]);
  const ranked = books
    .map((book, i) => ({ title: book.title, score: cosineSimilarity(queryVector, index[i]) }))
    .sort((x, y) => y.score - x.score)
    .slice(0, topK);
  console.log(`\n"${query}"`);
  for (const r of ranked) {
    console.log(`  ${r.score.toFixed(3)}  ${r.title}`);
  }
}

// For comparison: a naive keyword search over the same descriptions
function keywordSearch(query: string): string[] {
  const words = query.toLowerCase().split(/\W+/).filter((w) => w.length > 3);
  return books
    .filter((b) => words.some((w) => b.description.toLowerCase().includes(w)))
    .map((b) => b.title);
}

for (const query of process.argv.slice(2)) {
  await search(query);
  console.log('  keyword:', keywordSearch(query));
}

The flow has two parts. At startup we embed all the books once and keep the vectors in memory; that is our index. For each search we embed only the query, compare it with every book vector and take the top three. Next to it, for comparison, I put a naive keyword search that looks for words longer than four letters in the descriptions.

A Small Semantic Search in Memory

$ npx tsx semantic-search.ts "survival on a red planet" "a mystery about a missing person" "a wizard's journey to fight a dragon"

"survival on a red planet"
  0.647  The Martian
  0.577  Project Hail Mary
  0.549  Dune
  keyword: [ 'Dune' ]

"a mystery about a missing person"
  0.657  Gone Girl
  0.608  Project Hail Mary
  0.574  The Hobbit
  keyword: []

"a wizard's journey to fight a dragon"
  0.644  The Hobbit
  0.505  Project Hail Mary
  0.496  The Joy of Cooking
  keyword: [ 'The Hobbit' ]

In the first query the keyword search found "planet" in Dune's description and returned the wrong book. The Martian's description doesn't say "red planet", it says "Mars"; the embedding knows the two are the same place. In the second query the keyword search found nothing, because Gone Girl's description says "disappears", not "missing". In the third both methods found the right book: keyword search thanks to "dragon", the embedding despite "wizard" and "journey" not appearing in the description at all.

Keep one thing in mind: semantic search doesn't replace keyword search, it complements it. For things that must match exactly, like product codes, order numbers or ISBNs, classic search is still more reliable. Real systems usually combine the two (hybrid search).

The Same Search in C#: IEmbeddingGenerator

Microsoft.Extensions.AI has a separate abstraction for embeddings: IEmbeddingGenerator<TInput, TEmbedding>. OllamaSharp's OllamaApiClient implements both that and IChatClient. Let's index the same catalog in C# and compute cosine similarity both by hand and with TensorPrimitives.CosineSimilarity from System.Numerics.Tensors; that package already comes in as a dependency of Microsoft.Extensions.AI:

using System.Globalization;
using System.Numerics.Tensors;
using Microsoft.Extensions.AI;
using OllamaSharp;

// Print 0.647, not 0,647, whatever the machine locale is
CultureInfo.CurrentCulture = CultureInfo.InvariantCulture;

// OllamaApiClient implements IEmbeddingGenerator too, not only IChatClient
IEmbeddingGenerator<string, Embedding<float>> generator =
    new OllamaApiClient(new Uri("http://localhost:11434"), "nomic-embed-text");

(string Title, string Description)[] books =
[
    ("Dune", "A noble family takes control of a desert planet that holds the most valuable resource in the galaxy."),
    ("The Martian", "An astronaut stranded alone on Mars must grow food and fix his equipment to survive until rescue."),
    ("Project Hail Mary", "A science teacher wakes up on an interstellar ship with no memory and must save Earth from a dying sun."),
    ("The Hobbit", "A quiet homebody joins a band of dwarves on a quest to win back their mountain from a dragon."),
    ("Gone Girl", "On their fifth anniversary a wife disappears, and the husband becomes the main suspect."),
    ("The Joy of Cooking", "A classic kitchen reference with thousands of recipes, from simple breakfasts to holiday dinners."),
    ("Clean Code", "Practical advice for software developers on writing readable, maintainable programs."),
    ("Sapiens", "A short history of humankind, from hunter-gatherers to the modern economy."),
];

// Index once: one vector per book, with nomic's document prefix
GeneratedEmbeddings<Embedding<float>> index =
    await generator.GenerateAsync(books.Select(b => $"search_document: {b.Title}. {b.Description}"));

static float Cosine(ReadOnlySpan<float> a, ReadOnlySpan<float> b)
{
    float dot = 0, normA = 0, normB = 0;
    for (int i = 0; i < a.Length; i++)
    {
        dot += a[i] * b[i];
        normA += a[i] * a[i];
        normB += b[i] * b[i];
    }
    return dot / (MathF.Sqrt(normA) * MathF.Sqrt(normB));
}

foreach (string query in args)
{
    ReadOnlyMemory<float> q = await generator.GenerateVectorAsync($"search_query: {query}");
    Console.WriteLine($"\"{query}\" (dimensions: {q.Length})");

    IEnumerable<(string Title, float Manual, float Library)> ranked = books
        .Select((book, i) => (book.Title,
            Manual: Cosine(q.Span, index[i].Vector.Span),
            Library: TensorPrimitives.CosineSimilarity(q.Span, index[i].Vector.Span)))
        .OrderByDescending(r => r.Manual)
        .Take(3);

    foreach ((string title, float manual, float library) in ranked)
    {
        Console.WriteLine($"  {manual:F3} (TensorPrimitives {library:F3})  {title}");
    }
}

$ dotnet run -- "survival on a red planet" "a mystery about a missing person" "a wizard's journey to fight a dragon"
"survival on a red planet" (dimensions: 768)
  0.647 (TensorPrimitives 0.647)  The Martian
  0.577 (TensorPrimitives 0.577)  Project Hail Mary
  0.549 (TensorPrimitives 0.549)  Dune
"a mystery about a missing person" (dimensions: 768)
  0.657 (TensorPrimitives 0.657)  Gone Girl
  0.608 (TensorPrimitives 0.608)  Project Hail Mary
  0.574 (TensorPrimitives 0.574)  The Hobbit
"a wizard's journey to fight a dragon" (dimensions: 768)
  0.644 (TensorPrimitives 0.644)  The Hobbit
  0.505 (TensorPrimitives 0.505)  Project Hail Mary
  0.496 (TensorPrimitives 0.496)  The Joy of Cooking

The scores match the TypeScript ones to three decimal places; both languages got the same vectors from the same model. Our hand-written function and TensorPrimitives agree as well. In a real project use TensorPrimitives: it is SIMD-accelerated and tested.

I added the CultureInfo line after the first run. This machine's locale is Turkish, so in the first output the scores were printed as 0,647. That's fine for printing to the screen, but if you write the same string somewhere and read it back, it is a quiet source of bugs.

What Scores Tell You and What They Don't

Look at the search results again: for "a wizard's journey to fight a dragon", The Joy of Cooking got 0.496. A cookbook has nothing to do with a dragon question, yet the score isn't anywhere near zero. With this model, unrelated texts still stay in the 0.4–0.5 band. So scores aren't an absolute "relevant/irrelevant" measure, they are a relative ranking. If you want a fixed threshold (e.g. "don't show anything below 0.6"), pick it by measuring on your own data, and measure again when the model changes.

Now back to the opening question. I embedded the sentences with the clustering: prefix and compared them with the first one:

import ollama from 'ollama';

const anchor = 'Your order has shipped.';
const others = [
  'Your package is on its way.',
  'Your order has not shipped.',
  'Your order was cancelled.',
  'The weather is nice today.',
];

const { embeddings } = await ollama.embed({
  model: 'nomic-embed-text',
  input: [anchor, ...others].map((t) => `clustering: ${t}`),
});

// The vectors are already unit length, so the dot product is the cosine similarity
const dot = (a: number[], b: number[]) => a.reduce((sum, x, i) => sum + x * b[i], 0);

others.forEach((text, i) => {
  console.log(`${dot(embeddings[0], embeddings[i + 1]).toFixed(3)}  ${text}`);
});

$ npx tsx pairs.ts
0.873  Your package is on its way.
0.977  Your order has not shipped.
0.837  Your order was cancelled.
0.689  The weather is nice today.

"Your order has not shipped." scored 0.977, while "Your package is on its way.", which means the same thing, scored 0.873. The embedding captures very well that the two sentences talk about the same topic, but barely notices that a small word like "not" flips the meaning. An embedding answers "which texts are about the same thing?", not "which text is true?". This distinction will matter in the RAG post: the embedding finds the right paragraph, and reading that paragraph to produce the right answer is the chat model's job.

Turkish Queries and Multilingual Models

The nomic-embed-text model card lists only English as a language. Our catalog is in English; what if the query is in Turkish?

$ npx tsx semantic-search.ts "uzayda tek başına mahsur kalan bir astronot" "kayıp bir kadın ve şüpheli koca" "yemek tarifleri"

"uzayda tek başına mahsur kalan bir astronot"
  0.481  Project Hail Mary
  0.468  The Hobbit
  0.453  The Martian
  keyword: []

"kayıp bir kadın ve şüpheli koca"
  0.427  Clean Code
  0.415  Gone Girl
  0.408  The Joy of Cooking
  keyword: []

"yemek tarifleri"
  0.407  Project Hail Mary
  0.393  The Hobbit
  0.383  The Joy of Cooking
  keyword: []

(The queries mean "an astronaut stranded alone in space", "a missing woman and a suspicious husband" and "recipes".) In none of the three is the right book first. For "recipes" a science fiction novel is on top, and all the scores are in the unrelated band. The model doesn't understand the Turkish sentence; it most likely gets only a weak signal from words with shared roots such as "astronot".

If you expect Turkish content or Turkish queries, choose a multilingual embedding model. The Ollama library has options such as embeddinggemma (more than 100 languages, 768 dimensions), bge-m3 and qwen3-embedding. They aren't installed on this machine, so I don't show output; but remember that when you change the model you have to rebuild the whole index. Vectors from different models don't live in the same space and can't be compared with each other, even when their dimensions match.

Common Pitfalls

The first is mixing vectors from different models. If you embed the index with one model and the query with another, the scores are meaningless and you get no error message. Store which model produced each vector next to it.

The second is forgetting the prefix or using it inconsistently. I embedded the same catalog without prefixes and tried again; on this small data set the top book didn't change, but the scores moved: for "a wizard's journey to fight a dragon" The Hobbit went from 0.644 to 0.716. If you picked a threshold, that difference invalidates it. Do what the model card says and apply the same rule on the index side and the query side.

The third is embedding a long text in one piece. Ollama on this machine reports a context length of 2048 tokens for nomic-embed-text, and the truncate option of /api/embed is on by default; text beyond the limit is cut off silently. Squeezing a whole book into a single vector also averages away its details. The fix is splitting the text into chunks, which is the topic of the next post.

The fourth is recomputing the index on every request. Document vectors don't change; compute them once and store them. The only thing that needs embedding on each request is the query.

A Small Experiment

★☆☆ Add three books of your choice to books.ts and search with words that don't appear in their descriptions. Does the embedding find the right book, and what does the keyword search do?

★★☆ Try the sentences in pairs.ts again with the search_query: and search_document: prefixes. Does the score of the negative sentence change? Find your own "opposite meaning but high score" example.

★★☆ Add a threshold to semantic-search.ts: if the top score is below the value you choose, print "no good match". Measure which threshold works for the English and Turkish queries in this post.

★★★ Add the norm calculation from embed-basics.ts to the C# program and verify that the vectors come back unit length. Then write a version that uses only TensorPrimitives.Dot instead of cosine, and show that the scores don't change.

We can now find text by meaning. In the next post we will combine that with a chat model: we will split our shipping and return policies into chunks, embed them, put the chunks closest to the question into the prompt, and get gemma3:4b to answer correctly with information it has never seen, and to say "I don't know" when it doesn't know. That is called RAG.