Choosing an Embedding Model: Dimensions, Multilinguality, Turkish and Cost

Version note: The code in this post was built and run with .NET 10, Microsoft.Extensions.AI 10.11.0 and OllamaSharp 5.5.2 against nomic-embed-text (v1.5) on Ollama 0.34.4; vector sizes were measured with pgvector 0.8.7.

Guess First

In the previous post minimal RAG (Retrieval-Augmented Generation) never found the right chunk for five of the 30 questions. This time we look for the fault not in the chat model but in the embedding model. The experiment is simple: I translated Bookstore's documents and the 30 questions into English section by section; content, headings and section structure are the same. Same embedding model, same code. On the Turkish set the right section comes first for 16 of the 30 questions. On the English set, how many? Close to 16, or very different?

Reading the Leaderboard and Picking Candidates

When choosing an embedding model, the first place everyone looks is the MTEB (Massive Text Embedding Benchmark) leaderboard. Its multilingual version, MMTEB (Massive Multilingual Text Embedding Benchmark), was published in 2025; according to the paper it contains more than 500 tasks in 10 task categories across over 250 languages. Picking the top model by a single "average score" column is tempting and wrong. That average mixes very different tasks such as classification, clustering, STS (Semantic Textual Similarity) and retrieval; only retrieval tasks matter for RAG. On top of that, the average is dominated by English or multilingual tasks; you only see how a model behaves on Turkish retrieval with both a language filter and a task filter.

Besides the score, look at three more columns: parameter count (will it run on CPU (Central Processing Unit) or GPU (Graphics Processing Unit)?), embedding dimensions (storage and search cost) and max tokens (will your chunks fit?). And the license: not every model on the leaderboard is open for commercial use.

For candidates you can run locally, Ollama's embedding catalog is a good start. On 8 November 2026 the ones explicitly described as multilingual were bge-m3 (567M), snowflake-arctic-embed2 (568M), nomic-embed-text-v2-moe, paraphrase-multilingual (278M) and the 278M version of granite-embedding. qwen3-embedding (0.6b, 4b, 8b) and embeddinggemma (300m) also belong on the list; whether they are multilingual is stated by the model's own model card, not the Ollama page. On the cloud side, OpenAI's text-embedding-3-small produces 1536 dimensions and text-embedding-3-large 3072.

The measurements in this post use a single model: nomic-embed-text. Its latest tag on Ollama is v1.5: 137M parameters, 768 dimensions and a 2048-token context in Ollama. I did not download new models in this environment, so I could not measure the multilingual candidates; the post therefore answers not "which model is best?" but "how do you evaluate a model on your own data?". With the setup below you only need to change the model name and rerun the same measurement.

The Measurement Setup: recall@k and MRR

The 30 questions from the previous post record which section of which document holds each answer. The index uses the standard chunker, one chunk per Markdown section (53 chunks). Vectors live in memory; search is cosine similarity:

public sealed class InMemoryIndex(IEmbeddingGenerator<string, Embedding<float>> embedder,
    string documentPrefix = "search_document: ", string queryPrefix = "search_query: ", int? dimensions = null)
{
    private List<(Chunk Chunk, ReadOnlyMemory<float> Vector)> _items = [];

    public async Task AddAsync(IReadOnlyList<Chunk> chunks)
    {
        EmbeddingGenerationOptions options = new() { Dimensions = dimensions };
        GeneratedEmbeddings<Embedding<float>> vectors =
            await embedder.GenerateAsync(chunks.Select(c => documentPrefix + c.Content), options);
        _items = [.. _items, .. chunks.Zip(vectors, (c, v) => (c, v.Vector))];
    }

    public async Task<List<Hit>> SearchAsync(string query, int k, Func<Chunk, bool>? filter = null)
    {
        ReadOnlyMemory<float> q = await embedder.GenerateVectorAsync(queryPrefix + query,
            new EmbeddingGenerationOptions { Dimensions = dimensions });
        return _items
            .Where(x => filter is null || filter(x.Chunk))
            .Select(x => new Hit(x.Chunk, TensorPrimitives.CosineSimilarity(q.Span, x.Vector.Span)))
            .OrderByDescending(h => h.Score)
            .Take(k)
            .ToList();
    }
}

The Dimensions option comes from Microsoft.Extensions.AI's EmbeddingGenerationOptions; OllamaSharp maps it to the dimensions field of Ollama's /api/embed request. The measurement loop computes two metrics:

// Hit: is the chunk from the expected section of the expected document?
List<Question> questions = QuestionSet.Load(Path.Combine(corpus, "questions.json"));
int at1 = 0, at5 = 0; double mrr = 0;
foreach (Question q in questions)
{
    List<Hit> hits = await index.SearchAsync(q.QuestionText, 5);
    int rank = hits.FindIndex(h => QuestionSet.IsHit(q, h.Chunk)) + 1;
    if (rank == 1) at1++;
    if (rank > 0) { at5++; mrr += 1.0 / rank; }
    }

recall@k is the share of questions whose right section is in the first k results. In RAG, k is usually the number of chunks you put into the prompt; if recall@5 is low, the model never sees the right text. MRR (Mean Reciprocal Rank) rewards the position of the right section: first place scores 1, second 0.5, fifth 0.2, and not found 0. Recall answers "did it find it", MRR answers "how high did it find it".

The Turkish Penalty: Same Content in Two Languages

$ dotnet run -- recall "search_document: " "search_query: " 0
chunks=53 dims=768 recall@1=16/30 recall@5=25/30 MRR=0.668

$ CORPUS=corpus_en dotnet run -- recall "search_document: " "search_query: " 0
chunks=53 dims=768 recall@1=26/30 recall@5=29/30 MRR=0.894

What was your guess? Same model, same content, same section structure: in Turkish the right section is first for 16 of 30 questions, in English for 26. MRR goes from 0.668 to 0.894. The gap is consistent with nomic-embed-text's model card listing English as its only language. The model still catches something in Turkish text, but it cannot bridge "bedava" and "ücretsiz" (two words for free) or "turn into money" and "cannot be converted to cash".

Part of the gap starts in the tokenizer. nomic-embed-text is based on BERT (Bidirectional Encoder Representations from Transformers) and comes with an English vocabulary. The prompt_eval_count field in Ollama's embed response tells you how many tokens the model saw:

// Same meaning, two languages: how many tokens does nomic-embed-text's tokenizer count?
public static async Task TokenCountsAsync(OllamaApiClient ollama)
{
    string[] texts =
    [
        "search_query: Bir kitabı kaç gün içinde iade edebilirim?",
        "search_query: How many days do I have to return a book?",
        "search_query: Kitapsever seviyesine nasıl geçerim?",
        "search_query: How do I move up to the Book Lover tier?"
    ];
    foreach (string text in texts)
    {
        EmbedResponse r = await ollama.EmbedAsync(new EmbedRequest { Model = "nomic-embed-text", Input = [text] });
        Console.WriteLine($"{r.PromptEvalCount,3} tokens  {text}");
    }
}

$ dotnet run -- emb-tokens
 24 tokens  search_query: Bir kitabı kaç gün içinde iade edebilirim?
 17 tokens  search_query: How many days do I have to return a book?
 21 tokens  search_query: Kitapsever seviyesine nasıl geçerim?
 17 tokens  search_query: How do I move up to the Book Lover tier?

The same question is 24 tokens in Turkish and 17 in English. Turkish words are not split by their suffixes but by whatever pieces the vocabulary happens to contain; a word like "edebilirim" ("I can do") is not a meaningful unit to the model but several meaningless fragments. We saw the same effect in the chat model's tokenizer in the LLM (Large Language Model) Fundamentals post on tokens; with an embedding model the price is paid in retrieval quality.

The practical consequence: do not choose a model for Turkish content by its English benchmark score. Put a multilingual model at the top of the candidate list and measure with your own question set. This series continues with nomic-embed-text; that also makes it concrete, in later posts, why hybrid search and reranking are needed.

Prefixes: search_query and search_document

The nomic-embed-text v1.5 model card is explicit: adding a task prefix to the text is required. For RAG, documents start with search_document: and questions with search_query: ; clustering and classification have their own prefixes. Other models want different conventions (some query: and passage: , some a long instruction sentence). What happens if we give no prefix at all, or only on documents?

$ dotnet run -- recall "" "" 0
chunks=53 dims=768 recall@1=17/30 recall@5=24/30 MRR=0.664
$ dotnet run -- recall "search_document: " "" 0
chunks=53 dims=768 recall@1=17/30 recall@5=25/30 MRR=0.684
$ CORPUS=corpus_en dotnet run -- recall "" "" 0
chunks=53 dims=768 recall@1=27/30 recall@5=29/30 MRR=0.933

The result was not what I expected: in Turkish the run without prefixes found one more first place, and in English the MRR without prefixes is slightly higher. Honesty is needed here: on a 30-question set one question is a 3.3-point difference, and every difference in this table is inside that noise. With this data I cannot say "prefixes don't help"; I can only say I saw no measurable effect on this set. Do what the model card says, but remember the real rule: documents and queries must be embedded under the same convention, and that convention must be recorded with the vectors.

Dimensions, Matryoshka and Cost

nomic-embed-text v1.5 was trained with Matryoshka Representation Learning: the first 512, 256, 128 or 64 dimensions of a vector are meaningful on their own. According to the model card its MTEB score is 62.28 at 768 dimensions, 61.04 at 256 and 56.10 at 64. In Ollama the dimensions parameter truncates the vector and renormalizes it; I checked the length of a 128-dimension output and it was 1.0. On our set:

$ dotnet run -- recall "search_document: " "search_query: " 512
chunks=53 dims=512 recall@1=17/30 recall@5=25/30 MRR=0.673
$ dotnet run -- recall "search_document: " "search_query: " 256
chunks=53 dims=256 recall@1=18/30 recall@5=24/30 MRR=0.684
$ dotnet run -- recall "search_document: " "search_query: " 128
chunks=53 dims=128 recall@1=14/30 recall@5=24/30 MRR=0.579
$ dotnet run -- recall "search_document: " "search_query: " 64
chunks=53 dims=64 recall@1=14/30 recall@5=22/30 MRR=0.566

512 and 256 dimensions are on par with 768; at 128 recall@1 drops clearly, and at 64 recall@5 falls too. The cost side of these dimensions can be computed directly. In pgvector a vector value takes 4 × dimensions + 8 bytes and a halfvec 2 × dimensions + 8 bytes:

SELECT pg_column_size(array_fill(0.1::real, ARRAY[768])::vector)  AS v768,
       pg_column_size(array_fill(0.1::real, ARRAY[768])::halfvec) AS h768,
       pg_column_size(array_fill(0.1::real, ARRAY[256])::vector)  AS v256,
       pg_column_size(array_fill(0.1::real, ARRAY[1536])::vector) AS v1536;

 v768 | h768 | v256 | v1536
------+------+------+-------
 3080 | 1544 | 1032 |  6152

For a million chunks a 768-dimension vector column is about 3 GB of raw data alone, 1 GB at 256 dimensions; an ANN (Approximate Nearest Neighbor) index comes on top. In a brute-force search, dimensions turn directly into CPU time. I scanned 100,000 random vectors with TensorPrimitives.CosineSimilarity (on an Apple Silicon Mac, Release build):

$ dotnet run -c Release -- emb-scan 100000
dims=768 vectors=100,000 memory=293 MB scan=14.0 ms/query
dims=256 vectors=100,000 memory=98 MB scan=3.6 ms/query

Dimensions dropped to a third, time to a quarter. With memory going from 293 MB to 98 MB, the data fits the CPU caches better. The table tells you that with a Matryoshka model, 256 dimensions is a serious storage and speed win if your own measurement shows no loss. But you have to re-measure for every language and document set; my "no loss" on 30 questions does not transfer to your data.

Max Input and Silent Truncation

Every embedding model has an input limit. The nomic-embed-text model card says 8192 tokens, but Ollama runs the model with a 2048-token context. What happens to a chunk that exceeds it?

// Input longer than the context: does Ollama cut it silently or fail?
public static async Task LongInputAsync(OllamaApiClient ollama, string longText)
{
    EmbedResponse full = await ollama.EmbedAsync(new EmbedRequest { Model = "nomic-embed-text", Input = [longText] });
    EmbedResponse tail = await ollama.EmbedAsync(new EmbedRequest
    {
        Model = "nomic-embed-text",
        Input = [longText + "\n\nÖNEMLİ: Bu belgenin son maddesi iade süresini 60 güne çıkarır."]
    });
    float similarity = TensorPrimitives.CosineSimilarity(full.Embeddings[0], tail.Embeddings[0]);
    Console.WriteLine($"chars={longText.Length} tokens={full.PromptEvalCount} (with extra paragraph: {tail.PromptEvalCount})");
    Console.WriteLine($"cosine(original, original + extra paragraph) = {similarity:F6}");

    try
    {
        await ollama.EmbedAsync(new EmbedRequest { Model = "nomic-embed-text", Input = [longText], Truncate = false });
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Truncate=false -> {ex.GetType().Name}: {ex.Message}");
    }
}

Here the method appends a Turkish sentence saying "IMPORTANT: the last clause of this document raises the return window to 60 days."

$ dotnet run -- emb-long 1
chars=1709 tokens=728 (with extra paragraph: 754)
cosine(original, original + extra paragraph) = 0.999200

$ dotnet run -- emb-long 6
chars=10254 tokens=2048 (with extra paragraph: 2048)
cosine(original, original + extra paragraph) = 1.000000
Truncate=false -> OllamaException: the input length exceeds the context length

There are two lessons. First: with the return policy repeated six times, 10,254 characters were cut to 2048 tokens and the "return window becomes 60 days" paragraph at the end did not change the vector at all; the cosine is exactly 1. Ollama does not fail by default, it truncates silently. With Truncate = false the request is rejected with an exception. Prefer the latter in production: noticing a truncated chunk later is far more expensive than catching it at ingestion time.

The second is subtler: even below the limit, on a 728-token text, the new paragraph only moved the cosine down to 0.9992. The vector of a long chunk loses the meaning of any single sentence in the average. That is why we choose chunk size not by "does it fit" but by "is the information visible in the vector"; that is the subject of the fourth post. One more note for Turkish: this text averaged 2.3 characters per token, so a Turkish chunk hits the limit sooner than its English equivalent.

Switching Models: The Identity of a Vector

One day you will want to move to a better multilingual model. The most dangerous mistake is leaving the vectors in the index as they are and embedding queries with the new model. The vectors of two models do not live in the same space; even with equal dimensions, the cosine between them is meaningless. If the dimensions differ, you at least get a noisy error:

// Stored vectors have 256 dimensions, the new query has 768
public static async Task MismatchAsync(IEmbeddingGenerator<string, Embedding<float>> embedder)
{
    Embedding<float> stored = (await embedder.GenerateAsync(
        ["search_document: İade süresi 21 gündür."], new EmbeddingGenerationOptions { Dimensions = 256 }))[0];
    ReadOnlyMemory<float> query = await embedder.GenerateVectorAsync("search_query: Kaç gün içinde iade edebilirim?");
    Console.WriteLine($"stored={stored.Vector.Length} dims, query={query.Length} dims, model={stored.ModelId}");
    try
    {
        Console.WriteLine(TensorPrimitives.CosineSimilarity(stored.Vector.Span, query.Span));
    }
    catch (ArgumentException ex)
    {
        Console.WriteLine($"{ex.GetType().Name}: {ex.Message}");
    }
}

$ dotnet run -- emb-mismatch
stored=256 dims, query=768 dims, model=nomic-embed-text
ArgumentException: Input span arguments must all have the same length.

pgvector gives the same protection: the column type fixes the dimension and it refuses to compare vectors of different dimensions.

CREATE TABLE emb_demo (id int, embedding vector(768));
INSERT INTO emb_demo VALUES (1, array_fill(0.1::real, ARRAY[256])::vector);
ERROR:  expected 768 dimensions, not 256

SELECT '[1,2,3]'::vector <=> '[1,2]'::vector;
ERROR:  different vector dimensions 3 and 2

The real problem is switching between two models with the same dimensions: you get no error anywhere, results just quietly get worse. The only defence is to store, next to every vector, the "identity" that produced it: model name and version, dimensions, the prefix convention used. Embedding<float>.ModelId already gives you that; in the seventh post's doc_chunks table we keep it in an embedding_model column, and the query side compares its own profile against it.

Plan a model change like a data migration. Add a new column or table for the new model, make ingestion write to both models, re-embed old documents in the background, build the new index, run the measurement set on both indexes, and if the result is good switch the query side to the new column. Drop the old column only after a while. Re-embedding is not cheap: a million chunks can take hours depending on the model's speed; account for it from the start.

Production Notes

Measure in your own language with your own questions. The biggest number in this post did not come from the leaderboard but from the same content in two languages: recall@1 of 16 versus 26. No general benchmark tells you that for your documents.

Report the size of the question set with the result. On 30 questions one question is 3.3 points; you cannot separate 512 from 768 dimensions or the prefix effect with this set. For a real decision build a set of hundreds of questions; we cover this in the twelfth post.

An embedding call costs differently at ingestion and at query time. At ingestion you send thousands of chunks in batches; at query time each request embeds one short text and the time adds directly to latency. With a local model this is a few milliseconds; with a cloud API (Application Programming Interface) it is more, network latency included.

Choose the embedding model independently of the chat model. The model that writes the answer can be gemma3:4b or a cloud model; retrieval quality is decided by the embedding model alone.

Common Pitfalls

First, choosing by the leaderboard average. The average includes tasks you will not use in RAG and, most likely, languages that are not yours. Filter by retrieval tasks and your language, then measure with your own set.

Second, relying on silent truncation. Ollama truncates long input by default and does not tell you. Measure chunk token counts at ingestion, or turn truncation off and catch the error.

Third, using different conventions for documents and queries. Giving the prefix at ingestion and forgetting it at query time, or changing the Matryoshka dimension on one side only. A dimension mismatch throws an exception; a prefix mismatch throws nothing.

Fourth, switching models and searching the old vectors. Between two models with the same dimensions this mistake never reaches a log. Store the model name with every vector and check it on the query side.

Fifth, ignoring the tokenizer for Turkish content. Turkish text takes more tokens than English with the same content; chunk size, max input and cost estimates must account for it.

A Small Experiment

★☆☆ Change the recall command to run with k=10. What is recall@10 on the Turkish set? If the right section comes below position 5, it is a candidate that reranking (the ninth post) can rescue.

★★☆ If you have a multilingual embedding model in Ollama on your machine (such as bge-m3), change the model name of the OllamaApiClient you pass to InMemoryIndex. If the model needs no prefix, leave the prefixes empty and measure the 30 questions in both languages. Does the gap between Turkish and English close?

★★★ Double the 30 questions: rewrite each one in different words, the way a customer would (for example "bedava", "give back", "turn into money"). The recall@5 difference between the original and rewritten questions shows how well the embedding model catches synonyms.

The embedding model sets the ceiling of retrieval, but the quality of the text we give it matters at least as much. In the next post we move to ingestion: extracting clean text from PDF (Portable Document Format), HTML (HyperText Markup Language) and Markdown, attaching metadata to every chunk, and updating the index incrementally as documents change.