Relevance Tuning: Field Boosts, function_score, Fuzzy Search, Highlighting and "Did You Mean?"

Version note: The examples in this post were run with Elasticsearch 9.5.6, .NET 10 and Elastic.Clients.Elasticsearch 9.5.3, in the lab we built in post 2, on the 20,035-book books index from the end of post 5.

Guess First

The /books/search endpoint we wrote for Bookstore, the online bookstore used as the shared example across this site's series, runs with the field weights we guessed in post 4: title ^3, Turkish and Turkish-character-free title ^2, everything else 1. The product team comes up with three suggestions:

• Triple the weight of author matches.
• Turn on fuzziness: AUTO on every field to tolerate typos.
• Bring highly rated, recently published books forward (function_score).

Which of these improves search, and which breaks it? More importantly: how would you know? "I tried a few searches and it looks fine" is not a measurement. This post first sets up the measurement, then decides every change with it.

Measure First: a Relevance Set and _rank_eval

To measure relevance you need two things: queries picked from real searches, and for each query a judgment of "which books, and how good a result each is". For Bookstore we keep this in the repo as search/relevance/judgments.json. It has 20 queries: Turkish and English searches, author searches, spellings without Turkish characters and one typo:

[
  { "query": "ince memed", "ratings": { "9781000000016": 3, "9781000000023": 2 } },
  { "query": "sabahattin ali", "ratings": { "9781000000030": 2, "9781000000047": 2, "9781000000054": 2 } },
  { "query": "the hobbit", "ratings": { "9781000000252": 3, "9781000000207": 2 } },
  { "query": "jane austen", "ratings": { "9781000000238": 2, "9781000000245": 2 } },
  { "query": "suç ve ceza", "ratings": { "9781000000177": 3, "9781000000306": 2 } },
  { "query": "madona", "ratings": { "9781000000030": 3 } },
  ...
]

3 means "the book being looked for", 2 means "very relevant"; every book not listed counts as 0. In a real system these queries come from the search logs: the 100 most frequent queries, zero-result queries and the ones that reach customer support as complaints. The judgments come from someone who knows the catalog (at Bookstore, the Product Owner).

Elasticsearch's _rank_eval API takes this set, runs each query and scores the result with a metric. We use nDCG@5 as the metric, a 0-to-1 measure of the top five results that gives more credit when the good results are on top; the API also supports precision, recall, MRR and expected reciprocal rank. The important decision is to give _rank_eval exactly the query the endpoint runs. That is why we moved the code that builds the query from the API into BookSearchQuery in Bookstore.Search, and gathered every setting that affects ranking into one SearchTuning record. The lab's rank-eval command:

RankEvalRequest request = new()
{
    Indices = SearchClientFactory.BooksIndex,
    Metric = new RankEvalMetric { Dcg = new RankEvalMetricDiscountedCumulativeGain { K = 5, Normalize = true } },
    Requests = judgments.Select(j => new RankEvalRequestItem
    {
        Id = j.Query,
        Request = new RankEvalQuery { Query = BookSearchQuery.Build(new BookSearchFilter(j.Query), tuning), Size = 5 },
        Ratings = j.Ratings.Select(r => new DocumentRating { Index = SearchClientFactory.BooksIndex, Id = r.Key, Rating = r.Value }).ToList()
    }).ToList()
};

RankEvalResponse response = await LabClient.Create().RankEvalAsync(request);

The first measurement, with post 6's query (SearchTuning.Baseline):

dotnet run --project src/Bookstore.SearchLab -- rank-eval
nDCG@5 0.905
  0.000  madona                     top:
  0.693  jane austen                top: 9781000000313(-), 9781000000238(2), 9781000000245(2)
  0.787  crime and punishment       top: 9781000000306(3), 9781000000245(-), 9781000000320(-)
  0.787  suç ve ceza                top: 9781000000177(3), 9781000000184(-), 9781000000023(-)
  0.834  the hobbit                 top: 9781000000207(2), 9781000000252(3), 9781000000320(-)

The command prints only the queries that do not get a full score, with their top three hits; (-) is a book with no judgment in the set. The output already says a lot. "madona" finds nothing. For "jane austen", Jane Eyre comes first: the word "jane" in the title, with ^3, beats the author's name. In "the hobbit" the Turkish Hobbit is ahead of the English edition, because the one-word title is shorter and "the" appears in hundreds of synthetic books, so it earns almost nothing. "suç ve ceza" and "crime and punishment" cannot find the edition of the same work in the other language; that is not something text matching solves, it needs a synonym rule or a "work" field linking the two editions. That is exactly the set's job: separating what tuning fixes from what the model has to fix.

Tuning by Measuring: Boosts, tie_breaker and Phrases

The rank-eval command takes every setting as a parameter; I changed one thing per run:

tie_breaker and most_fields looked sensible on paper and made things worse once measured: counting description matches in the score pushed books whose description mentions the same name ahead in author searches. The two improvements are each an extra should clause. The author query's boost became 3 and the Turkish-character-free author name (authors.name.folded) was added; the latter did not change the set's score, but it lets a user who types "yasar kemal" match on both words instead of only "kemal". The phrase clause adds points if the user's words appear in the title in the same order, and excludes no result:

if (tuning.PhraseBoost is float phraseBoost)
{
    // The user's words in the same order in the title: an extra should clause, never a requirement.
    clauses.Add(new MatchPhraseQuery { Field = "title.folded", Query = text, Slop = 1, Boost = phraseBoost });
}

The result went into the code as SearchTuning.Tuned, and the endpoint now uses it:

// Measured with rank-eval (post 07): nDCG@5 0.905 -> 0.979 on search/relevance/judgments.json.
public static SearchTuning Tuned { get; } = Baseline with
{
    AuthorFields = ["authors.name", "authors.name.folded"],
    AuthorBoost = 3,
    Fuzziness = "AUTO",
    PhraseBoost = 2,
    RatingAndRecency = true
};

Twenty queries is a small set; do not read 0.979 as "search is 98 percent right". Its value is that it shows immediately which query a change breaks. The RAG series post "Measuring RAG: Recall@k, MRR, nDCG, Faithfulness and Groundedness" covers these metrics and how they differ in detail.

Fuzzy Search: fuzziness, AUTO and prefix_length

fuzziness allows a certain number of character edits (insertion, deletion, substitution, swapping two adjacent letters) for each term in the query. AUTO picks that number from the term's length: exact match for 1 to 2 characters, one edit for 3 to 5, two for longer terms. prefix_length: 1 assumes the first letter is typed correctly; it sharply cuts the number of candidate terms and therefore the cost.

dotnet run --project src/Bookstore.SearchLab -- search madona
"madona": 0 hits
dotnet run --project src/Bookstore.SearchLab -- search madona --fuzzy AUTO
"madona": 1 hits
   25.70  Kürk Mantolu Madonna (tr, 4.80, 2019)

dotnet run --project src/Bookstore.SearchLab -- search dostoyevsky
"dostoyevsky": 0 hits
dotnet run --project src/Bookstore.SearchLab -- search dostoyevsky --fuzzy AUTO
"dostoyevsky": 2 hits
    8.64  Suç ve Ceza (tr, 4.80, 2022)
    8.55  Crime and Punishment (en, 4.70, 2019)

The author's name is "Dostoyevski" in the Turkish edition and "Dostoevsky" in the English one; "dostoyevsky" is one letter away from both and found both editions. Do not rely on fuzziness for Turkish character mistakes: "kurk" is one edit away from "kürk", but the folded subfield from post 4 already solves that for free; fuzzy should be kept for real typos.

The price of fuzzy is irrelevant matches. The five-letter "yaban" now finds two books:

dotnet run --project src/Bookstore.SearchLab -- search yaban --fuzzy AUTO
"yaban": 2 hits
   41.58  Yaban (tr, 4.10, 2016)
    7.36  Dokuzuncu Hariciye Koğuşu (tr, 4.30, 2015)

The second book's description contains "hastanede yatan"; "yatan" is one letter away from "yaban". The score gap is large, so it does no harm here, but turning fuzzy on for fields full of short words (category names, codes) fills the result list with unrelated words. One more restriction: the cross_fields type does not accept fuzziness; when I tried, it returned a 400 with Fuzziness not allowed for type [cross_fields].

function_score: Rating and Recency Signals

Among books with equal text matches, which should come first? Bookstore's index has no sales count, but it has the average rating (averageRating) and the publication date (publishedOn). function_score changes the text score with multipliers computed from these fields. My first attempt combined both functions as multipliers (score_mode: multiply) and computed recency with a five-year Gauss curve. The result dropped from 0.979 to 0.962: the 2023 Turkish Hobbit beat the 2020 English edition in the "the hobbit" search again. The recency factor was multiplying older books by almost zero. The final version is gentler:

// Text relevance stays the main signal; rating and recency only nudge it.
private static Query WithRatingAndRecency(Query textQuery) => new FunctionScoreQuery
{
    Query = textQuery,
    Functions =
    [
        new FunctionScore
        {
            FieldValueFactor = new FieldValueFactorScoreFunction
            {
                Field = "averageRating",
                Modifier = FieldValueFactorModifier.Log1p,
                Missing = 3
            }
        },
        new FunctionScore
        {
            Weight = 0.5,
            Gauss = new DateDecayFunction
            {
                Field = "publishedOn",
                Placement = new DecayPlacement<DateMath, Duration> { Origin = DateMath.Now, Scale = "3650d", Offset = "365d", Decay = 0.5 }
            }
        }
    ],
    ScoreMode = FunctionScoreMode.Sum,
    BoostMode = FunctionBoostMode.Multiply
};

The rating function is compressed with log1p: the factor for a book rated 2.5 and one rated 5 stays between 0.54 and 0.78. Recency gives full credit within the last year, then drops to half over ten years, and weight: 0.5 limits its contribution. The two functions are summed and multiplied with the text score. In this form the set gave 0.979 again, so it does not break queries looking for a specific book. Its benefit shows up in broad searches where text matches are equal:

dotnet run --project src/Bookstore.SearchLab -- search silent garden --author-boost 3 --fuzzy AUTO --phrase 2
"silent garden": 5382 hits
   33.44  The Silent Garden (en, 4.00, 2015)
   33.44  The Silent Garden (en, 3.90, 2024)
   33.44  The Silent Garden (en, 4.80, 2021)
   33.44  The Silent Garden (en, 3.10, 2025)
   33.44  The Silent Garden (en, 3.00, 2005)

dotnet run --project src/Bookstore.SearchLab -- search silent garden --tuned
"silent garden": 5382 hits
   42.19  The Silent Garden (en, 4.90, 2024)
   40.39  The Silent Garden (en, 4.80, 2021)
   39.55  The Silent Garden (en, 3.90, 2024)
   37.18  The Silent Garden (en, 3.10, 2025)
   34.10  The Silent Garden (en, 3.80, 2018)

Without signals, five books with the same title come back with equal scores, in index order. With signals the highly rated and recent ones move forward; the 2005 book rated 3.0 dropped out of the top five. If a field like a sales count is added, add it with the same field_value_factor pattern and a compression like log1p; multiplying by the raw sales count puts a bestseller at the top of every search.

rescore: Applying an Expensive Signal to the Top Results Only

Computing a phrase match for every document can be expensive on large indices. rescore rescores the top N results of the main query (window_size per shard) with a second query. This is how the same "the hobbit" problem was solved in Dev Tools:

GET /books/_search?filter_path=hits.hits._score,hits.hits._source.title,hits.hits._source.language
{
  "size": 3,
  "_source": ["title", "language"],
  "query": { "multi_match": { "query": "the hobbit", "fields": ["title^3", "title.tr^2", "title.folded^2", "title.en", "description", "description.tr", "description.folded", "description.en"] } },
  "rescore": {
    "window_size": 50,
    "query": {
      "rescore_query": { "match_phrase": { "title.folded": { "query": "the hobbit", "slop": 1 } } },
      "query_weight": 1,
      "rescore_query_weight": 2
    }
  }
}
{"hits":{"hits":[{"_score":60.25035,"_source":{"title":"The Hobbit","language":"en"}},{"_score":39.341927,"_source":{"title":"Hobbit","language":"tr"}},{"_score":7.362587,"_source":{"title":"Wuthering Heights","language":"en"}}]}}

# same query without rescore
{"hits":{"hits":[{"_score":39.341927,"_source":{"title":"Hobbit","language":"tr"}},{"_score":36.15021,"_source":{"title":"The Hobbit","language":"en"}},{"_score":7.362587,"_source":{"title":"Wuthering Heights","language":"en"}}]}}

We chose the should clause because a _rank_eval request only accepts a query, and on 20 thousand books the cost is negligible. On millions of documents, or when the rescore query is a script or an expensive phrase query, rescore is the right tool. Just remember that only results inside window_size change: a book outside the window cannot move up, however many points the rescore would give it.

Highlighting and Safe Rendering

We use highlighting to mark the matched words on the results page. Our query matches the title's folded and tr subfields, but the text shown to the user is title; matched_fields carries the subfield matches onto the main field's text (in 9.5 it worked with the default unified highlighter):

private static Highlight BuildHighlight() => new()
{
    Encoder = HighlighterEncoder.Html,
    PreTags = ["<mark>"],
    PostTags = ["</mark>"],
    Fields = new Dictionary<Field, HighlightField>
    {
        // The query hits title.folded / title.tr too; matched_fields maps those hits back onto the original title.
        ["title"] = new HighlightField { NumberOfFragments = 0, MatchedFields = Fields.FromStrings(["title", "title.folded", "title.tr"]) },
        ["description"] = new HighlightField { NumberOfFragments = 1, FragmentSize = 160, MatchedFields = Fields.FromStrings(["description", "description.folded", "description.tr"]) }
    }
};

curl "http://localhost:5080/books/search?q=raif%20berlin&pageSize=1"
{"total":2,"page":1,"pageSize":1,"items":[{"isbn":"9781000000030","title":"Kürk Mantolu Madonna","authors":["Sabahattin Ali"],"language":"tr","price":199.00,"score":19.795294,"titleHighlight":null,"descriptionHighlight":"<mark>Raif</mark> Efendi&#x27;nin <mark>Berlin&#x27;de</mark> bir sergide gördüğü portre ve Maria Puder ile yaşadığı aşkın romanı."}],"didYouMean":null}

The apostrophe arriving as ' is the work of encoder: html: since highlight text comes back as HTML, Elasticsearch escapes the special characters in the text and leaves only its own <mark> tags raw. Why does it matter? The highlight text is written into the DOM with innerHTML on the client, and document content is not always trustworthy: HTML pasted into a description field in the back office, or malicious input. I tried it on a separate books-xss index:

PUT /books-xss/_doc/9789999999984?refresh=wait_for
{ "isbn": "9789999999984", "title": "Draft", "description": "A new edition <img src=x onerror=alert(1)> with notes" }

"highlight": { "pre_tags": ["<mark>"], "post_tags": ["</mark>"], "fields": { "description": {} } }
{"hits":{"hits":[{"highlight":{"description":["A new <mark>edition</mark> <img src=x onerror=alert(1)> with notes"]}}]}}

"highlight": { "encoder": "html", "pre_tags": ["<mark>"], "post_tags": ["</mark>"], "fields": { "description": {} } }
{"hits":{"hits":[{"highlight":{"description":["A new <mark>edition</mark> &lt;img src=x onerror=alert(1)&gt; with notes"]}}]}}

Passing the response without the encoder to innerHTML is a stored XSS hole. With the encoder the text becomes harmless. On the Angular side the [innerHTML] binding sanitizes as well, but do not turn off the first line of defense because a second one exists; the client will not always be Angular. The Frontend Security series post "A DOM XSS Sink Inventory for Angular: ElementRef, Renderer2, Dynamic URLs, HTML-Producing Libraries and a Code Review Checklist" examines sinks like innerHTML one by one.

"Did You Mean?": the Term and Phrase Suggesters

Suggesters return, alongside the search results, corrections for the words in the query drawn from the terms in the index. The term suggester looks at each word separately; the phrase suggester produces one corrected sentence for the whole phrase:

GET /books/_search?filter_path=suggest
{ "size": 0, "suggest": { "text": "kurk mantlu madona", "title-terms": { "term": { "field": "title.folded", "suggest_mode": "always" } } } }
{"suggest":{"title-terms":[{"text":"kurk","offset":0,"length":4,"options":[]},{"text":"mantlu","offset":5,"length":6,"options":[{"text":"mantolu","score":0.8333333,"freq":2}]},{"text":"madona","offset":12,"length":6,"options":[{"text":"madonna","score":0.8333333,"freq":2}]}]}}

GET /books/_search?filter_path=suggest
{
  "size": 0,
  "suggest": {
    "text": "saatleri ayarlma enstitusu",
    "did-you-mean": {
      "phrase": {
        "field": "title.folded", "size": 1,
        "direct_generator": [ { "field": "title.folded", "suggest_mode": "always" } ],
        "collate": { "query": { "source": { "match": { "title.folded": { "query": "{{suggestion}}", "operator": "and" } } } }, "prune": true }
      }
    }
  }
}
{"suggest":{"did-you-mean":[{"text":"saatleri ayarlama enstitusu","offset":0,"length":26,"options":[{"text":"saatleri ayarlama enstitusu","score":4.9556263E-7,"collate_match":true}]}]}}

collate tries the suggested phrase with a real query; with prune: false, suggestions that find nothing are not returned at all. Note that the suggestion comes from the terms of the folded field, so it is lowercase and without Turkish characters ("enstitusu"). That is enough to show it as a link and run the same search when it is clicked, because the query also matches on the folded field. If the suggestion must keep Turkish characters, use the title field; then spellings like "kurk" will not be corrected.

In the endpoint we send the suggester in the same request as the search. Since fuzziness already finds most typos, zero results are now rare; we return the suggestion whenever it differs from what the user typed, even when there are results:

curl "http://localhost:5080/books/search?q=kurk%20madona&pageSize=1"
{"total":2,"page":1,"pageSize":1,"items":[{"isbn":"9781000000030","title":"Kürk Mantolu Madonna","authors":["Sabahattin Ali"],"language":"tr","price":199.00,"score":55.550564,"titleHighlight":"<mark>Kürk</mark> Mantolu <mark>Madonna</mark>","descriptionHighlight":null}],"didYouMean":"kurk madonna"}

The UI can say "showing results for kurk madonna" or "Did you mean: kurk madonna". You can also see that "madona" matched Madonna through fuzziness and that the highlight marks it.

Production Notes

Put the relevance set in CI. The rank-eval command can run in the pipeline and fail when the score drops below a threshold; every pull request that changes analyzers, synonyms or boosts gets measured. The set itself is a living file: zero-result searches and complaints should be added every month.

Watch the cost of fuzzy. Each fuzzy term expands into similar terms from the index (at most 50 by default); on long queries and large term dictionaries this shows up in query time. prefix_length and max_expansions are the first two values to tune.

Limit the cost of highlighting. Keep the returned text small with number_of_fragments and fragment_size; on long descriptions highlighting can be a noticeable part of the query time. Highlighting is needed on the result list only, not on the detail page.

Do not show the score to users, and do not make decisions with it. An Elasticsearch score is for ordering within one query; it cannot be compared across queries, and rules like "hide results scoring under 10" break as the index grows.

Common Pitfalls

Tuning boosts by guesswork. Two "sensible" changes in this post (tie_breaker and most_fields) lowered the score once measured. Tuning without measuring fixes one query, breaks another, and nobody notices.

Turning fuzzy on for every field. In short words one letter is a completely different word; fields like category, language and codes should not be searched fuzzily. Solve Turkish character problems with folding, not fuzziness.

Multiplying by a raw popularity signal. A signal added without compression (log1p, a low weight) overrides text matching; our first attempt showed this measurably.

Writing highlight text into innerHTML without the encoder. Every piece of text in the index, whatever its source, should be treated as untrusted data.

Writing the relevance set after the change. The set must be written before the change; otherwise you unknowingly pick queries where the change looks good.

A Small Experiment

★☆☆ Add the queries "kurk mantolu madonna sabahattin" and "middlemarch eliot" to judgments.json, give their judgments and run rank-eval --tuned. Did the score change?

★★☆ Add the rule "suç ve ceza, crime and punishment" to the bookstore-synonyms set (in Dev Tools, as in post 4). Measure again with rank-eval --tuned: were the cross-language queries fixed, did any other query break?

★★★ Add rescore to the endpoint (instead of the phrase clause) and compare the two approaches on the same query list. Since a _rank_eval request cannot take a rescore, how do you measure it? Hint: a small command that calls the endpoint for each query and computes nDCG itself.

In the next post we move to the search box itself: suggesting as the user types. We will set up search_as_you_type, edge_ngram and the completion suggester side by side on the same index, bring up Kürk Mantolu Madonna when the user types "kür", and measure what each approach costs in index size.