Autocomplete and Search-as-You-Type: edge_ngram, search_as_you_type and the Completion Suggester

Version note: The examples in this post were run with Elasticsearch 9.5.6, .NET 10, ASP.NET Core 10 Output Cache and Elastic.Clients.Elasticsearch 9.5.3, in the lab we built in post 2, on the 20,035-book books index.

Guess First

We are adding suggestions to the search box of Bookstore, the online bookstore used as the shared example across this site's series: book titles will appear below the box as the user types. The user has typed "kür". Will the /books/search endpoint we built in post 7 find anything? Even if it does, is calling that endpoint on every keystroke the right thing to do? And one more: should a user who types "mantolu", a word from the middle of the title, be offered Kürk Mantolu Madonna, and which of the three approaches Elasticsearch offers for this can do it?

The answer to the first question is no: "kür" is not a word, it is the start of a word. The terms in the search fields are whole words ("kurk", "kürk"); "kür" matches none of them, and fuzziness cannot reach "kurk" by changing one of three letters either. For autocomplete, the beginnings of words have to be in the index as well.

Three Approaches and Extending the Mapping in Place

Elasticsearch has three ways to "suggest as you type". We will add all three as new subfields of the title and compare them:

• edge_ngram: our own analyzer; it indexes the beginnings of each word ("k", "ku", "kur", "kurk") as separate terms. The most flexible, and the query side is a normal match.
• search_as_you_type: a ready-made field type; it creates hidden subfields for word pairs, triples and their prefixes itself, and is queried with a multi_match of type bool_prefix.
• The completion suggester: a separate data structure for suggestions rather than search; it works on an FST held in memory. Very fast, but it only matches from the start and its filtering is limited.

The new subfields need a new analyzer. As we saw in post 3, an existing field in the mapping cannot be changed, but new fields and new multi-fields can be added. Adding an analyzer, however, requires the index to be closed. In the lab, in Dev Tools, as the elastic user:

POST /books/_close

PUT /books/_settings
{
  "analysis": {
    "filter": {
      "autocomplete_edge": { "type": "edge_ngram", "min_gram": 1, "max_gram": 15 }
    },
    "analyzer": {
      "tr_autocomplete": {
        "type": "custom",
        "tokenizer": "standard",
        "filter": ["apostrophe", "tr_lowercase", "asciifolding", "autocomplete_edge"]
      }
    }
  }
}

POST /books/_open

PUT /books/_mapping
{
  "properties": {
    "title": {
      "type": "text",
      "fields": {
        "autocomplete": { "type": "text", "analyzer": "tr_autocomplete", "search_analyzer": "tr_folded" },
        "sayt": { "type": "search_as_you_type", "analyzer": "tr_folded" },
        "complete": { "type": "completion", "analyzer": "tr_folded" }
      }
    }
  }
}

POST /books/_update_by_query?refresh=true&filter_path=took,total,updated,version_conflicts,failures
{"took":1156,"total":20035,"updated":20035,"version_conflicts":0,"failures":[]}

The new subfields are only populated for documents written from now on; for the existing 20,035 books we run _update_by_query without a body. It reindexes every document from its _source, and the new fields get built. There are two caveats. A closed index cannot be searched; a few seconds in the lab are fine, but in production that is an outage, and the way there is the new index and alias from post 11. The second is that _update_by_query uses internal versioning: Dune's _version went from 22121 to 22122. In post 5 we set the version from SQL Server's rowversion; the next real change will get a larger rowversion than this, so the lab is fine, but running _update_by_query often can push the version in the index ahead of real changes, and such a change is rejected with a 409 as "already up to date" and silently lost. On an index that uses external versions, the safe way to extend the mapping is still a full load into a new index. search/books.index.json was updated with the same fields; an index created from scratch does not need these steps.

edge_ngram: Our Own Autocomplete Analyzer

tr_autocomplete adds an edge_ngram filter to the end of the tr_folded chain from post 4:

GET /books/_analyze?filter_path=tokens.token
{ "analyzer": "tr_autocomplete", "text": "Kürk Mantolu" }
{"tokens":[{"token":"k"},{"token":"ku"},{"token":"kur"},{"token":"kurk"},{"token":"m"},{"token":"ma"},{"token":"man"},{"token":"mant"},{"token":"manto"},{"token":"mantol"},{"token":"mantolu"}]}

Order matters. edge_ngram comes after lowercasing and folding; had we done it the other way round, uppercase pieces with Turkish characters like "Kü" would enter the index, and a user typing "kur" would not find anything. On the search side, search_analyzer is tr_folded, which produces no ngrams: the "kür" the user typed becomes a single term ("kur") and matches the "kur" piece in the index. Without a search analyzer, "kür" would also be split into "k", "ku", "kur", and every book starting with "k" would be a hit.

GET /books/_search?filter_path=took,hits.total.value,hits.hits._score,hits.hits._source.title
{ "size": 3, "_source": ["title"], "query": { "match": { "title.autocomplete": { "query": "kür", "operator": "and" } } } }
{"took":9,"hits":{"total":{"value":1},"hits":[{"_score":14.40775,"_source":{"title":"Kürk Mantolu Madonna"}}]}}

{ "match": { "title.autocomplete": { "query": "kurk ma", "operator": "and" } } }
{"took":4,"hits":{"total":{"value":1},"hits":[{"_score":18.88143,"_source":{"title":"Kürk Mantolu Madonna"}}]}}

{ "match": { "title.autocomplete": { "query": "madonna", "operator": "and" } } }
{"took":2,"hits":{"total":{"value":1},"hits":[{"_score":14.40775,"_source":{"title":"Kürk Mantolu Madonna"}}]}}

{ "match": { "title.autocomplete": { "query": "the sil", "operator": "and" } } }
{"took":3,"hits":{"total":{"value":694},"hits":[{"_score":5.7208166,"_source":{"title":"The Silent Mountain"}},{"_score":5.7208166,"_source":{"title":"The Silent River"}},{"_score":5.7208166,"_source":{"title":"The Silent Harbour"}}]}}

"kür" and the Turkish-character-free "kurk ma" find the same book; "madonna" catches it from a word in the middle of the title. operator: and is required here: every typed word (even a half-typed last one) has to be the start of a word in the title. The price is index size: every prefix up to 15 characters is a separate term. Lowering max_gram reduces the size but has a trap: the index has no prefixes longer than that, while the search side does not cut the word. With max_gram: 5, a user typing "mantolu" finds nothing, because the longest piece in the index is "manto". Either pick max_gram to fit the long words in the catalog, or add a truncate filter of the same length to the search analyzer.

search_as_you_type

The search_as_you_type field offers the same idea ready-made. One line was added to the mapping; behind it, Elasticsearch created the title.sayt._2gram, title.sayt._3gram (word pairs and triples) and title.sayt._index_prefix (their prefixes) fields. The query type is bool_prefix: it searches the last word as a prefix and the earlier ones as whole terms.

GET /books/_search?filter_path=took,hits.total.value,hits.hits._score,hits.hits._source.title
{
  "size": 3, "_source": ["title"],
  "query": {
    "multi_match": {
      "query": "kurk ma", "type": "bool_prefix",
      "fields": ["title.sayt", "title.sayt._2gram", "title.sayt._3gram"]
    }
  }
}
{"took":4,"hits":{"total":{"value":1658},"hits":[{"_score":12.862545,"_source":{"title":"Kürk Mantolu Madonna"}},{"_score":1.0,"_source":{"title":"A Library in Map 67"}},{"_score":1.0,"_source":{"title":"Mountain of the Distant Map"}}]}}

"operator": "and"
{"took":3,"hits":{"total":{"value":1},"hits":[{"_score":12.862545,"_source":{"title":"Kürk Mantolu Madonna"}}]}}

With the default or, the prefix "ma" also brought back 1,657 other books containing "Map"; even if the order is right, the total and everything after the first hit are meaningless. With operator: and the result is the same as with edge_ngram. The word-pair fields give a higher score to titles where the words appear in the typed order; that is where the gap of 12.86 versus 1.0 for "kurk ma" comes from. The advantage of this type is that it needs no tuning; the disadvantages are the index size we will see shortly and an analysis chain that is less controllable than edge_ngram.

The completion Suggester

The completion field works differently: it is queried with suggest, not query, and the prefix is looked up on an FST in memory:

GET /books/_search?filter_path=took,suggest.*.options.text
{ "_source": false, "suggest": { "titles": { "prefix": "kür", "completion": { "field": "title.complete", "size": 3, "skip_duplicates": true } } } }
{"took":2,"suggest":{"titles":[{"options":[{"text":"Kürk Mantolu Madonna"}]}]}}

"prefix": "the sil"
{"took":3,"suggest":{"titles":[{"options":[{"text":"The Silent Bridge"},{"text":"The Silent City"},{"text":"The Silent Clock"}]}]}}

"prefix": "mantolu"
{"took":1}

"prefix": "kurk mantlu", "completion": { "field": "title.complete", "size": 3, "fuzzy": { "fuzziness": 1 } }
{"suggest":{"titles":[{"options":[{"text":"Kürk Mantolu Madonna"}]}]}}

There is no suggestion for "mantolu" (the option list was empty, so filter_path dropped all of it): completion only matches from the start of the title. If you want suggestions from words in the middle, you have to give each book several input values ("Kürk Mantolu Madonna", "Mantolu Madonna", "Madonna"), which complicates the code that writes data to the index. skip_duplicates collapses synthetic books with the same title, and fuzzy tolerates typos.

Category filtering in completion is done with contexts. I tried it first on a separate books-ac-test index, because adding contexts to a field has a price:

PUT /books-ac-test
{
  "mappings": {
    "properties": {
      "categories": { "type": "keyword" },
      "title": { "type": "text", "fields": { "complete": { "type": "completion",
        "contexts": [ { "name": "category", "type": "category", "path": "categories" } ] } } }
    }
  }
}

GET /books-ac-test/_search
{ "suggest": { "titles": { "prefix": "the sil", "completion": { "field": "title.complete" } } } }
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Missing mandatory contexts in context query"}], ...},"status":400}

{ "suggest": { "titles": { "prefix": "the sil", "completion": { "field": "title.complete", "contexts": { "category": ["Mystery"] } } } } }
{"suggest":{"titles":[{"options":[{"text":"The Silent Clock"},{"text":"The Silent Garden"}]}]}}

On a completion field with contexts, every query has to provide a context; there is no "all categories" option, so you either list them all or add a second field without contexts. Had I tried this on the live index, there would have been no way to take the contexts back out of the mapping. We left Bookstore's main field without contexts.

Index Size: Measuring with _disk_usage

The _disk_usage API shows the cost of the three approaches field by field. Before measuring I merged the index down to one segment (_forcemerge?max_num_segments=1); for comparison I had taken the same measurement before the fields were added:

POST /books/_disk_usage?run_expensive_tasks=true&filter_path=books.store_size,books.all_fields.total,books.fields.title*.total

# before adding the fields
{"books":{"store_size":"5mb","all_fields":{"total":"4.9mb"},"fields":{"title":{"total":"149kb"},"title.en":{"total":"123.6kb"},"title.folded":{"total":"148.9kb"},"title.keyword":{"total":"238kb"},"title.sort":{"total":"296.8kb"},"title.tr":{"total":"148.9kb"}}}}

# after _update_by_query and _forcemerge
{"books":{"store_size":"13.7mb","all_fields":{"total":"13.4mb"},"fields":{"title":{"total":"290.5kb"},"title.autocomplete":{"total":"604.4kb"},"title.complete":{"total":"681.7kb"},"title.en":{"total":"240.4kb"},"title.folded":{"total":"290.5kb"},"title.keyword":{"total":"347.4kb"},"title.sayt":{"total":"168.2kb"},"title.sayt._2gram":{"total":"212.1kb"},"title.sayt._3gram":{"total":"192.3kb"},"title.sayt._index_prefix":{"total":"2.1mb"},"title.sort":{"total":"406.2kb"},"title.tr":{"total":"290.3kb"}}}}

The first thing that stands out is that even title, which we never touched, went from 149 KB to 290 KB. The cause is _update_by_query: the old copy of every book was marked as deleted, but Elasticsearch keeps deleted copies for a while (a 12-hour retention lease by default) for replica recovery; _stats showed 40,076 deleted Lucene documents (books plus nested authors). So the right comparison is not the total size, but the new fields against the old title.folded's 148.9 KB: the edge_ngram field is about 4 times that (604 KB), completion about 4.6 times (682 KB), and search_as_you_type with its hidden fields 2.6 MB, 18 times. The biggest item is the prefix field. With short titles these numbers are small; apply the same ratio to the descriptions of millions of books and the index size multiplies. That is why autocomplete is applied only to short fields (title, author name).

The ASP.NET Core Endpoint: /books/suggest

For Bookstore we chose edge_ngram: it suggests from words in the middle as well, it is a normal query so category filters and everything from post 6 work, and its size is acceptable. We kept the endpoint separate from /books/search because its needs differ: no highlighting, no total, just a few titles.

app.MapGet("/books/suggest", async (string? q, string? category, ElasticsearchClient client, ILoggerFactory loggerFactory, CancellationToken cancellationToken) =>
{
    // One or two letters match thousands of titles; start suggesting from the second character.
    if (string.IsNullOrWhiteSpace(q) || q.Trim().Length < 2)
    {
        return Results.Ok(Array.Empty<BookSuggestion>());
    }

    List<Query> filters = [];
    if (!string.IsNullOrWhiteSpace(category))
    {
        filters.Add(new TermQuery { Field = "categories", Value = category });
    }

    // The token is cancelled when the browser aborts the request (a newer keystroke); the Elasticsearch call stops too.
    SearchResponse<Book> response = await client.SearchAsync<Book>(s => s
        .Size(MaxSuggestions)
        .Source(new SourceConfig(new SourceFilter { Includes = Fields.FromStrings(["isbn", "title"]) }))
        .TrackTotalHits(new TrackHits(false))
        // The catalog has several books with the same title; show each title once.
        .Collapse(new FieldCollapse { Field = "title.keyword" })
        .Query(new BoolQuery
        {
            Must = [new MatchQuery { Field = "title.autocomplete", Query = q, Operator = Operator.And }],
            Should = [new MatchPhrasePrefixQuery { Field = "title.folded", Query = q }],
            Filter = filters
        }), cancellationToken);
    ...
    return Results.Ok(response.Hits.Select(h => new BookSuggestion(h.Source!.Isbn, h.Source.Title)).ToList());
})
.CacheOutput(CachePolicy);

Every line has a reason. The _source filter shrinks what Elasticsearch sends: the response to the same eight-result "the sil" query is 3,273 bytes without the filter and 1,155 bytes with _source: ["isbn", "title"]. For an endpoint called on every keystroke, that difference is directly latency and bandwidth. TrackTotalHits(false) turns off the total count; a suggestion list needs no count. collapse reduces synthetic books with the same title to one row; in the first try "The Silent River" appeared twice in the list. The should with match_phrase_prefix gives extra points to a user who types the words in the title's order. On errors the endpoint returns an empty list: an error message in the suggestion box gives the user nothing.

Output Cache keeps the response for the same q and category for 30 seconds; popular prefixes ("ha", "the", "kü") are typed the same way by thousands of users:

// Autocomplete answers are the same for everyone: cache them for a short time (post 08).
builder.Services.AddOutputCache(options => options.AddPolicy(BookSuggestEndpoints.CachePolicy, policy => policy
    .Expire(TimeSpan.FromSeconds(30))
    .SetVaryByQuery("q", "category")));

WebApplication app = builder.Build();

app.UseOutputCache();

curl "http://localhost:5080/books/suggest?q=k%C3%BCr"
[{"isbn":"9781000000030","title":"Kürk Mantolu Madonna"}]

curl -s -w '\n%{size_download} bytes\n' "http://localhost:5080/books/suggest?q=the%20sil"
[{"isbn":"9782000091639","title":"The Silent Mountain"},{"isbn":"9782000091882","title":"The Silent River"},{"isbn":"9782000092346","title":"The Silent Harbour"},{"isbn":"9782000092759","title":"The Silent Promise"},{"isbn":"9782000092797","title":"The Silent Library"},{"isbn":"9782000095279","title":"The Silent Kingdom"},{"isbn":"9782000095750","title":"The Silent Window"},{"isbn":"9782000099024","title":"The Silent Forest"}]
430 bytes

curl "http://localhost:5080/books/suggest?q=the%20sil&category=Mystery"
[{"isbn":"9782000091882","title":"The Silent River"},{"isbn":"9782000098614","title":"The Silent Kingdom"},{"isbn":"9782000101598","title":"The Silent Clock"},{"isbn":"9782000102397","title":"The Silent Window"},{"isbn":"9782000104957","title":"The Silent Road"},{"isbn":"9782000105893","title":"The Silent Library"},{"isbn":"9782000115243","title":"The Silent House"},{"isbn":"9782000122616","title":"The Silent Voyage"}]

curl "http://localhost:5080/books/suggest?q=madonna"
[{"isbn":"9781000000030","title":"Kürk Mantolu Madonna"}]

curl "http://localhost:5080/books/suggest?q=k"
[]

I sent the "the sil" request two more times; the API log, which writes one line per request to Elasticsearch, has only five lines: "kür", "kurk ma", "the sil", "the sil" with a category, and "madonna". The two repeated requests came from the cache. The Caching series post "ASP.NET Core Output Cache in Depth: Policies, Vary-By, Tag Eviction, Resource Locking and the Redis Store" covers policies and using a Redis store across several servers.

On the client there are two rules. First, debounce: do not send a request on every keystroke, send it when the user pauses for 200 to 300 ms. Second, cancellation: when a new request goes out, cancel the previous one, so a late old response does not overwrite the new one. In Angular that is the debounceTime and switchMap pair; the RxJS series posts "Time-Based Operators and Schedulers: debounceTime, throttleTime, auditTime, timeout and Polling" and "Higher-Order Mapping: Choosing Between switchMap, mergeMap, concatMap and exhaustMap" cover both in detail. When switchMap cancels the old HTTP request, ASP.NET Core fires RequestAborted; the CancellationToken in the endpoint carries it to the Elasticsearch client, and the unneeded query is cut off there too.

Production Notes

Autocomplete will be your most frequently called endpoint. Every user produces 3 to 5 suggestion requests per search; plan capacity for suggestion traffic rather than search, and track took as a separate metric.

Keep suggestions consistent with search results. A book that shows up as a suggestion should also be in the search results when the user presses Enter. In our setup both endpoints use the same index and the same analysis chain (tr_folded); a different suggestion source (a separate table, a separate index) can lose that consistency.

Keep the cache short. A change in a book's price or stock does not change the suggestion, but a newly added or withdrawn book does; 30 seconds is in the same range as the sync delay from post 5.

Do not extend the mapping in place in production. The close/open steps in this post close the index, and _update_by_query can clash with external versions; the new index and alias approach from post 11 avoids both.

Common Pitfalls

Running the full search query on every keystroke. A query with highlighting, suggesters, a total count and function_score is needlessly expensive for suggestions; write a separate, small query for them.

Applying ngrams on the search side too. Without a search_analyzer, what the user types is also split into prefixes, and every title starting with "k" matches.

Putting edge_ngram before lowercasing and folding. Uppercase pieces with Turkish characters enter the index; a user typing "kur" cannot find "Kürk".

Applying ngrams to long fields. On fields like the description the size multiplies, and it is useless for autocomplete anyway; apply it only to short fields like the title and author name.

Leaving bool_prefix with or. The prefix of the last word brings in hundreds of unrelated documents; the total and the lower part of the ranking become meaningless.

Adding contexts to a completion field that you cannot take back later. Contexts become mandatory; try it on a separate index first.

A Small Experiment

★☆☆ Add author suggestions to /books/suggest: define an autocomplete subfield for authors.name too (remember it is nested) and return Sabahattin Ali's books when the user types "sabah".

★★☆ Build a separate index named books-ac with tr_autocomplete's max_gram lowered from 15 to 8, copy the same data with _reindex, and compare the size of title.autocomplete with _disk_usage. What does a user who types "saatleri ayarlama" see?

★★★ Write a second endpoint based on the completion suggester (/books/suggest/fast) and run a small load test calling both endpoints with 1,000 random prefixes. How large is the difference in took and end-to-end time? Is losing suggestions from middle words worth it?

In the next post we move to the left side of the search page: category, price range, language and rating facets. We will compute the counts in the same request with aggregations, use post_filter so a selected filter does not distort the counts in its own facet, and see on an index with several shards why the counts from a terms aggregation can be approximate.