From Elasticsearch 8 to 9.x and OpenSearch 3: What's New, What Was Removed and Which to Choose

Version note: The examples in this post were run with Elasticsearch 9.5.6 and 8.19.25, OpenSearch 3.9.0, .NET 10, Elastic.Clients.Elasticsearch 9.5.3 and OpenSearch.Client 2.3.0. What is said about Elastic Cloud and Amazon OpenSearch Service comes from the Elastic and Amazon documentation.

Guess First

The search code we wrote for Bookstore, the online bookstore used as the shared example across this site's series, runs against Elasticsearch 9.5. OpenSearch is a fork of Elasticsearch 7.10.2; both run on Lucene, both use the same REST paths (/books/_search, _bulk, _analyze), and as we saw in post 1, many teams think of them as interchangeable. In the lab I started an OpenSearch 3.9.0 container, loaded the 36 books with _bulk, and pointed the syntax command from post 10 at it without changing any code:

$env:BOOKSTORE_ES_URL = "http://localhost:9203"
dotnet run --project src/Bookstore.SearchLab -- syntax

Can Elastic.Clients.Elasticsearch 9.5.3 talk to this server? If not, where does the error come from: the query itself, the connection, or somewhere else?

The Upgrade Path from 8.x to 9.x

First, Elasticsearch's own upgrade path. According to Elastic's upgrade preparation documentation the rule is clear: before moving to 9.x you must move to the latest 8.19.x release; 9.1 and later can only be reached from 8.19. Indices created in 7.x or earlier must be reindexed, deleted or archived as read-only before the upgrade; otherwise a 9.x node does not start. I walked this path in a small lab. I started a node with 8.19.25 (the latest 8.x image in Elastic's registry at the time of writing), created a simple books-v1 index with five books and tried two parameters that were removed in 9.0:

POST /books-v1/_search
{ "query": { "range": { "price": { "from": 100, "to": 200 } } } }

HTTP/1.1 200 OK
Warning: 299 Elasticsearch-8.19.25-... "Deprecated field [from] used, this field is unused and will be removed entirely"
Warning: 299 Elasticsearch-8.19.25-... "Deprecated field [to] used, this field is unused and will be removed entirely"

POST /books-v1/_search
{ "query": { "match": { "title": "dune" } }, "highlight": { "force_source": true, "fields": { "title": {} } } }

HTTP/1.1 200 OK
Warning: 299 Elasticsearch-8.19.25-... "[highlight][1:65] Deprecated field [force_source] used, this field is unused and will be removed entirely"

8.19 works and only returns a Warning header; most applications never log that header. The same requests on 9.5.6:

{"error":{"type":"parsing_exception","reason":"[range] query does not support [from]","line":1,"col":36},"status":400}

{"error":{"type":"x_content_parse_exception","reason":"[1:50] [highlight] unknown field [force_source]"},"status":400}

The warning becomes a 400 after the upgrade. That is why two things happen on 8.19: searching the application logs for Warning: 299 headers, and running the deprecation API. The API cannot see your code, but it scans the cluster's settings, templates and indices. To try it, I created a component template on 8.19 whose mapping sets _source.mode (the attribute has no effect in 9.0):

GET /_migration/deprecations
{
  "cluster_settings": [],
  "node_settings": [],
  "templates": {
    "legacy-source": [
      {
        "level": "critical",
        "message": "Configuring source mode in mappings is deprecated.",
        "details": "Configuring source mode in mappings is deprecated and will be removed in future versions. Use [index.mapping.source.mode] index setting instead.",
        "resolve_during_rolling_upgrade": false
      }
    ]
  },
  "index_settings": {},
  ...
}

Then I stopped the node and started the 9.5.6 image on the same data volume (a full restart on a single node; in a multi-node cluster the nodes are upgraded one at a time, which is called a rolling upgrade):

GET /
"number" : "9.5.6",
"minimum_wire_compatibility_version" : "8.19.0",
"minimum_index_compatibility_version" : "8.0.0"

GET /books-v1/_count
{"count":5, ...}

GET /_migration/deprecations?filter_path=index_settings
{
  "index_settings": {
    "books-v1": [
      {
        "level": "critical",
        "message": "Old index with a compatibility version < 9.0",
        "details": "This index has version: 8.19.9-8.19.23",
        "resolve_during_rolling_upgrade": false,
        "_meta": { "reindex_required": true }
      }
    ]
  }
}

There are three pieces of information here. 9.5.6 can read indices created in 8.0 and later (minimum_index_compatibility_version), and requires at least 8.19 for node-to-node communication. The index created in 8.x works, but the deprecation API already marks it "critical": it will have to be reindexed before the next major version. The alias-based switch from post 11 exists for exactly this; in Bookstore it means migrate books-vN. Finally I tried going back and opened the same volume with 8.19.25:

"log.level":"ERROR","message":"fatal exception while booting Elasticsearch", ...
"error.type":"java.lang.IllegalArgumentException","error.message":"indexCreatedVersionMajor is in the future: 10"

9.x writes index files in the Lucene 10 format, and 8.x cannot read them. An upgraded node cannot be rolled back; the only way back is restoring a snapshot taken before the upgrade. Do not start an upgrade without checking the last successful snapshot of the SLM policy from post 12.

What 9.0 Removed and Changed

The 9.0 list on Elastic's breaking changes page is long; here are the items that affect a search application most, grouped from Bookstore's point of view:

• Queries: the legacy parameters of the range query (from, to and similar) and force_source in highlighting are gone; date_histogram no longer accepts boolean values; the default field of random_score is now _seq_no, so random orderings may change. Bookstore's queries use gte/lt and were not affected.
• APIs: bulk action lines are parsed more strictly (we saw this in post 5); timeouts return 429 instead of 5xx (the retry decision in post 10); error JSON follows a consistent format; the old local parameter of the alias APIs was removed.
• Mappings: metadata field definitions no longer accept type, fields, copy_to and boost; the mode attribute of _source no longer has any effect (the deprecation above).
• Removed features: frozen indices can no longer be read (deal with them before the upgrade) and the tech-preview _knn_search API is gone; vector search uses the knn query. A semantic_text field now behaves like a normal text field.
• Security: TLSv1.1 was removed from the default protocols; very old clients may fail to connect.
• Analysis: the Snowball stemmers were upgraded. The Turkish stemmer from post 4 is a Snowball stemmer too; I analyzed the 236 distinct words in Bookstore's Turkish titles and descriptions on 8.19.25 and on 9.5.6 with the same filter chain (turkish lowercase plus the turkish stemmer), and the stems were identical. Run the same comparison on your own data; if the stems changed, the terms written into the old index no longer match the terms of new queries, and you need a reindex.

There are breaking changes between 9.1 and 9.5 as well, so do not assume minor versions are "safe". The one that could affect Bookstore is that in 9.1 ES|QL queries started returning partial results by default instead of failing: a report that ignores the is_partial flag in the response can mistake incomplete data for complete data (for a report like the esql command from post 10, pass allow_partial_results=false). In 9.4 sequence numbers were turned off for time series indices, so optimistic concurrency with if_seq_no does not work there; Bookstore's books index is not a time series index, so the approach from post 5 is not affected.

What Arrived Between 9.1 and 9.5

A few new features that we used while writing this series or that matter for a search application. In 9.1 (together with 8.19), the ES|QL LOOKUP JOIN command became generally available: it joins the result of a query with a small index created with index.mode: lookup. I put the Turkish names of the categories into such an index:

PUT /category-labels
{
  "settings": { "index.mode": "lookup" },
  "mappings": { "properties": { "category": { "type": "keyword" }, "label_tr": { "type": "keyword" } } }
}

POST /_query?format=txt
{ "query": "FROM books | WHERE language == \"tr\" | STATS books = COUNT(*) BY category = categories | LOOKUP JOIN category-labels ON category | SORT books DESC | LIMIT 5" }

     books     |     category     |   label_tr
---------------+------------------+---------------
21             |Fiction           |Roman
17             |Classics          |Klasikler
2              |Historical Fiction|null
2              |Science Fiction   |Bilim Kurgu
2              |Fantasy           |Fantastik

A category without a match in the lookup index comes back as null; this is a left join. In 9.2 LOOKUP JOIN learned to match on several fields and expressions. 9.5.0 brought separate management APIs for reindexing (GET /_reindex, GET /_reindex/<task_id>, cancel and rethrottle); the migrate command from post 11 tracks progress with them. Beyond these, most of what arrived during 9.x is about vector search, semantic search, inference and observability; the parts that matter for a search page (hybrid search, RRF and retrievers) are covered in the RAG series post "Hybrid Search and Reciprocal Rank Fusion: Combining Keyword and Vector Search". The most reliable way to follow the versions is the release notes and breaking changes pages; blog posts describe the highlights, not what breaks.

OpenSearch 3: a Shared Root, a Diverging Path

OpenSearch split from Elasticsearch 7.10.2 in 2021 after Elasticsearch's license change; it is licensed under Apache 2.0 and has been part of the OpenSearch Software Foundation under the Linux Foundation since 2024 (post 1). The current version is 3.9.0 (on Docker Hub since 2026-10-26). In April 2026 the OpenSearch Software Foundation announced a long-term support program: the first LTS release is 3.6, each LTS release is supported for at least 18 months, and every major version will have at least one LTS release. Elasticsearch has no official LTS name; its counterpart is that the last minor of the previous major keeps receiving patches: 8.19 is still updated while 9.x is out (the 8.19.25 image was released in November 2026).

The differences I saw in the lab. The default image ships with the security plugin and asks for an admin password on first start:

OpenSearch 2.12.0 onwards, the OpenSearch Security Plugin a change that requires an initial password for 'admin' user.
Please define an environment variable 'OPENSEARCH_INITIAL_ADMIN_PASSWORD' with a strong password string.
If a password is not provided, the setup will quit.

For the experiments I turned the security plugin off (DISABLE_SECURITY_PLUGIN=true, DISABLE_INSTALL_DEMO_CONFIG=true); do not do that in production. Then I tried to send Bookstore's mapping as is:

PUT /books        (search/books.index.json)
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Failed to build analyzers: [search_synonyms]"}], ...
 "reason":"[search_synonyms] synonym requires either `synonyms` or `synonyms_path` to be configured", ...}

The synonyms_set from post 4 and the synonyms API that manages it belong to Elasticsearch; in OpenSearch, synonyms are given as a list in the mapping or from a file on the server. The analysis-icu plugin is not in the default image, but it installed with bin/opensearch-plugin install analysis-icu (a restart is required). The analysis chain itself comes from the shared Lucene and is the same: on the same 236 words, the turkish analyzer produced the same 218 tokens on both engines. There is no ES|QL; OpenSearch's counterparts are PPL and the SQL plugin:

POST /_query
{"error":"Incorrect HTTP method for uri [/_query] and method [POST], allowed: [GET, DELETE, PUT, HEAD]","status":405}

POST /_plugins/_ppl
{ "query": "source=books | where price < 170 | stats count() as books, avg(price) as avg_price by language" }

"datarows": [ [1, 150.0, "en"], [4, 147.5, "tr"] ]

In short, the core query language, the Query DSL (match, bool, aggregations), is largely shared, but everything added on either side after 7.10 (the synonyms API, ES|QL, LOOKUP JOIN and retrievers on one side; PPL, its own security plugin, and the k-NN and neural search plugins on the other) is separate. Use the word "compatible" for the core left over from 7.10.

The .NET Side: Two Separate Clients

The answer to the guess:

Fluent:      POST /_search {...} -> -1 hit(s)
Initializer: POST /books/_search {...} -> -1 hit(s)

Invalid Elasticsearch response built from a unsuccessful (406) low level call on POST: /_search
 Exception: Request failed to execute. Call: Status code 406 from: POST /_search. ServerError: Type:  Reason: "Content-Type header [application/vnd.elasticsearch+json;compatible-with=9] is not supported"

The query is right and the connection works; the error is in a header. Elastic's 9.x client sends every request with the content type application/vnd.elasticsearch+json;compatible-with=9 (version information telling the server "I speak the 9.x API"), and OpenSearch does not recognise that type and returns a 406. The same application's index-exists check, a HEAD request, went through without a problem; requests without a body do not carry that header. So the Elastic client can look as if it "partly works" against OpenSearch, which makes the error show up later. OpenSearch's own client is OpenSearch.Client; it was forked from NEST, Elasticsearch's .NET client up to 7.x, at version 7.10, and its latest version on NuGet today is 2.3.0. I wrote the same search with it in a small project:

// Post 13: the same search against OpenSearch with OpenSearch.Client (a fork of NEST 7.10).
string url = Environment.GetEnvironmentVariable("BOOKSTORE_OS_URL") ?? "http://localhost:9200";
ConnectionSettings settings = new ConnectionSettings(new Uri(url)).DefaultIndex("books");
OpenSearchClient client = new(settings);

ISearchResponse<BookHit> response = await client.SearchAsync<BookHit>(s => s
    .Size(5)
    .Query(q => q.Bool(b => b
        .Must(m => m.Match(mt => mt.Field(f => f.Title).Query("dune")))
        .Filter(f => f.Term(t => t.Field(x => x.Language).Value("tr"))))));

// NEST naming: IsValid instead of IsValidResponse, ApiCall instead of ApiCallDetails.
Console.WriteLine($"{response.ApiCall.HttpMethod} {response.ApiCall.Uri.PathAndQuery} -> HTTP {response.ApiCall.HttpStatusCode}, IsValid {response.IsValid}, {response.Total} hit(s)");

$env:BOOKSTORE_OS_URL = "http://localhost:9203"
dotnet run --project src/Bookstore.OpenSearchLab
POST /books/_search?typed_keys=true -> HTTP 200, IsValid True, 1 hit(s)
  9781000000191 Dune (tr) 420.0

The lambda syntax looks a lot like the fluent syntax from post 10, but the types, namespaces and names differ (OpenSearch.Client, ISearchResponse<T>, IsValid). Unlike the Elastic client in post 10, this client's descriptor search used DefaultIndex and went to /books/_search. The serializer is different too: the Elastic client uses System.Text.Json, OpenSearch.Client its own serializer inherited from NEST. Moving between the two engines is not a package swap in the code but a rewrite of the search layer. Having Bookstore's queries gathered in BookSearchRequestBuilder (post 10) makes that job smaller but does not remove it.

Managed Options and Version Lag

The two most common options in the cloud are Elastic Cloud (Elastic's own service, with the latest Elasticsearch releases and the paid subscription features) and Amazon OpenSearch Service. According to the AWS documentation, Amazon OpenSearch Service supports OpenSearch 3.5, 3.3 and 3.1, a few versions behind today's 3.9.0; it runs legacy Elasticsearch only up to 7.10, the last open source version before the license change. For anyone who needs Elasticsearch 8.x or 9.x, AWS's managed service is not an option. On the other hand, Amazon OpenSearch Service includes document level security and field level security, which require a paid subscription in Elasticsearch (post 12). This section is based on the official documentation.

Which One to Choose?

Some practical rules. If the organization's cloud is AWS and the team wants a managed service, OpenSearch is the result of a decision that has already been made; in that case read this series alongside the OpenSearch documentation and replace the sections that use ES|QL and the synonyms API with their counterparts. If you need Elastic's latest features, ES|QL and support from Elastic itself, choose Elasticsearch. If the license policy allows nothing but Apache 2.0, choose OpenSearch; Elasticsearch's AGPLv3 option (post 1) is enough for some organizations and not for others, so ask your legal team. If the team already runs one of them, the cost of moving to the other (client, queries, operational tooling) is usually larger than the difference in features.

In Production

Plan an upgrade like a project: move to the latest 8.19.x, clear the deprecation API and the Warning: 299 headers in the application logs, take a snapshot, and try it on a test cluster first. An upgraded node cannot be rolled back.

Upgrade the client and the server together. A 9.x client works with a 9.x server; do not use a 9.x client against an 8.x server (post 10). The client should stay on 8.x until the servers are upgraded and be updated afterwards.

Read the minor versions too. Items such as the ES|QL partial results change in 9.1 and the time series change in 9.4 arrived in minor releases. Before every upgrade, check the breaking changes section of every minor in between, and run rank-eval from post 7 and the tests from post 10 against the new version.

Moving from Elasticsearch to OpenSearch (or the other way round) is not an upgrade but a migration. Plan as if snapshots cannot move between the two engines; reload the data from the source (SQL Server for Bookstore) into the new engine.

Common Pitfalls

Ignoring deprecation warnings. The Warning header on 8.19 is a 400 error on 9.x.

Trying to go from 8.x to 9.x without 8.19. 9.1 and later can only be reached from 8.19.

Not taking a snapshot before the upgrade. 8.x cannot open an index written by 9.x (indexCreatedVersionMajor is in the future: 10).

Saying "OpenSearch is compatible with Elasticsearch". The core is shared; Elastic's 9.x client gets a 406, and there is no synonyms API or ES|QL.

Trusting that some of the Elastic client's requests work against OpenSearch. Requests without a body go through, searches do not; you find out in production, not in the first test.

Not checking the managed service's version. Amazon OpenSearch Service is a few versions behind OpenSearch's latest release.

A Small Experiment

★☆☆ Start a node with 8.19.25 and send a query using random_score to books-v1 ("random_score": { "seed": 42 }, without a field). Does it return a Warning? Run the same query on 9.5.6 and compare the order of the results.

★★☆ Adapt search/books.index.json to OpenSearch: a synonyms list instead of synonyms_set, and install the ICU plugin in the Dockerfile. Load the 36 books and compare the "kurk mantolu" and "novel" searches from post 4 on both engines.

★★★ Add an IBookSearch interface to Bookstore.Search and write two implementations, one for Elasticsearch and one for OpenSearch. Which parts stay shared (query parameters, result model) and which diverge (facets, did-you-mean, autocomplete)? Is the cost of this abstraction worth the chance of switching search engines?

This is the end of the series. Starting from where LIKE falls short, we built inverted indexes, mappings, Turkish analyzers, SQL Server sync, the Query DSL, relevance measurement, autocomplete, facets, the .NET client, an alias-based lifecycle and production operations for Bookstore. The next step is where matching words is not enough: results that are similar in meaning. The RAG series covers vector search on the same Elasticsearch and how to combine it with the BM25 search from this series.