Collection Endpoints: The Contract for Pagination, Filtering, Sorting, Field Selection and Search

Version note: The code in this post was built and run with C# 14 and .NET 10 (ASP.NET Core 10.0.15); the HTTP examples are real responses from this application, and the OpenAPI 3.2 document was linted with Redocly CLI 2.78.0 and Spectral 6.17.1.

Guess First

The book list of Bookstore, the online bookstore used as the shared example across this site's series, has been paginated since the first post and returns a nextPageToken. The company that built the integration for Nevin Hanım's school looked at this token out of curiosity. What it saw on the API as it stood at the end of the previous post:

$ curl -s "http://127.0.0.1:52318/books?pageSize=2" | jq -r .nextPageToken
b2Zmc2V0PTI

$ echo 'b2Zmc2V0PTI=' | base64 -d
offset=2

$ echo -n 'offset=20' | base64
b2Zmc2V0PTIw

$ curl -s "http://127.0.0.1:52318/books?pageSize=2&pageToken=b2Zmc2V0PTIw" | jq -c '.items[] | {isbn, title}'
{"isbn":"9781451673319","title":"Fahrenheit 451"}
{"isbn":"9780441172719","title":"Dune"}

$ curl -s "http://127.0.0.1:52318/books?catgory=dystopia&pageSize=2" | jq -c '.items[] | {title, category}'
{"title":"Emma","category":"classics"}
{"title":"Pride and Prejudice","category":"classics"}

The company now builds tokens itself and jumps to whatever page it wants. In the last request category is misspelled; the API returned the whole catalog without an error, and for a week the integration listed classic novels to the school as dystopias. Two questions: what happens to this integration the day we change the token format, and whose fault is it? And who pays for silently ignoring a misspelled parameter?

What Is in a Collection Contract?

The contract of a list endpoint is not only the schema of the items it returns. How many items a page holds, how the end of the list is signalled, what the order is, how ties are broken, the names and types of the filters, what happens to an unknown parameter, whether a total count comes back; these are all behaviours the client observes and codes against. Hyrum's Law from the first post shows its teeth here more than anywhere: every behaviour you did not document is also a behaviour someone depends on.

The guidelines largely agree on this. Google's AIP-158 defines the page_size, page_token and next_page_token fields: if the page size is missing or 0 the default is used, a value above the maximum is reduced to the maximum, and a negative value is an error; the server may return fewer items than asked, and the only thing that says the collection has ended is an empty next_page_token. Tokens must be opaque and not decodable by the client; the AIP states plainly that plain Base64 is not enough for that. If other parameters change between pages, an error must be returned. The Azure guidelines express the same idea with nextLink, maxpagesize and the rule that nextLink is left out entirely on the last page, and recommend not returning a total count. Zalando prefers cursor-based pagination to offsets and advises avoiding total counts.

Opaque Tokens: Nothing to Read, Nothing to Change

Bookstore's first token was exactly what the AIP warns about: the Base64 of offset=2. A client decoded it and made it part of the contract. The new token carries three things: which query it belongs to (a digest of the filters and sort), the sort key of the last item returned and an expiry time. It is all encrypted and signed; for the client it is a meaningless string:

$ curl -i "http://127.0.0.1:52318/books?category=classics&inStock=true&sort=-price&pageSize=3&fields=isbn,title,price"
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sat, 14 Nov 2026 23:00:58 GMT
Server: Kestrel
Content-Language: en
Transfer-Encoding: chunked
Link: </books?category=classics&inStock=true&sort=-price&pageSize=3&fields=isbn,title,price&pageToken=CfDJ8AAAAAAAAAAAAAAAAAAAAAA2vE5Z5s_kODkG5xK34K8XilhiSsq5suBMdqTVH0uVoJIPj-VFfU5Sb-YPH6cdkwFu-Kge2fUOax1FgLMINmMUZYpv262wHawQ_HYm4z9rYRk_im_ldgX1rPr_I6lq8V2DKZEYBtERZVmyGPbsuUfVbuWrU163v-9VdJRU8UpgLmHfMtNOPiCn99i-D-7lu8s>; rel="next"

{"items":[{"isbn":"9781400079988","title":"War and Peace","price":{"amount":549.00,"currency":"TRY"}},{"isbn":"9780060934347","title":"Don Quixote","price":{"amount":459.00,"currency":"TRY"}},{"isbn":"9780140449136","title":"The Brothers Karamazov","price":{"amount":419.00,"currency":"TRY"}}],"nextPageToken":"CfDJ8AAAAAAAAAAAAAAAAAAAAAA2vE5Z5s_kODkG5xK34K8XilhiSsq5suBMdqTVH0uVoJIPj-VFfU5Sb-YPH6cdkwFu-Kge2fUOax1FgLMINmMUZYpv262wHawQ_HYm4z9rYRk_im_ldgX1rPr_I6lq8V2DKZEYBtERZVmyGPbsuUfVbuWrU163v-9VdJRU8UpgLmHfMtNOPiCn99i-D-7lu8s"}

$ TOKEN=$(curl -s "http://127.0.0.1:52318/books?category=classics&inStock=true&sort=-price&pageSize=3" | jq -r .nextPageToken)

$ curl -s "http://127.0.0.1:52318/books?category=classics&inStock=true&sort=-price&pageSize=3&fields=isbn,title,price&pageToken=$TOKEN" | jq -c '.items[]'
{"isbn":"9780143058144","title":"Crime and Punishment","price":{"amount":359.00,"currency":"TRY"}}
{"isbn":"9780142437247","title":"Moby-Dick","price":{"amount":339.00,"currency":"TRY"}}
{"isbn":"9780141439549","title":"Middlemarch","price":{"amount":319.00,"currency":"TRY"}}

$ curl -s "http://127.0.0.1:52318/books?category=dystopia&inStock=true&sort=-price&pageSize=3&pageToken=$TOKEN" | jq .
{
  "type": "https://api.example.com/problems/invalid-page-token",
  "title": "Invalid page token",
  "status": 400,
  "detail": "The page token is invalid, has expired or belongs to a different query; start again without pageToken.",
  "instance": "/books",
  "traceId": "00-1f0cde1007f77ad7d89ec532c5088fb3-c27416d52d8d5d3b-00"
}

$ curl -s "http://127.0.0.1:52318/books?pageToken=b2Zmc2V0PTIw" | jq -c '{type, status}'
{"type":"https://api.example.com/problems/invalid-page-token","status":400}

The address of the next page is in two places: nextPageToken in the body and the Link header defined by RFC 8288, Web Linking. The Angular client uses the body; the header is for generic tools and scripts that walk page by page without reading the body. The fields parameter did not change for the second page, but it could have; it changes only the view of a page, not its content. When category changes the token is rejected, because continuing from the middle of a different query makes no sense. The old token the company built by hand now gets a 400 too.

The token carries not an offset but the sort key of the last item. That is keyset pagination, where the next page is defined as "everything after the last key seen": if books are added to or removed from the catalog while a user walks the pages, a list with offsets repeats or skips items, whereas a keyset continues after the last key seen. How this is implemented in the database, which index it needs and the measurements are in the "Pagination Strategies in Depth: Offset, Keyset and Opaque Cursor Tokens" post of the Solution Patterns series; here we design only the contract. What the contract says: what is inside the token is none of the client's business, it is valid for three days, and it can only be used with the same filters and sort.

The answer to the first question in Guess First is uncomfortable: when the format changed the company's integration broke, and technically that is their fault, because the documentation said the token was opaque. But breaking a consumer is always our problem. The real mistake was handing out a decodable token on day one. How to announce such a change and give consumers time to move, together with the Deprecation and Sunset headers, is the subject of post 06.

Page Size and Total Count

The page size rules follow AIP-158: missing or 0 means 10, anything above 50 is reduced to 50, a negative value is an error. The reason for reducing to the maximum rather than failing is that clients will not break when we change the maximum later. The catalog has 26 books, so asking for 500 gives the whole catalog on one page and no token:

$ curl -s "http://127.0.0.1:52318/books?pageSize=-1" | jq -c '{status, errors}'
{"status":400,"errors":{"pageSize":["out-of-range"]}}

$ curl -s "http://127.0.0.1:52318/books?pageSize=0" | jq '.items | length'
10

$ curl -s "http://127.0.0.1:52318/books?pageSize=500" | jq '{count: (.items | length), nextPageToken}'
{
  "count": 26,
  "nextPageToken": null
}

The null in the last output is how jq shows a missing member; in the JSON there is no nextPageToken at all. We do not return a total count. A total means a separate COUNT query on every page, and on large tables that query can cost more than the page itself. It is also a number that changes while the pages are being walked. The Angular screen works with a "Show more" button; if a "1,284 books in total" label is really needed, a separate endpoint is considered, or, as AIP-158 allows, a value documented as an estimate.

Filtering: Where Do Plain Parameters Stop?

Bookstore's filters are plain query parameters named after the fields in the representation: category, author, inStock, minPrice/maxPrice for ranges and createdAfter/createdBefore for dates. An unknown parameter is no longer silently ignored; the Azure guidelines also ask for an error on an unsupported parameter. That is the answer to the second question in Guess First: the cost of the misspelling was paid by the school, which never saw the error.

$ curl -s "http://127.0.0.1:52318/books?catgory=dystopia" | jq -c '{type, status, errors}'
{"type":"https://api.example.com/problems/validation-failed","status":400,"errors":{"catgory":["unknown-parameter"]}}

$ curl -s -H "Authorization: Bearer linus" "http://127.0.0.1:52318/institutions/lakeside-school/orders?status=Confirmed,Shipped&createdAfter=2026-10-01T00:00:00Z&pageSize=1" | jq -c '{ids: [.items[] | {id, unitId, status, createdAt}], more: (.nextPageToken != null)}'
{"ids":[{"id":"01a1a612-9c80-7e46-8590-67e08b9d2434","unitId":"east-campus","status":"Confirmed","createdAt":"2026-11-04T10:20:00Z"}],"more":true}

$ curl -s -H "Authorization: Bearer linus" "http://127.0.0.1:52318/institutions/lakeside-school/orders?status=shipped" | jq -c '{status, errors}'
{"status":400,"errors":{"status":["invalid-value"]}}

In the institutional order history status takes several values separated by commas, and the values are case-sensitive as in the representation; shipped is not a value. The limit of plain parameters shows up when "A or B" and nested conditions arrive. At that point there are three ways to go:

AIP-160 defines a filter field with its own grammar: AND, OR, NOT, comparison operators and : for presence checks. It asks for an error on an unknown field or a malformed expression. Bookstore stays with plain parameters for now; the schools need a few fixed filters. If an expression language is needed one day, a filter parameter is added next to the existing ones; that is an additive change. Opening the query fully to the client is GraphQL's territory and is covered in the gRPC and GraphQL series.

Sorting, Field Selection and Search

sort is a comma-separated list of fields; a leading - means descending (-price,title). Only documented fields are allowed, and a unique field is appended to every sort: the ISBN for books, the id for orders. Without that tie-breaker the order of two books with the same price can change on every query, and the keyset loses items between pages. The default sort is now written down too: by title. The list in the earlier posts came back in insertion order without that being documented; a client relying on it would be broken by this change. An undocumented order is still an order.

fields, a sparse fieldset, returns only the requested members of each item; in the first request above the items held only isbn, title and price. It shrinks the payload for the mobile app's list screen, but it has a cost: different shapes of the same URL take separate cache entries, the item type in a client generated from the contract becomes an "every member optional" type, and every new field must be added to the selectable list on purpose. Internal fields cannot be selected:

$ curl -s "http://127.0.0.1:52318/books?sort=stock" | jq -c '{status, errors}'
{"status":400,"errors":{"sort":["unknown-field"]}}

$ curl -s "http://127.0.0.1:52318/books?fields=isbn,supplier" | jq -c '{status, errors}'
{"status":400,"errors":{"fields":["unknown-field"]}}

$ curl -s "http://127.0.0.1:52318/books?q=tolkien&fields=title,price" | jq -c '.items[]'
{"title":"The Hobbit","price":{"amount":349.00,"currency":"TRY"}}
{"title":"The Lord of the Rings","price":{"amount":899.00,"currency":"TRY"}}

$ curl -i -X QUERY -H "Content-Type: application/json" -d '{"isbns":["9780441172719","9780261102217","9780000000002"]}' http://127.0.0.1:52318/books
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Sat, 14 Nov 2026 22:56:29 GMT
Server: Kestrel
Content-Language: en
Transfer-Encoding: chunked

{"items":[{"isbn":"9780441172719","title":"Dune","author":"Frank Herbert","category":"science-fiction","price":{"amount":489.00,"currency":"TRY"},"inStock":true},{"isbn":"9780261102217","title":"The Hobbit","author":"J. R. R. Tolkien","category":"fantasy","price":{"amount":349.00,"currency":"TRY"},"inStock":true}]}

q is a single parameter for free-text search in title and author. A real search engine, with relevance and typo tolerance, needs a system such as Elasticsearch behind it; on the contract side what matters is that what q searches and the order of the results are written down. The last request is a different need: asking which books from a school's 500-ISBN list are in the catalog. That list does not fit into a URL; a POST is not safe, so it is not cached and resending it is not considered safe. RFC 10008, published in June 2026, defined QUERY for exactly this gap: the criteria travel in the body, but the method is safe and idempotent like GET. OpenAPI 3.2 knows the query operation as well. Of the tools we tried, Redocly CLI validates the query operation in 3.2; Spectral 6.17.1 does not include that operation in its rules.

In the Contract and in the Linter

The pagination rules are written in the shared parameter definitions, and every list endpoint references them:

    PageSize:
      name: pageSize
      in: query
      description: >-
        Maximum number of items in the page. Absent or 0 means 10; values above 50
        are reduced to 50; negative values return 400. A page may hold fewer items.
      schema:
        type: integer
        minimum: 0
        default: 10
    PageToken:
      name: pageToken
      in: query
      description: >-
        Opaque nextPageToken from the previous page. Valid for 3 days and only with the
        same filters and sort; pageSize and fields may change. Otherwise 400 invalid-page-token.
      schema:
        type: string

The "every list paginated from its first version" rule (BS-30) looks in Spectral for pageSize and pageToken on every GET whose operationId starts with list. The first time the rule ran it caught our own catalog: GET /problems is a bounded list of 16 rows. Instead of paginating it we wrote the exception down as an ADR (ADR-0003) and turned the rule off for that one operation in .spectral.yaml. The rule's output for a change that deliberately removed the orders' pageToken parameter:

$ npx spectral lint contract/bookstore.openapi.yaml

/home/user/bookstore/contract/bookstore.openapi.yaml
 316:18  error  bookstore-paginated-lists  #/paths/~1orders/get/parameters is a list without pageSize and pageToken  paths./orders.get.parameters

✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints)

In ASP.NET Core

The token is produced with ASP.NET Core's Data Protection API: CreateProtector("Bookstore.PageToken.v1").ToTimeLimitedDataProtector() encrypts and signs the payload, Protect(payload, TimeSpan.FromDays(3)) adds the expiry, and Unprotect throws on a tampered or expired token, which we turn into invalid-page-token. The sample uses UseEphemeralDataProtectionProvider(), so the keys live in memory and tokens become invalid when the application restarts; a deployment with several instances needs a persistent key ring shared by all of them. The Link header is built from the current query with QueryHelpers.AddQueryString, and QUERY is mapped with MapMethods("", ["QUERY"], LookupBooks); Kestrel accepts the method without extra configuration. Applying filters, sorting and pagination in the database with EF Core is covered in the "Pagination, Filtering and Sorting" post of the ASP.NET Core Web API series.

Common Pitfalls

Unlimited page size. An endpoint that accepts pageSize=100000 can strain the database and memory with a single request. The maximum must be written in the contract.

A cursor the client parses. A Base64 offset, a JSON document or a date becomes the contract of the first curious consumer. Encrypt the token, or at least sign it, and write in the documentation that it is opaque.

Offsets on changing data. If a book is added to the first page while the user moves to the second, the last book of the first page shows up again at the top of the second. A keyset with a unique tie-breaker prevents that.

A total count on every page. Adding a second query to every page request for a number shown in one corner of the screen can be the most expensive part of the list.

Ignoring unknown parameters, and an unlimited filter language. The first turns a misspelled filter into an "everything" result; the second gives the client the power to run a query on every unindexed field.

A Small Experiment

★☆☆ Write a small shell loop that walks all pages of the book list by following the address in the Link header. Under which condition does the loop stop?

★★☆ After fetching a page, delete a book on that page as Ayşe, then continue with nextPageToken. Did an item disappear or repeat on the next page? What would have happened with the first post's offset token?

★★★ Design support for an AIP-160 style expression in the institutional order history with a filter parameter: write into the contract which fields and operators you support, which problem type you return for an unsupported field, and what happens when it is combined with the existing status and createdAfter parameters.

This post completes the first block of the series: why a contract is an architectural decision, resources and URLs, methods and status codes, the error contract and collections. The second block starts with how a contract changes over time: in the next post we look at breaking change rules, versioning strategies and the Deprecation and Sunset headers.