Blog
Mert Özen's blog posts on software development, backend, frontend, and DevOps.
Bulk, Batch and Asynchronous APIs: Bulk Operations, Partial Success, Webhooks or Polling, and the Rate Limit Contract
The contract of a bulk API spells out the size limit, which errors reject the whole request and which reject a single line, how partial success is reported, how the outcome is delivered, and the quota. At Bookstore we design the schools' bulk order as a 201 with a line-by-line report, a 422 with allOrNothing and a long-running operation with Prefer: respond-async, add the webhook to the contract, and see the RateLimit header draft and 429 with real responses.
Getting Data into Elasticsearch: the Bulk API, Refresh, Versioning and Syncing with SQL Server
You write data to Elasticsearch with the Bulk API, not one document at a time; but bulk is not atomic, a written document is not searchable until a refresh, and dual writes from SQL Server silently lose data. With real output: IndexAsync versus bulk, BulkAll and refresh_interval, per-item bulk failures, if_seq_no and external versions, a BackgroundService polling with rowversion, and MIN_ACTIVE_ROWVERSION so open transactions are not missed.