Document Modeling 1: Embed or Reference? From Access Patterns to Schema

Version note: The examples in this post were run with MongoDB 9.0.8 Community Server and mongosh 2.6. The scripts gave the same result on MongoDB 8.0.38; the only difference is the message text of the 16 MB error, noted below.

Guess First

The book page of Bookstore, the online bookstore used as the shared example across this site's series, shows a book together with its reviews. Someone who has heard "data read together is stored together" suggests embedding the reviews in the book as an array. The average review in the lab is 142 bytes. Let's write a script that adds reviews to Dune in groups of 10,000:

// 04 - unbounded array: embedding every review of a book in the book
db.lab_books.drop();
const template = db.reviews.findOne({ body: { $exists: true } }, { _id: 0, bookId: 0 });
const book = db.books.findOne({ _id: 26 });
db.lab_books.insertOne(book);
print("book alone           :", bsonsize(db.lab_books.findOne({ _id: 26 })), "bytes");

const batch = Array.from({ length: 10000 }, () => template);
let pushed = 0;
try {
    while (true) {
        db.lab_books.updateOne({ _id: 26 }, { $push: { reviews: { $each: batch } } });
        pushed += batch.length;
        if (pushed % 20000 === 0) {
            print(`${String(pushed).padStart(6)} reviews  :`, bsonsize(db.lab_books.findOne({ _id: 26 })), "bytes");
        }
    }
} catch (e) {
    print(`after ${pushed} reviews: ${e.name} code ${e.code}`);
    print(e.message);
}
db.lab_books.drop();

After how many reviews is the write rejected? And the real question: before the limit is reached, how much data is carried every time the book page opens?

book alone           : 339 bytes
 20000 reviews  : 2829243 bytes
 40000 reviews  : 5669243 bytes
 60000 reviews  : 8509243 bytes
 80000 reviews  : 11349243 bytes
100000 reviews  : 14189243 bytes
after 110000 reviews: MongoServerError code 10334
Plan executor error during update :: caused by :: Resulting document after update is larger than 16777216

110,000 reviews fitted; the $push that would have taken it to 120,000 was rejected. The limit is 16 MiB, that is 16,777,216 bytes; on MongoDB 8.0 the same error comes with the message BSONObj size: 17049243 (0x104269B) is invalid, which also shows the size the attempted write would have reached. But the error is not the real problem. With 20,000 reviews on a popular book, the book document is 2.8 MB; every request that reads this book, from the book page, the catalog list or the cart, moves that much data from disk to cache and from cache to the network. A projection shrinks what goes over the wire, but the document is still stored and read as a whole. Every new review also makes the server rewrite this large document. In this post we make decisions of this kind starting from the workload and from measurements, not from intuition.

Workload First: Bookstore's Access Patterns

In relational modeling you model the data itself first and think about access while choosing indexes. In document modeling the order is reversed: the first step of MongoDB's schema design process is to identify the application's workload, that is, to list which operation reads or writes which data and how often. For Bookstore the list looks roughly like this:

The list also needs numbers, because a label such as "one-to-many" says nothing about size. Let's look at the data in the lab:

// 04 - document sizes per collection ($bsonSize)
for (const name of ["books", "orders", "reviews", "customers"]) {
    const [s] = db.getCollection(name).aggregate([
        { $group: { _id: null, n: { $sum: 1 },
                    avg: { $avg: { $bsonSize: "$$ROOT" } }, max: { $max: { $bsonSize: "$$ROOT" } } } }
    ]).toArray();
    print(`${name.padEnd(9)} n=${String(s.n).padStart(3)}  avg=${Math.round(s.avg)} B  max=${s.max} B`);
}

// Reviews per book and items per order
const [r] = db.reviews.aggregate([
    { $group: { _id: "$bookId", n: { $sum: 1 } } },
    { $group: { _id: null, books: { $sum: 1 }, avg: { $avg: "$n" }, max: { $max: "$n" } } }
]).toArray();
print(`reviews per book: books=${r.books} avg=${r.avg.toFixed(1)} max=${r.max}`);
const [o] = db.orders.aggregate([
    { $group: { _id: null, avg: { $avg: { $size: "$items" } }, max: { $max: { $size: "$items" } } } }
]).toArray();
print(`items per order : avg=${o.avg.toFixed(2)} max=${o.max}`);

books     n= 77  avg=344 B  max=542 B
orders    n=379  avg=429 B  max=696 B
reviews   n=181  avg=142 B  max=172 B
customers n= 40  avg=159 B  max=185 B
reviews per book: books=58 avg=3.1 max=9
items per order : avg=1.79 max=4

An order carries 1.79 items on average and the largest order has 4; we can see that the whole order, with its items and payments, is under 700 bytes. A book, on the other hand, has at most 9 reviews today. That number is misleading: the number of items in an order has a natural upper bound (a cart), the number of reviews on a book does not. Embedding reviews because today's data is small means setting off towards the document in "Guess First". You build the model not from today's numbers but from how far they can grow.

Deciding by the Cardinality of the Relationship

A classification widely used in the MongoDB community comes from William Zola's 2014 series "6 Rules of Thumb for MongoDB Schema Design" on the MongoDB blog: saying one-to-many is not enough, you have to say how many "many" is. Applied to Bookstore, the decisions come out like this:

• One-to-few, a handful to a few dozen and bounded: embed. An order's items and payments, a book's author summary. Read together with the parent and meaningless without it.
• One-to-many, hundreds to thousands or with no clear upper bound: the child lives in its own collection and carries the parent's id. A book's reviews: each review has a bookId, the book has no list of reviews.
• One-to-squillions, events and records that can grow into millions: always a reference on the child side, often grouped as well. A book's page views; with the bucket pattern in post 5.
• Many-to-many, books and authors: the array of references goes on the bounded side. A book has a few authors, so authors lives in the book; an author's number of books can grow, so the author document has no list of books.
• One-to-one: embed almost always; the only good reason to keep it separate is to keep a large, rarely read part away from a frequently read document.

The consequence of the many-to-many line shows up in queries too: the author page searches books on authors.authorId for "this author's books". That field is inside an array, and an index on it later will be a multikey index (post 6).

// Many-to-many: the reference for an author's books lives in the book
printjson(db.books.find({ "authors.authorId": 18 }, { _id: 0, title: 1 }).sort({ _id: 1 }).toArray());
printjson(db.authors.findOne({ _id: 18 }));

[
  {
    title: 'Dune'
  },
  {
    title: 'Dune Messiah'
  }
]
{
  _id: 18,
  firstName: 'Frank',
  lastName: 'Herbert',
  birthYear: 1920
}

Bookstore's Decisions

The shape of the data we loaded in post 2 is the result of these rules. Let's see the whole model in one picture:

Order items and payments are inside the order: they are bounded, always read with the order, and must stay consistent with the order total. The title and unitPrice in an item are not a copy of the book but a record of the moment of sale. Copying a few fields of a source along with the reference is called an extended reference in MongoDB literature; customer: { customerId, name } in the order is of the same kind. The book carries a summary of its authors and category because the catalog list and the book page show them every time; the author's full details live in the authors collection and the category tree in categories. Reviews are separate because they are unbounded and read page by page. The customer.name in a review is a copy too, so that showing a review does not require reading the customer.

One decision is open to debate: stock. stock.onHand lives in the book; the benefit is that the conditional $inc from post 3 runs atomically on a single document and the book page shows stock status without an extra read. The cost is that the most frequently read document also carries a frequently written field: every sale rewrites the book, and layers that cache the book must invalidate it often (see the Caching series post "Cache Invalidation Stratejileri: TTL, Explicit Silme, Event ile Invalidation ve Versioned Key'ler"). With Bookstore's single warehouse and a reasonable sales rate the cost is small. Once several warehouses, reservations and stock movements arrive, stock should move to a separate inventory collection owned by Bookstore's Inventory module; a good example of the model changing along with the workload.

The Cost of Copies and Acceptable Staleness

Embedding and extended references create copies, and each copy needs answers to two questions: where is the source of truth, and what happens to the copy when the source changes? Let's see how many books must be updated when a category is renamed, and how an order item reacts to a price change on the book:

// 04 - duplicated fields: how many documents change when a category is renamed?
const r = db.books.updateMany(
    { "category.categoryId": 3 },
    { $set: { "category.name": "Sci-Fi" } }
);
print("category rename:", r.matchedCount, "books");
db.categories.updateOne({ _id: 3 }, { $set: { name: "Sci-Fi" } });

// The price in an order item is a snapshot: it does not follow the book
const oldPrice = db.books.findOne({ _id: 47 }).listPrice;
db.books.updateOne({ _id: 47 }, { $set: { listPrice: Decimal128("320.00") } });
print("book 47 listPrice:", db.books.findOne({ _id: 47 }).listPrice,
      " order 1 item unitPrice:", db.orders.findOne({ _id: 1 }).items[0].unitPrice);
db.books.updateOne({ _id: 47 }, { $set: { listPrice: oldPrice } });

category rename: 9 books
book 47 listPrice: Decimal128('320.00')  order 1 item unitPrice: Decimal128('290.90')

The category rename touched 9 books, and it is a separate operation from updating the category itself; for a short while the category page may show the new name while book cards show the old one. Category names change a few times a year and books are read hundreds of times a second, so the trade-off clearly favors reads. The order item, on the other hand, was not affected by the price change on the book, and must not be: order 1 was sold at 290.90 TRY. It helps to sort copies into three groups from this angle:

When deciding which field to copy, the rule is simple: small fields that are read often and change rarely are good candidates; copying a field that changes often means writing to many documents on every change. Listening to the source collection with a change stream for background updates is in post 10. The relational counterpart, the question of who keeps denormalized data consistent, is in the Data Modeling and Normalization series post "Denormalization: Ne Zaman, Nasıl ve Tutarlılığı Kim Korur?".

The Aggregate Boundary and Single-Document Atomicity

In MongoDB a write to a single document is atomic, no matter how many fields or subdocuments it changes. That is the strongest tool you have when building the model: if you put data that must stay consistent together in one document, you do not need a transaction. Let's record the payment for pending order 379; adding the payment and setting the status to Paid must happen together or not at all:

// 04 - single-document atomicity: payment and status change together
db.orders.updateOne({ _id: 379 }, { $set: { status: "Pending", payments: [] } });

for (let attempt = 1; attempt <= 2; attempt++) {
    const r = db.orders.updateOne(
        { _id: 379, status: "Pending" },
        {
            $set: { status: "Paid" },
            $push: { payments: { method: "card", amount: Decimal128("440.00"), paidAt: ISODate("2026-06-30T08:12:00Z") } }
        }
    );
    print(`attempt ${attempt}: matched ${r.matchedCount}, modified ${r.modifiedCount}`);
}
printjson(db.orders.findOne({ _id: 379 }, { _id: 0, status: 1, total: 1, payments: 1 }));

attempt 1: matched 1, modified 1
attempt 2: matched 0, modified 0
{
  status: 'Paid',
  total: Decimal128('440.00'),
  payments: [
    {
      method: 'card',
      amount: Decimal128('440.00'),
      paidAt: ISODate('2026-06-30T08:12:00.000Z')
    }
  ]
}

The first attempt added the payment and changed the status. When the same notification came from the payment provider a second time, status: "Pending" in the filter did not match; the payment was not added twice. Because status and payment live in the same document, an intermediate state such as "payment added but status unchanged" is never visible to any reader. The aggregate definition in the Domain-Driven Design series post "Aggregate Tasarımı: Consistency Boundary, Küçük Aggregate'ler ve Id ile Referans" maps directly onto this: an aggregate is the boundary that must stay consistent together, and a document is MongoDB's unit of atomic writes. In a good document model the two usually line up; the order is one aggregate and one document.

The work outside the boundary is clear too: an order decreases the stock of several books, and each book is a separate document. There are two ways here. The first is a multi-document transaction; it works on a replica set and we use it from .NET in post 10, but it is more expensive than a single-document write and must be kept short. The second is to treat stock as a separate step from the order: the order is written as Pending, each book is reserved with a conditional $inc, and if one fails the completed ones are undone and the order is cancelled. That is the single-database form of the approach known as a saga in microservices. Whichever you choose, embedding all the books into the order to "put everything in one document" is no solution; growing the aggregate hurts concurrency and document size.

In Production

Keep the access pattern list next to the model. Each embedding and copying decision rests on one of its lines; when the team asks for a new screen or report, that list is the first place to look. When the list changes the model may change too, and in MongoDB a model change is a migration job (post 5).

Write an upper bound for every growing array. If the bound does not come from a business rule (a cart, at most 3 recently viewed), move the array to a collection. Measure the largest documents in your data with $bsonSize regularly; getting close to 16 MB is a very late warning, and documents beyond a few hundred KB are usually a sign of a modeling mistake.

Code the update path for every copied field. Each row of the table above is either an updateMany, a background job or a deliberate "never". An update path nobody wrote means book cards showing an old name years later.

Common Pitfalls

Turning the relational schema into collections one to one. Let's put orders and items into separate collections as in SQL, join the order detail with $lookup and measure how much work the server does:

const pipeline = [
    { $match: { _id: 1 } },
    { $lookup: { from: "order_items_flat", localField: "_id", foreignField: "orderId", as: "items" } },
    { $lookup: { from: "customers", localField: "customerId", foreignField: "_id", as: "customer" } }
];

orders_flat: 379  order_items_flat: 680
without index on orderId
  order_items_flat: docsExamined 680, collectionScans 1
  customers: docsExamined 1, collectionScans 0
with index on orderId
  order_items_flat: docsExamined 2, collectionScans 0
  customers: docsExamined 1, collectionScans 0
embedded order: docsExamined 1

Without an index on orderId, all 680 items were scanned to find the two items of a single order. With the index this dropped to 2, but the order detail is still a read from three collections and an aggregation pipeline, while in the embedded model it is a single document. And because items and the order live in separate documents, consistency with the total now needs a transaction, and MongoDB has neither the foreign keys nor the JOIN optimizer of a relational database. If you run $lookup on every read, either the model or the choice of database is wrong; the right place for $lookup is reports and infrequent reads (post 7).

Embedding everything. The opposite mistake is common too: embedding all of a customer's orders in the customer, all reviews in the book. "Guess First" showed where that ends. Before embedding something, ask: is it read without the parent, is it updated on its own, what is the upper bound of its count? If the answer to any of these argues against embedding, keep it separate.

Modeling for today's data. In the lab a book has at most 9 reviews, and someone looking at that number might find embedding safe. A model is built for the possible size of the data, not today's; a campaign, a viral book or a bot attack can change the numbers overnight.

A Small Experiment

★☆☆ Add the authors and categories collections to 04-sizes.js. Work out how many extra reads it would mean for a catalog list of 20 books to read authors from authors every time instead of embedding the author summary.

★★☆ Write a script that updates an author's name (for example Frank Herbert to "Frank P. Herbert") both in authors and in the copies in books. Use arrayFilters and print matchedCount and modifiedCount. What changes when the script runs twice?

★★★ Sketch a design that moves stock into a separate collection called inventory: what should the document's _id be, how does the structure change when a warehouse is added, how does the book page show stock status? Write the flow that decreases the stock of two books when an order is placed, once with a transaction and once with conditional $inc and undo steps, and compare how the two behave on failure.

In this post we decided what to embed and what to reference. In the next post we add MongoDB's schema patterns on top of these decisions: a computed review summary in the book, a subset of the latest reviews, hourly view buckets, keeping print, ebook and audiobook editions in one collection, schema validation and how the schema evolves over time.