Process Memory in Practice: mmap, Copy-on-Write, Shared Memory and Container Memory Limits

Version note: The code in this post was built and run with .NET 10; tests use xUnit v3 4.0.3. Measurements were taken in mcr.microsoft.com/dotnet/sdk:10.0 containers based on Ubuntu 24.04, on a Linux 6.x kernel with cgroup v2, using Docker's --memory and --cpus limits. GC behavior comes from the dotnet/runtime source code and the GC configuration documentation on Microsoft Learn, and Windows details from Microsoft Learn.

Guess First

Bookstore, the online bookstore used as the shared example across this site's series, collects orders in memory for a report, and the API runs in a container with a 256 MB limit. In the previous post a process that asked for 1 GB of native memory was killed by the kernel without seeing a single error. What happens if the same process asks for 1 GB of managed memory instead, as 1 MB byte[] chunks: is it silently OOMKilled again, or do you get an exception you can catch? And if so, at which MB?

Second question: two processes map the same 64 MB file, and each shows 98 MB RSS in top. How much RAM do these two processes really hold: 196 MB, or less? In this post we read a process's memory from the inside.

How the Heap Comes from the OS: mmap, mprotect and VirtualAlloc

A process asks for memory through system calls. On Linux the basic tool is mmap: it opens a new range in the address space, and that range is either backed by a file or anonymous, meaning memory with no file that starts out zeroed. mprotect changes a range's permissions and munmap gives the range back. C's malloc serves small requests from its own heap and calls mmap directly for large ones. On Windows the counterparts are VirtualAlloc and VirtualFree.

The .NET GC uses these tools with the reserve and commit split we saw in the previous post. In the Linux layer of dotnet/runtime the three steps are plain to see: reserve is an mmap with no permissions, PROT_NONE; commit makes part of the range readable and writable with mprotect; decommit maps the same part again with no permissions using MAP_FIXED. According to the comment in the source, that last choice tells the kernel clearly that the pages are no longer needed and guarantees that re-committed pages come back zeroed. On Windows the same steps are the MEM_RESERVE and MEM_COMMIT flags of VirtualAlloc and the MEM_DECOMMIT flag of VirtualFree.

You can watch this with strace. In a container with a 512 MB limit I had the lab app take 64 MB of managed memory and filtered the output down to the lines that fall inside the range the GC reserved:

strace -f -qq -e trace=mmap,mprotect,munmap -o mm.txt dotnet Bookstore.OsLab.dll fill 64

252   mmap(NULL, 2013270016, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeebfff000
252   mprotect(0xfffeee000000, 4096, PROT_READ|PROT_WRITE) = 0
252   mprotect(0xfffeee001000, 65536, PROT_READ|PROT_WRITE) = 0
252   mprotect(0xfffeeec01000, 1048576, PROT_READ|PROT_WRITE) = 0
252   mprotect(0xfffeeed01000, 1048576, PROT_READ|PROT_WRITE) = 0
...
252   mmap(0xfffeeec21000, 32374784, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xfffeeec21000

The first line is a reserve of about 1.9 GB; we come back to why it has this size in the container section. The mprotect calls after it are commits: first small pieces at the start of regions, then 1 MB for every 1 MB array. There were 78 calls and 64 MB of commit in this range; the final mmap with MAP_FIXED is the decommit in which the GC gives 31 MB back to the OS after collecting. strace ran on arm64 in this environment; addresses depend on the architecture, and on x64 Linux the same lines show addresses starting with 7f while the system calls are identical. How regions are managed, the GC's internal algorithm, belongs to the upcoming Compilers and Programming Languages series.

Memory-Mapped Files: Putting a File into the Address Space

The file-backed use of mmap opens up a different world. Instead of copying the file into a buffer with read, the kernel places the file's pages directly into the address space; when the process touches that address the page comes from the page cache, read from disk if needed. That was one of the sources of the major faults in the previous post. It is also what the runtime does when it loads assemblies: in the second post we saw Bookstore.OsLab.dll sitting in /proc/<pid>/maps as a file.

In .NET the API for this is MemoryMappedFile. The lab app has a command that maps a 64 MB CSV file, standing in for Bookstore's catalog export, read-only and reads each of its pages once:

public static IEnumerable<string> Share(string path, int megabytes, TimeSpan hold)
{
    CreateCatalogExport(path, megabytes);
    long size = (long)megabytes << 20;
    yield return $"before map     {RollupLine()}";

    using MemoryMappedFile file = MemoryMappedFile.CreateFromFile(path, FileMode.Open, null, 0, MemoryMappedFileAccess.Read);
    using MemoryMappedViewAccessor view = file.CreateViewAccessor(0, size, MemoryMappedFileAccess.Read);
    ReadEveryPage(view, size);
    yield return $"pages read     {RollupLine()}";

    Thread.Sleep(hold);
    yield return $"after {hold.TotalSeconds,2:F0} s     {RollupLine()}";
}

RollupLine reads four values from /proc/self/smaps_rollup. According to proc(5), this file sums the smaps values of all the process's mappings into a single record. RSS is the process's pages in RAM; Shared_Clean is unmodified pages that another process has mapped as well; Private_Dirty is written pages that belong to this process alone. PSS, the proportional set size, divides every shared page by the number of processes sharing it.

Shared Pages: Why RSS Does Not Add Up

Now the second question. I had two processes map the same file; A waits 10 seconds and B starts 4 seconds after A:

(dotnet Bookstore.OsLab.dll map-share /tmp/catalog-export.csv 64 10 | sed "s/^/A /") &
sleep 4
dotnet Bookstore.OsLab.dll map-share /tmp/catalog-export.csv 64 10 | sed "s/^/B /"

A before map     Rss=  30 MB  Pss=  12 MB  Shared_Clean=  23 MB  Private_Dirty=   6 MB
A pages read     Rss=  97 MB  Pss=  77 MB  Shared_Clean=  26 MB  Private_Dirty=   7 MB
A after 10 s     Rss=  98 MB  Pss=  43 MB  Shared_Clean=  90 MB  Private_Dirty=   7 MB
B before map     Rss=  30 MB  Pss=  11 MB  Shared_Clean=  23 MB  Private_Dirty=   6 MB
B pages read     Rss=  97 MB  Pss=  44 MB  Shared_Clean=  90 MB  Private_Dirty=   7 MB
B after 10 s     Rss=  98 MB  Pss=  78 MB  Shared_Clean=  26 MB  Private_Dirty=   7 MB

While A was alone, the file's 64 MB went entirely into its PSS. When B read the same file the kernel did not allocate new pages; B's page table pointed at the same physical pages. The Shared_Clean of both A and B rose to 90 MB and their PSS fell to 43 and 44 MB. Their RSS values are 98 and 97 MB, 195 MB in total; but what is held in RAM is around 43 + 44 = 87 MB. The answer to the question: less. A's last line was taken while B was running and B's last line after A had exited; once A was gone the pages belonged to B alone and its PSS went back up to 78 MB.

Even the "before map" lines show 23 MB of Shared_Clean: libc and the runtime's native libraries are already shared with the other processes in the container. Shared libraries are shared through the same mechanism, file-backed mappings. Ten .NET processes on the same machine use the code pages of libcoreclr.so as a single copy. That is why adding up processes' RSS values and comparing the total with the machine's memory gives the wrong result; PSS is what adds up.

Shared memory itself is this mechanism: two processes map the same file, or the same named memory object, in shared mode, and each sees what the other writes. On Windows the counterparts are CreateFileMapping and MapViewOfFile, and named mappings without a file exist too. The details of the MemoryMappedFile API and how it compares with streams belong to the .NET I/O series; its place in inter-process communication is in the eighth post.

Copy-on-Write: fork and Private Mappings

What happens when a shared page is written? If the mapping is private, the p in maps, the kernel makes a copy of the page at the moment of the write, switches the process's page table to the copy, and the write goes to the copy. This is called copy-on-write. It is what makes fork cheap: according to Linux's fork(2) man page, the child does not copy the parent's pages; both share the same pages as copy-on-write, and most of fork's cost is duplicating the page tables. In the second post we saw that .NET uses vfork, and that the reason is to avoid marking the parent's writable pages copy-on-write.

We cannot call fork from .NET, but we can see the same mechanism with MemoryMappedFileAccess.CopyOnWrite. This access mode maps the file privately:

using MemoryMappedFile file = MemoryMappedFile.CreateFromFile(path, FileMode.Open, null, 0, MemoryMappedFileAccess.CopyOnWrite);
using MemoryMappedViewAccessor view = file.CreateViewAccessor(0, size, MemoryMappedFileAccess.CopyOnWrite);

ReadEveryPage(view, size);
yield return $"read only      {RollupLine()}";

// Change the first byte of every page: the kernel makes a private copy of the page
int page = Environment.SystemPageSize;
for (long offset = 0; offset < size; offset += page)
{
    view.Write(offset, (byte)'#');
}
yield return $"after writes   {RollupLine()}";
yield return $"in view: '{(char)view.ReadByte(0)}'  on disk: '{(char)File.ReadAllBytes(path)[0]}'";

read only      Rss=  96 MB  Pss=  77 MB  Shared_Clean=  25 MB  Private_Dirty=   6 MB
after writes   Rss=  98 MB  Pss=  78 MB  Shared_Clean=  26 MB  Private_Dirty=  71 MB
in view: '#'  on disk: '9'

RSS barely moved, but its contents changed: the file's 64 MB in the page cache were replaced by 64 MB of written pages private to the process, and Private_Dirty went from 6 to 71 MB. The file on disk stayed the same; the change lives only in this process's copy. The writable global data of shared libraries works the same way: the rw-p part from the second post is shared until a process writes to it.

ASLR and W^X: Why Do Addresses Change Every Time?

In the second post we said addresses change on every run and left the reason for here. I added a command to the lab app that prints the start addresses of regions and ran it twice:

dotnet Bookstore.OsLab.dll layout

dotnet host     aaaae26a0000
[heap]          aaab1f891000
malloc(64)      aaab1f918960
malloc(16 MB)   fffb7a8fd010
libcoreclr.so   ffffacea0000
[stack]         ffffe7f6a000

dotnet host     aaaab44c0000
[heap]          aaaadd0ac000
malloc(64)      aaaadd133950
malloc(16 MB)   fffb74c8f010
libcoreclr.so   ffffab030000
[stack]         ffffc4b22000

The program, the heap, the mmap region the large malloc got, the libraries and the stack are in a different place each time. This is ASLR, address space layout randomization. This output comes from arm64; on x64 Linux the program and heap usually start with 55 or 56, and libraries and the stack with 7f, but the randomness is the same. The purpose is defensive: even if attackers find a memory bug, they cannot know in advance the address of the code or data they want to reach. According to the kernel's sysctl documentation, when /proc/sys/kernel/randomize_va_space is 1 the mmap regions, the stack and the code of a position-independent executable are placed randomly, and at 2 the heap is too; in the container the value was 2. On Windows the counterpart is images marked with /DYNAMICBASE, which Microsoft's linker documentation says is on by default.

The second protection is about execute permission. On Linux it is NX, the CPU refusing to run the bytes of a page as code, and on Windows DEP, Data Execution Prevention. According to Microsoft's DEP documentation, an application that tries to run code from the heap or the stack gets an access violation. That is why the heap and the stack are rw-p in the maps output above, with no x. A JIT, however, produces code by its nature. According to the default value in the runtime source, .NET runs with W^X, write xor execute, on every architecture except RISC-V: a page is never writable and executable at the same time:

dotnet Bookstore.OsLab.dll wx
DOTNET_EnableWriteXorExecute=0 dotnet Bookstore.OsLab.dll wx

mappings: writable=76 executable=28 writable+executable=0
mappings: writable=84 executable=28 writable+executable=7

By default there is not a single mapping that is both writable and executable; with W^X turned off there are 7. The documentation of some profilers asks you to turn this setting off; in production it should stay on.

The Container Memory Limit: How Does .NET See It?

Now the first question. A container's memory limit is a cgroup setting. According to the kernel's cgroup v2 documentation, memory.max is the hard limit; if usage reaches it and cannot be reduced, the OOM killer runs inside the cgroup. That was the SIGKILL in the previous post. The .NET runtime reads this file at startup and sets a heap hard limit for the GC. According to Microsoft's GC configuration documentation, when no limit is given explicitly the default in a container is the larger of 20 MB and 75% of the container limit; this setting arrived in .NET Core 3.0.

docker run --rm --memory=256m --memory-swap=256m -v ${PWD}/out:/app -w /app mcr.microsoft.com/dotnet/sdk:10.0 dotnet Bookstore.OsLab.dll gc-limits

GCHeapHardLimit         = 201326592
GCRegionRange           = 1006632960
HeapCount               = 1
GCDynamicAdaptationMode = 1
IsServerGC              = False
TotalAvailableMemory    = 192 MB
memory.max              = 268435456

75% of 256 MB: 192 MB. GC.GetGCMemoryInfo().TotalAvailableMemoryBytes also returns this cap rather than the container limit. The answer to the previous post's third exercise is here too: according to the GC source in dotnet/runtime, with a single hard limit the region range is five times the limit, 960 MB. The 1.9 GB reserve we saw in the strace from the 512 MB container was also five times 384 MB.

Now let us ask for 1 GB of managed memory. FillManaged takes a 1 MB array at every step, fills it and keeps it in a list:

try
{
    byte[] chunk = new byte[1 << 20];
    // Really write to the pages so that RSS reflects them
    Array.Fill(chunk, (byte)1);
    cart.Add(chunk);
}
catch (OutOfMemoryException ex)
{
    failure = $"OutOfMemoryException after {cart.Count} MB: {ex.Message}";
    break;
}

docker run --rm --memory=256m --memory-swap=256m -v ${PWD}/out:/app -w /app mcr.microsoft.com/dotnet/sdk:10.0 dotnet Bookstore.OsLab.dll fill 1024

TotalAvailableMemoryBytes = 192 MB, memory.max = 268435456
   64 MB  GC.GetTotalMemory=  64 MB  GC committed=  18 MB  RSS= 115 MB  memory.current=94 MB
  128 MB  GC.GetTotalMemory= 128 MB  GC committed=  78 MB  RSS= 184 MB  memory.current=161 MB
OutOfMemoryException after 189 MB: Exception of type 'System.OutOfMemoryException' was thrown.
at failure  GC.GetTotalMemory= 189 MB  GC committed= 189 MB  RSS= 247 MB  memory.current=224 MB
after GC    GC.GetTotalMemory=   0 MB  GC committed= 158 MB  RSS= 216 MB  memory.current=193 MB

This time the process did not die. When it reached the 192 MB cap the GC refused the allocation, and at the 189th MB a catchable OutOfMemoryException arrived; oom_kill in the container's memory.events stayed at 0. The cap sits at 75% so that the remaining quarter is left for memory outside the GC heap: the runtime itself, JIT-compiled code, native libraries, thread stacks. This protection covers only the managed heap. Code that takes native memory, as in the previous post, or a library that grows on the native side while the managed heap is small, can drive the container to the OOM killer without ever hitting the GC's cap.

GC.GetTotalMemory, RSS and memory.current

The same output has four numbers, and each measures something else. GC.GetTotalMemory is the total of the objects on the heap; it dropped to 0 after the GC. TotalCommittedBytes is the committed heap as seen by the last GC; because it belongs to the last GC rather than to the moment of measurement, it shows 18 MB on the 64 MB line. Even after the GC it is 158 MB: the GC does not decommit freed memory right away. RSS is all of the process's pages in RAM, shared libraries included. memory.current is the memory the kernel charges to this cgroup: according to the cgroup documentation a page is charged to the cgroup that first created it and stays there until it is released, and the page cache is included. The OOM decision is made on this last number, not on GC.GetTotalMemory.

The picture changes when Server GC is on. According to Microsoft's documentation, Server GC uses a separate heap and a separate GC thread for every logical CPU, and it can be resource-intensive; according to ASP.NET Core's memory documentation it is the default GC for ASP.NET Core apps. DATAS, Dynamic Adaptation To Application Sizes, the default for Server GC since .NET 9, exists to balance this. In a container with 2 GB and 4 CPUs I ran the churn command, which makes 4 GB of short-lived allocations in total, with three settings:

docker run --rm --memory=2g --memory-swap=2g --cpus=4 -v ${PWD}/out:/app -w /app mcr.microsoft.com/dotnet/sdk:10.0 sh -c '
dotnet Bookstore.OsLab.dll churn 4096
DOTNET_gcServer=1 dotnet Bookstore.OsLab.dll churn 4096
DOTNET_gcServer=1 DOTNET_GCDynamicAdaptationMode=0 dotnet Bookstore.OsLab.dll churn 4096'

gen0 GCs=  920  GC committed=  61 MB  peak RSS= 133 MB
gen0 GCs=  103  GC committed= 101 MB  peak RSS= 167 MB
gen0 GCs=    9  GC committed= 529 MB  peak RSS= 593 MB

Workstation GC stayed at 133 MB with 920 small GCs. Server GC without DATAS ran only 9 GCs but climbed to 593 MB; code doing the same work used enough memory to be OOMKilled had the container limit been tight. With DATAS on: 103 GCs and 167 MB. The measurement was repeated twice; the DATAS line was 188 MB on the second run, and the others stayed within a few MB.

Production Notes

Put the memory alert of a .NET service running in a container on memory.current, or on the "working set" metric the orchestrator derives from it, and watch it as a time series together with RSS. The graph of an OOMKilled pod has two typical shapes. If RSS rises smoothly and is cut off just below the limit, while GC.GetTotalMemory is low in the last moments, what grows is native memory. If RSS draws a sawtooth and the peaks touch the limit, the GC's hard limit is too close to the limit or the non-GC memory is larger than the quarter. The oom_kill counter in memory.events is the kernel-side evidence of the event.

We covered how requests and limits relate to the GC in the resource requests and limits post of the published Kubernetes series. The one rule to add from here: if you turn on Server GC for many small pods on the same node, do not turn DATAS off; if you do turn it off, cap the heap with GCHeapCount or GCHeapHardLimitPercent.

Common Pitfalls

Assuming the runtime does not see the container limit. That was true before .NET Core 3.0; today the runtime reads the cgroup. Still, an old explicit setting such as DOTNET_GCHeapHardLimit takes precedence over the container limit; check Dockerfiles and deployment manifests for forgotten GC settings.

Taking GC.GetTotalMemory for the process's memory. This method counts only live managed objects; committed but empty heap, native memory, shared libraries and the page cache are left out. In the measurement above it said 0 MB after the GC while the container was using 193 MB.

Adding up RSS values and declaring "the machine is out of memory." Shared pages are counted separately in every process's RSS; with many processes running from the same image, PSS or the cgroup counter gives the real consumption.

A Small Experiment

★☆☆ Run the gc-limits command with 128, 512 and 1024 MB limits. Confirm that GCHeapHardLimit is 75% of the limit and GCRegionRange five times that; then pass DOTNET_GCHeapHardLimitPercent=0x32, which is 50%, and observe the difference.

★★☆ Run the fill command with a 256 MB limit and DOTNET_gcServer=1. Compare the MB at which the exception arrives with Workstation GC and check memory.events.

★★★ Change map-share to use MemoryMappedFileAccess.ReadWrite and have two processes write to the same file. Measure whether one process sees what the other wrote and how Shared_Dirty changes; compare the result with CopyOnWrite.

In this post we read memory from inside the process: how it is taken from the OS, how it is shared with files and other processes, and how the container limit reaches .NET. In the next post we move on to persistence: the file system, inodes and file descriptors, the page cache, and when the data you "wrote" actually reaches the disk.