C# Threads and the ThreadPool: A Short History of Async from APM and EAP to TPL

Version note: The code in this post uses C# 10 and the .NET 6 APIs; it was built and run with the .NET 10 SDK.

Guess First

In the Bookstore we handed invoice printing to a separate thread. The main program starts the thread, writes one line and ends. Printing the invoice takes half a second. What happens to that thread when the program shuts down? Does the "Invoice printed" line make it to the screen?

Thread worker = new Thread(() =>
{
    Thread.Sleep(500);
    Console.WriteLine("Invoice printed");
});
worker.Start();
Console.WriteLine("Main finished");

The Answer: Foreground and Background Threads

It does. The program prints:

Main finished
Invoice printed

The main code finished, but the process lived for another half second and printed the invoice. The reason is that threads created with new Thread are foreground threads by default. In .NET, a process does not shut down until its last foreground thread finishes. The main thread is a foreground thread too; when it ends, the runtime checks whether any other foreground threads are left and, if so, waits for them.

Mark the same thread as background and the picture changes:

Thread worker = new Thread(() =>
{
    Thread.Sleep(500);
    Console.WriteLine("Invoice printed");
})
{
    IsBackground = true
};
worker.Start();
Console.WriteLine("Main finished");

Main finished

A background thread does not keep the process alive. The moment the main thread ends, the process shuts down, and the unfinished work is cut off where it stands, without any warning. Every thread in the thread pool is a background thread; so if you start the same work with Task.Run and do not wait for it, the result is the same:

_ = Task.Run(() =>
{
    Thread.Sleep(500);
    Console.WriteLine("Invoice printed");
});
Console.WriteLine("Main finished");

Main finished

The first lesson here: starting a piece of work does not guarantee it finishes. If you want it to finish, you have to wait for it. The await we saw in the previous post is the modern way to do that.

The Thread Class: The Ground Floor

A thread is the operating system's unit of code execution. The System.Threading.Thread class in .NET is a thin wrapper around it: when you call new Thread(...) and Start(), the operating system really does open a new thread. You can give the thread a name and wait for it to finish with Join():

decimal total = 0;
Thread calculator = new Thread(() =>
{
    decimal[] prices = { 12.99m, 9.50m, 15.00m };
    total = prices.Sum();
    Console.WriteLine($"{Thread.CurrentThread.Name} ran on thread {Environment.CurrentManagedThreadId}");
})
{
    Name = "TotalCalculator"
};

calculator.Start();
calculator.Join(); // blocks the main thread until the thread finishes
Console.WriteLine($"Total: {total.ToString(System.Globalization.CultureInfo.InvariantCulture)}");

TotalCalculator ran on thread 4
Total: 37.49

The code works, but look at the shape of it. A thread cannot return a value; we write the result into the outer total variable and block the main thread with Join() so that we can read it correctly. There is no built-in way to learn when the thread finished, to get its result or to chain two jobs one after the other; you build all of that by hand.

Errors are even harsher. An exception that is not caught inside a thread takes down the whole process:

try
{
    Thread printer = new Thread(() => throw new InvalidOperationException("Printer is offline"));
    printer.Start();
    printer.Join();
}
catch (InvalidOperationException ex)
{
    Console.WriteLine($"Caught: {ex.Message}"); // never reached
}

Unhandled exception. System.InvalidOperationException: Printer is offline

A stack trace follows this line and the process exits. The try/catch could not catch the exception, because it was thrown on another thread's call stack. Every thread has its own call stack, and a catch only sees errors on its own stack. Keep this in mind: one of the most important things Task brings is catching the error and carrying it to the code that is waiting for it.

Threads Are Expensive: The ThreadPool

Every time a thread opens, the operating system reserves a stack, sets up its own structures and tears it all down when the work is done. For short jobs that overhead is bigger than the job itself. Let's run 1000 jobs that do nothing, in two ways:

using System.Diagnostics;

const int jobCount = 1000;

Stopwatch watch = Stopwatch.StartNew();
Thread[] threads = new Thread[jobCount];
for (int i = 0; i < jobCount; i++)
{
    threads[i] = new Thread(() => { });
    threads[i].Start();
}
foreach (Thread thread in threads)
{
    thread.Join();
}
Console.WriteLine($"new Thread: {watch.ElapsedMilliseconds} ms");

watch.Restart();
using CountdownEvent remaining = new CountdownEvent(jobCount);
for (int i = 0; i < jobCount; i++)
{
    ThreadPool.QueueUserWorkItem(_ => remaining.Signal());
}
remaining.Wait();
Console.WriteLine($"ThreadPool: {watch.ElapsedMilliseconds} ms, pool threads: {ThreadPool.ThreadCount}");

new Thread: 39 ms
ThreadPool: 0 ms, pool threads: 7

On my machine, across five separate runs, opening and closing 1000 threads took between 33 and 51 ms; the thread pool finished the same 1000 jobs in under a millisecond every time (ElapsedMilliseconds printed 0) using between 4 and 8 threads. The numbers change from run to run and machine to machine, but the gap is always of this order. The thread pool is a pool of workers that does not close the threads it has opened and gives new work to an idle thread. When a thread finishes its job it does not die; it takes the next job from the queue.

The pool has its own rules. The minimum number of threads it keeps ready is usually the number of cores; when it has to go above that, as we saw in the previous post, it adds threads slowly and cautiously. That stinginess is deliberate: for CPU-bound work, more threads than cores do not make the work faster, they only add switching between threads.

ThreadPool.GetMinThreads(out int minWorkers, out int minIo);
ThreadPool.GetMaxThreads(out int maxWorkers, out int maxIo);
Console.WriteLine($"Cores: {Environment.ProcessorCount}");
Console.WriteLine($"Worker threads min: {minWorkers}, max: {maxWorkers}");

using ManualResetEventSlim done = new ManualResetEventSlim();
ThreadPool.QueueUserWorkItem(_ =>
{
    Thread current = Thread.CurrentThread;
    Console.WriteLine($"Pool thread: {current.IsThreadPoolThread}, background: {current.IsBackground}");
    done.Set();
});
done.Wait();

Cores: 10
Worker threads min: 10, max: 32767
Pool thread: True, background: True

QueueUserWorkItem, like Thread, returns no result and does not carry errors; it only says "run this job at some point". The thread pool is infrastructure, not a programming model. What .NET looked for over a decade was a readable model on top of this infrastructure. Let's look at the stops along that search.

APM: Begin and End

The first model, which came with .NET Framework 1.0 (2002), is the Asynchronous Programming Model, APM for short. An asynchronous operation is split into two methods: BeginXxx starts the work and returns an IAsyncResult, EndXxx gets the result. When the work is done, the callback you passed in is called:

using System.Text;

string path = Path.Combine(Path.GetTempPath(), "catalog.txt");
File.WriteAllText(path, "Dune;Foundation;Hyperion");

FileStream stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, useAsync: true);
byte[] buffer = new byte[1024];
using ManualResetEventSlim done = new ManualResetEventSlim();

stream.BeginRead(buffer, 0, buffer.Length, result =>
{
    int read = stream.EndRead(result);
    Console.WriteLine($"Callback on thread {Environment.CurrentManagedThreadId}, sync: {result.CompletedSynchronously}");
    Console.WriteLine(Encoding.UTF8.GetString(buffer, 0, read));
    stream.Dispose();
    done.Set();
}, null);

Console.WriteLine($"BeginRead returned on thread {Environment.CurrentManagedThreadId}");
done.Wait();
File.Delete(path);

Callback on thread 1, sync: True
Dune;Foundation;Hyperion
BeginRead returned on thread 1

The output comes with a surprise: the callback ran on the same thread, before BeginRead even returned. The file is small and in the operating system's cache, so the read finished immediately; CompletedSynchronously says exactly that. Everyone writing APM code had to handle both cases: the callback sometimes runs on another thread and sometimes on the same thread, right away. On top of that, you had to carry the result out of the callback, catch errors from the exception EndRead throws, and open a new BeginXxx inside the callback to chain two operations. A three-step flow meant three nested callbacks.

EAP: The Async Method and Its Completed Event

.NET Framework 2.0 (2005) tried an easier model: the Event-based Asynchronous Pattern, EAP for short. A method named XxxAsync starts the work, and when it finishes the XxxCompleted event fires. BackgroundWorker, which you still come across in desktop applications, is an example of this model:

using System.ComponentModel;

using ManualResetEventSlim done = new ManualResetEventSlim();
BackgroundWorker worker = new BackgroundWorker();

worker.DoWork += (sender, e) =>
{
    Thread.Sleep(300); // as if the stock count were running
    e.Result = 42;
};
worker.RunWorkerCompleted += (sender, e) =>
{
    Console.WriteLine($"Stock count: {e.Result}");
    done.Set();
};

worker.RunWorkerAsync();
Console.WriteLine("RunWorkerAsync returned");
done.Wait();

RunWorkerAsync returned
Stock count: 42

EAP's real trick was bringing the Completed event back to the environment it was started from: in WinForms the event fires on the UI thread, so you can safely update a label on the screen from inside it. That idea later becomes part of the foundation of async/await under the name SynchronizationContext. But EAP was scattered too. The result is read from an e.Result of type object, the error from e.Error; chaining two operations still meant nesting events. Microsoft's documentation today marks both APM and EAP as "legacy" and does not recommend them for new code.

TPL: Enter Task

.NET Framework 4.0 (2010) brought the Task Parallel Library, TPL for short. At its center is a single idea: represent asynchronous work as an object. Task and Task<TResult> are "work in progress or already done". You can ask for its status, get its result, read its error and attach the next job to run when it finishes:

Task<int> countTask = Task.Run(() =>
{
    Thread.Sleep(300); // as if the stock count were running
    return 42;
});

Task printTask = countTask.ContinueWith(previous =>
    Console.WriteLine($"Stock count: {previous.Result}, status: {previous.Status}"));

Console.WriteLine("Task.Run returned");
await printTask; // there was no await in 2010; you would block with printTask.Wait()

Task.Run returned
Stock count: 42, status: RanToCompletion

Compare it with the total example we wrote with a thread: there is no shared outer variable, the result lives in the task itself. The thread pool runs the work; we do not manage threads. With ContinueWith we can say "when this finishes, do that", and inside ContinueWith it is safe to read previous.Result because the task has already finished. A task also catches and stores the exception; it does not crash the process the way a thread does. We will go into the details in the exceptions post of this series.

TPL also built bridges to the older models. Task.Factory.FromAsync turns APM's Begin/End pair into a single task. We will see that bridge below, together with async/await.

async/await: The Same Task, Readable Code

TPL got the model right, but as ContinueWith chains grew longer the code became unreadable again. C# 5 and .NET Framework 4.5 (2012) added the last piece: async and await. The compiler splits the straight-line code you write into continuations behind the scenes. Let's put the same file read side by side in the language of the different eras:

using System.Text;

string path = Path.Combine(Path.GetTempPath(), "catalog.txt");
await File.WriteAllTextAsync(path, "Dune;Foundation;Hyperion");
byte[] buffer = new byte[1024];

using (FileStream stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, useAsync: true))
{
    // TPL bridge: turns APM's Begin/End pair into a single task
    int read = await Task.Factory.FromAsync(stream.BeginRead, stream.EndRead, buffer, 0, buffer.Length, null);
    Console.WriteLine($"FromAsync: {Encoding.UTF8.GetString(buffer, 0, read)}");
}

using (FileStream stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, useAsync: true))
{
    // today's way: TAP
    int read = await stream.ReadAsync(buffer, 0, buffer.Length);
    Console.WriteLine($"ReadAsync: {Encoding.UTF8.GetString(buffer, 0, read)}");
}

File.Delete(path);

FromAsync: Dune;Foundation;Hyperion
ReadAsync: Dune;Foundation;Hyperion

No callbacks, no events, no IAsyncResult. The code reads top to bottom like synchronous code, but the thread is released at the await points. Today's model is called the Task-based Asynchronous Pattern, TAP for short: an asynchronous operation is a single method whose name ends in Async and that returns a Task or Task<T>. All of .NET's newer APIs follow this pattern, and the rest of this series is built entirely on it.

An important detail: async/await did not bring a new runtime mechanism. Underneath it are the Task and thread pool of 2010. That is why, in the next post, we will get to know Task itself up close before moving on to await.

Common Pitfalls

Opening a new Thread for every small job. As we measured, opening a thread is expensive. For short-lived jobs use the thread pool, which in practice today means Task.Run or async APIs. You open your own thread only for long-running work that lives for the whole process and needs special settings.

Trying to stop a thread from outside. .NET Framework had Thread.Abort(), which cut a running thread off wherever it was and left locks and half-finished work lying around. On .NET 5 and later this method compiles, but it gives warning SYSLIB0006 and throws an exception when called:

Thread worker = new Thread(() => Thread.Sleep(5000));
worker.Start();
worker.Abort();

Program.cs(3,1): warning SYSLIB0006: 'Thread.Abort()' is obsolete: 'Thread.Abort is not supported and throws PlatformNotSupportedException.' (https://aka.ms/dotnet-warnings/SYSLIB0006)
Unhandled exception. System.PlatformNotSupportedException: Thread abort is not supported on this platform.

In modern .NET the way to stop a piece of work is for the work itself to check at regular intervals whether it has been asked to stop. The tool for that is CancellationToken; we will see it in the cancellation post of this series.

Mistaking a task for a thread. A Task is a piece of work, not a thread. A task started with Task.Run runs on a pool thread, but behind a task like await Task.Delay(1000) there is no thread at all while it waits. The idea that "every task opens a thread" cannot explain why the 100 queries in the previous post finished in one second.

Giving critical work to a background thread. When the process shuts down, background threads are cut off without notice. Do not leave work that must not stop halfway, such as printing an invoice or writing the last line to a file, without waiting for it.

A Small Experiment

★☆☆ In the example at the top, add worker.Join() right after worker.Start() and run it with IsBackground = true. Does "Invoice printed" show up this time? Why?

★★☆ In the measurement comparing threads with the thread pool, raise jobCount to 10,000. How do the two timings change? How many threads did the pool end up using? Remember that the numbers will be different on your machine.

★★★ Rewrite the BackgroundWorker example with Task.Run and await. If you throw an exception inside DoWork, where does the error land in each version: in e.Error for EAP, and where in the Task version?

We have seen that a thread is an expensive worker, the thread pool is a ready pool of workers, and a Task is the "work in progress" object built on top of that pool. In the next post we look closely at Task and Task<T>: a task's lifecycle, the ways to create one, the sneaky difference between Task.Run and Task.Factory.StartNew, and the right way to get the result.