Goroutines and Channels: Go's Concurrency Model

Version note: The code in this post was built and run with Go 1.27.4; the module's go line is 1.26.0 and the code also compiles with Go 1.26.11. The measurements were taken on the machine I used for this post with GOMAXPROCS=4; your numbers will differ.

Guess First

In Bookstore, the online bookstore used as the shared example across this site's series, we want to carry a book's stock count over a channel. If you think of it like writing to a queue in C# and reading it back, this program looks fine:

// Lab 07: an unbuffered send with nobody receiving.
package main

import "fmt"

func main() {
    stock := make(chan int)
    stock <- 12 // blocks until another goroutine receives
    fmt.Println(<-stock)
}

Does it print 12, or does something happen?

go run ./labs/07-deadlock

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
main.main()
    /home/user/bookstore/labs/07-deadlock/main.go:8 +0x38
exit status 2

The channel opened with make(chan int) has no buffer. A send on such a channel waits until another goroutine takes the value. There is no other goroutine in the program, so main would have waited on line 8 forever; the runtime saw that every goroutine was asleep and stopped the program. A channel is not a queue but a point where two goroutines meet. In this post we look at goroutines, the scheduler that runs them, and channels, using Bookstore's stock check.

Goroutines: A Cheap Function Call

When you call a function with the go keyword, it starts running in a new goroutine and the caller continues without waiting. Effective Go describes a goroutine as "lightweight, costing little more than the allocation of stack space". I measured how cheap it is: I start 100,000 goroutines and park all of them on a channel.

// Lab 07: what 100,000 parked goroutines cost.
package main

import (
    "fmt"
    "runtime"
    "sync"
    "time"
)

func main() {
    const n = 100_000
    fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))

    var before runtime.MemStats
    runtime.ReadMemStats(&before)

    start := time.Now()
    release := make(chan struct{})
    var wg sync.WaitGroup
    for range n {
        wg.Add(1)
        go func() {
            defer wg.Done()
            <-release // park until main closes the channel
        }()
    }
    fmt.Printf("started %d goroutines in %v\n", n, time.Since(start).Round(time.Millisecond))

    var after runtime.MemStats
    runtime.ReadMemStats(&after)
    fmt.Println("goroutines:", runtime.NumGoroutine())
    fmt.Printf("stack in use: %.1f MB (%.1f KB per goroutine)\n",
        float64(after.StackInuse-before.StackInuse)/1e6,
        float64(after.StackInuse-before.StackInuse)/n/1e3)

    close(release) // one close wakes every receiver
    wg.Wait()
    fmt.Println("after close:", runtime.NumGoroutine())
}

GOMAXPROCS=4 go run ./labs/07-goroutines

GOMAXPROCS: 4
started 100000 goroutines in 77ms
goroutines: 100001
stack in use: 204.9 MB (2.0 KB per goroutine)
after close: 1

100,000 goroutines started in 77 ms and each used 2 KB of stack. That matches the stackMin = 2048 constant in the runtime source. A goroutine's stack is not fixed; in a deep call chain the runtime grows it by copying it to a larger area. For comparison, Windows reserves 1 MB of stack for a thread by default. Doing the same job with 100,000 threads is not even an option; in Go, starting a goroutine for an HTTP request, a background job or a channel reader is routine.

A single close(release) woke every receiver and the count went back to 1, which is main. We waited for the goroutines to finish with sync.WaitGroup; we will look at it, and at WaitGroup.Go from Go 1.25, in detail in post 08.

The Scheduler: G, M and P

Goroutines are not OS threads. The runtime spreads many goroutines (G) over a few OS threads (M); this is called M:N scheduling. The third piece in between is P, the processor: an M has to hold a P to run Go code, and the number of Ps equals GOMAXPROCS. The default is the number of logical CPUs on the machine; since Go 1.25, on Linux the container's cgroup CPU limit is used if it is lower.

Every P has its own local run queue, and there is also a global run queue that all Ps look at. When a P's queue runs empty it steals work from another P's queue. When a goroutine starts waiting on a channel it is parked, and the same M runs the next goroutine in the queue; the thread does not sit idle. Network I/O works the same way: a goroutine waiting for data on a socket is handed to the network poller and goes back into a queue when the data arrives. Since Go 1.14 a long-running loop cannot hold the CPU forever either; the runtime can stop goroutines asynchronously through preemption.

Asynchronous Without async/await: No Function Colours

In the C# Asenkron Programlama series' "C# async/await Nasıl Çalışır: State Machine ve Continuation" post we saw that the compiler turns every async method into a state machine and releases the thread at await. The price is the split Bob Nystrom described in his 2015 essay "What Color is Your Function?": a method that calls an async method has to be async as well, Task spreads up the call stack, and putting async code behind a synchronous interface gets awkward.

Go has no such split. http.Get, conn.Read, a channel receive or a database query are written as ordinary blocking calls. What blocks is the goroutine, not the thread; meanwhile the scheduler runs other goroutines on the same thread. In Go the runtime does the job of C#'s state machine, and you just write straight-line code. That is also why the net/http server starts a goroutine per connection; it explains why the handler panic in post 05 only affected that connection. In return, you have to start concurrent work explicitly with go and collect its results with a channel or a WaitGroup; the language has no ready-made counterpart of C#'s Task.WhenAll.

Channels: Unbuffered, Buffered, close and range

Go's concurrency model builds on Tony Hoare's 1978 work on CSP, Communicating Sequential Processes. Effective Go puts it in one sentence: "Do not communicate by sharing memory; instead, share memory by communicating." Instead of keeping data in a variable that several goroutines access, you pass it from its owner to the next goroutine over a channel. The theory of the model is in the Concurrency Teorisi series' "Concurrency Modelleri ve Concurrent Kodu Doğrulamak" post; here we look at the practice.

// Lab 07: unbuffered and buffered channels, close and range, select.
package main

import (
    "fmt"
    "time"
)

var start = time.Now()

func logf(format string, args ...any) {
    fmt.Printf("%4dms  "+format+"\n", append([]any{time.Since(start).Milliseconds()}, args...)...)
}

// reserve sends n reservation IDs and closes the channel when done.
func reserve(out chan<- string, n int) {
    for i := 1; i <= n; i++ {
        out <- fmt.Sprintf("RES-%d", i)
        logf("sent RES-%d", i)
    }
    close(out) // only the sender closes
}

func main() {
    // 1) Unbuffered: every send waits for a receive.
    unbuffered := make(chan string)
    go reserve(unbuffered, 3)
    time.Sleep(100 * time.Millisecond)
    for id := range unbuffered { // stops when the channel is closed and empty
        logf("got  %s", id)
        time.Sleep(100 * time.Millisecond)
    }

    // 2) Buffered: sends do not wait while there is room.
    start = time.Now()
    buffered := make(chan string, 3)
    go reserve(buffered, 3)
    time.Sleep(100 * time.Millisecond)
    for id := range buffered {
        logf("got  %s", id)
    }

    // 3) Receiving from a closed channel: zero value and ok == false.
    id, ok := <-buffered
    fmt.Printf("closed: %q %t\n", id, ok)

    // 4) select: wait for whichever is ready first, or give up.
    slow := make(chan string)
    go func() {
        time.Sleep(300 * time.Millisecond)
        slow <- "stock: 12"
    }()
    select {
    case v := <-slow:
        fmt.Println(v)
    case <-time.After(100 * time.Millisecond):
        fmt.Println("inventory did not answer in 100ms")
    }

    // 5) default makes select non-blocking.
    select {
    case v := <-slow:
        fmt.Println("late answer:", v)
    default:
        fmt.Println("nothing ready yet")
    }
    fmt.Println("late answer:", <-slow) // the goroutine is still waiting to send
}

go run ./labs/07-channels

 102ms  got  RES-1
 102ms  sent RES-1
 214ms  got  RES-2
 217ms  sent RES-2
 316ms  got  RES-3
 316ms  sent RES-3
   0ms  sent RES-1
   0ms  sent RES-2
   0ms  sent RES-3
 102ms  got  RES-1
 102ms  got  RES-2
 102ms  got  RES-3
closed: "" false
inventory did not answer in 100ms
nothing ready yet
late answer: stock: 12

In the first block reserve tried to send RES-1 at millisecond 0, but since main slept for 100 ms the "sent" line came at millisecond 102. On an unbuffered channel the send and the receive complete together; the value passes straight from one goroutine to the other. The next two reservations also moved at the receiver's pace. The right side of the figure shows this rendezvous. On the buffered channel with capacity 3, all three sends finished at millisecond 0; the sender does not wait for the receiver until the buffer is full.

chan<- string and <-chan string are directional channel types. reserve can only send; if it tries to receive by mistake, it gets a compile error. close means "no more values will come"; for range ends when the channel is closed and drained, and receiving from a closed channel returns the zero value and ok == false without waiting. The sending side always closes the channel.

select picks whichever of several channel operations is ready first. Since time.After returns a channel that delivers a value after a given time, it turns into a simple timeout; default runs without waiting if no case is ready. For HTTP requests we will give timeouts with context; that is the topic of post 08.

A nil channel blocking forever is a useful property: if you set a channel variable to nil inside a select, that case is never chosen again. It is a common way to read from two sources and switch one off once it is done.

Compared with C# Channels

System.Threading.Channels from the C# Asenkron Programlama series' "C# Channels ile Producer/Consumer: Backpressure, Bounded Channel ve Pipeline" post is conceptually very close to Go channels. Channel.CreateBounded<T>(3) is the counterpart of a buffered channel, Writer.Complete() of close, and await foreach (var x in reader.ReadAllAsync()) of for range. A bounded channel making the writer wait when it is full is the same backpressure as a Go send waiting on a full buffer.

There are differences too. In C# a channel is a library; in Go it is part of the language, and waiting on several channels at once with select is the language's own syntax. In Go make(chan T) opens a channel with zero capacity by default, one that makes the sender meet the receiver; in C# you always choose the capacity yourself, with CreateBounded or CreateUnbounded. Writing to a closed channel means TryWrite returning false or an exception in C#, but a panic in Go. I tried all three panics one by one in the lab:

func try(name string, f func()) {
    defer func() { fmt.Printf("%-16s %v\n", name, recover()) }()
    f()
}

func main() {
    try("send on closed", func() {
        ch := make(chan int, 1)
        close(ch)
        ch <- 1
    })
    try("close twice", func() {
        ch := make(chan int)
        close(ch)
        close(ch)
    })
    try("close nil", func() {
        var ch chan int
        close(ch)
    })

go run ./labs/07-panics

send on closed   send on closed channel
close twice      close of closed channel
close nil        close of nil channel

Pipeline, Fan-out and Fan-in: Checking a Cart's Stock

In Bookstore we have to ask the inventory service about every book in the cart, and every call takes 50 ms. Asking for six books one after the other takes 300 ms. I wrote it as a three-stage pipeline: generate puts the ISBNs into a channel, three check workers read from that same channel (fan-out), and merge collects the three result channels into one (fan-in).

// generate is the first stage: it turns a slice into a channel.
func generate(isbns []string) <-chan string {
    out := make(chan string)
    go func() {
        defer close(out)
        for _, isbn := range isbns {
            out <- isbn
        }
    }()
    return out
}

// check is a worker: it reads ISBNs until in is closed.
func check(in <-chan string) <-chan result {
    out := make(chan result)
    go func() {
        defer close(out)
        for isbn := range in {
            b, err := lookup(isbn)
            out <- result{isbn: isbn, stock: b.Stock, err: err}
        }
    }()
    return out
}

// merge is fan-in: it copies every input channel into one output channel.
func merge(ins ...<-chan result) <-chan result {
    out := make(chan result)
    var wg sync.WaitGroup
    for _, in := range ins {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for r := range in {
                out <- r
            }
        }()
    }
    go func() {
        wg.Wait()
        close(out) // close once, after every copier is done
    }()
    return out
}

func main() {
    cart := []string{"9780441172719", "9780547928227", "9780141439549",
        "9780141439587", "9780141439686", "9780000000000"}

    start := time.Now()
    for _, isbn := range cart {
        if _, err := lookup(isbn); err != nil {
            fmt.Println("sequential:", err)
        }
    }
    fmt.Println("sequential:", time.Since(start).Round(10*time.Millisecond))

    start = time.Now()
    isbns := generate(cart)
    workers := []<-chan result{check(isbns), check(isbns), check(isbns)} // fan-out: 3 readers, one channel
    for r := range merge(workers...) {
        if r.err != nil {
            fmt.Println("  error:", r.err)
            continue
        }
        fmt.Printf("  %s stock=%d\n", r.isbn, r.stock)
    }
    fmt.Println("pipeline with 3 workers:", time.Since(start).Round(10*time.Millisecond))
}

go run ./labs/07-stockcheck

sequential: isbn 9780000000000: book not found
sequential: 310ms
  9780141439549 stock=4
  9780547928227 stock=0
  9780441172719 stock=12
  9780141439686 stock=3
  error: isbn 9780000000000: book not found
  9780141439587 stock=7
pipeline with 3 workers: 100ms

With three workers the time dropped from 310 ms to 100 ms: six jobs, finished in two rounds of three. The order of the results, however, is not the cart's order; whichever worker finishes first delivers first. If order matters, carry an index with each result and sort afterwards. Note the detail in merge: the output channel is closed by a separate goroutine that waits for every copier to finish. If one of the copiers closed it, the others would write to a closed channel and panic. That the go func() in the loop can use in directly is thanks to the Go 1.22 loop variable change we discussed in post 03.

Goroutine Leaks: Goroutines That Never End

The garbage collector does not collect a blocked goroutine. If a goroutine tries to write to a channel that nobody reads, it stays in memory until the program ends. The most common cause is the collecting side returning early:

// firstMissing starts one goroutine per ISBN and returns at the first error.
func firstMissing(isbns []string) error {
    errs := make(chan error) // unbuffered: each sender waits for a receiver
    for _, isbn := range isbns {
        go func() {
            _, err := catalog.Store{}.ByISBN(isbn)
            if err == nil {
                time.Sleep(50 * time.Millisecond) // known books also ask the inventory service
            }
            errs <- err
        }()
    }
    for range isbns {
        if err := <-errs; err != nil {
            return err // the remaining senders never get a receiver
        }
    }
    return nil
}

go run ./labs/07-leak

call 1: book not found, goroutines=4
call 2: book not found, goroutines=7
call 3: book not found, goroutines=10

Every call took the first error, returned and left three goroutines behind. If this code were in an HTTP handler, every request would leak three goroutines; days later the memory graph would creep upwards and the cause would not be easy to spot in the code. The fix in this example is simple: make room in the channel for every sender.

errs := make(chan error, len(isbns)) // room for every sender

With this change goroutines=1 remained after all three calls. The more general fix is being able to tell goroutines "I no longer want the result"; in Go that is cancellation with context. In post 08 we will see it, along with the goroutineleak profile that became generally available in Go 1.27.

In Production

A panic in a goroutine stops the whole process. The Recover middleware from post 05 only protects the handler's own goroutine; if a goroutine you start inside a handler panics, the API goes down completely:

go run ./labs/07-panics crash

send on closed   send on closed channel
close twice      close of closed channel
close nil        close of nil channel
panic: assignment to entry in nil map

goroutine 19 [running]:
main.main.func5()
    /home/user/bookstore/labs/07-panics/main.go:37 +0x34
created by main.main in goroutine 1
    /home/user/bookstore/labs/07-panics/main.go:34 +0xd0
exit status 2

The defer/recover in main could do nothing, because recover only catches a panic in its own goroutine. Every goroutine that starts background work should carry its own recover where needed.

Starting goroutines without a limit. A goroutine is cheap, but the work behind it is not. Opening a goroutine for every incoming order, each going to the inventory service, can exhaust that service and the database connection pool at once. A fixed number of workers, as in the lab, is the simplest way to limit parallelism; in post 08 we will do the same with less code using errgroup.SetLimit.

GOMAXPROCS in containers. Before Go 1.25, a Go service in a container limited to 2 CPUs on a 64-core host opened 64 Ps and was constantly throttled because of the limit. From 1.25 on, the runtime takes the cgroup limit into account on Linux; keep this difference in mind when you put services built with an older release into containers.

Common Pitfalls

Treating a channel as a queue. That is the deadlock in Guess First: you cannot write to and read from an unbuffered channel in the same goroutine. Growing the buffer "to make the problem go away" usually just postpones the bug.

Calling close on the receiving side. If the receiver closes the channel, a goroutine still trying to send panics. Closing is always the sender's job; with several senders, a single place that waits for all of them closes it, as in merge.

A goroutine whose result is never read. An early return, an unread channel and workers with no cancellation are the classic trio behind goroutine leaks. At every go line, ask "when and how will this goroutine end?"

Relying on order. Results from a fan-in arrive in the order the jobs finish. If a test passes one day and fails the next, this is the first suspect.

Using a shared variable without protection. If goroutines write to the same map or counter without a channel, there is a data race. Bookstore's Service.nextID counter and memstore are in exactly this situation; in the next post we will find and fix it with the race detector.

A Small Experiment

★☆☆ Fix labs/07-deadlock in two different ways: first make the channel make(chan int, 1), then leave it unbuffered and move the send into a separate goroutine. Write the difference between the two fixes in one sentence.

★★☆ Set the number of workers in labs/07-stockcheck to 1, 2, 6 and 12 and compare the times. Why did it stop getting faster after 6?

★★★ Change merge so that it returns the results in the cart's order. Hint: let generate send the index along with the ISBN; write the results into a slice by index and print that slice when the channel is closed.

We have seen goroutines and channels, but a real service is missing a piece: how do we tell goroutines that a request has been cancelled? In the next post we continue with context.Context, sync.Mutex and WaitGroup, a worker pool with errgroup, the race detector and goroutine leak detection. That is also where we fix Bookstore's order number counter and memstore, which break under concurrent use.