LINQ Grouping: GroupBy, IGrouping and ToLookup

Version note: The code in this post uses C# 10 and the .NET 6 APIs; it was built and run with the .NET 10 SDK.

Guess First

In this post we add one field to our bookstore data: the book's genre. Put the file below next to Program.cs as Data.cs:

record Book(string Isbn, string Title, string Author, decimal Price, int Stock, int Year, string Genre);

static class Store
{
    public static List<Book> Books { get; } = new()
    {
        new("978-0441172719", "Dune", "Frank Herbert", 12.99m, 4, 1965, "Science Fiction"),
        new("978-0553293357", "Foundation", "Isaac Asimov", 9.99m, 0, 1951, "Science Fiction"),
        new("978-0451524935", "1984", "George Orwell", 8.99m, 7, 1949, "Dystopia"),
        new("978-0261102217", "The Hobbit", "J.R.R. Tolkien", 10.99m, 2, 1937, "Fantasy"),
        new("978-0547928210", "The Fellowship of the Ring", "J.R.R. Tolkien", 15.99m, 3, 1954, "Fantasy"),
        new("978-0553382563", "I, Robot", "Isaac Asimov", 7.99m, 5, 1950, "Science Fiction"),
        new("978-0060850524", "Brave New World", "Aldous Huxley", 11.49m, 1, 1932, "Dystopia"),
        new("978-0451526342", "Animal Farm", "George Orwell", 6.99m, 9, 1945, "Satire"),
        new("978-0441569595", "Neuromancer", "William Gibson", 10.49m, 0, 1984, "Science Fiction"),
        new("978-0441478125", "The Left Hand of Darkness", "Ursula K. Le Guin", 9.49m, 6, 1969, "Science Fiction"),
    };
}

We group the books by genre in two different ways: one with GroupBy, the other with ToLookup. Then we add a new science fiction book to the list and ask both sides how many science fiction books there are. What does this code print?

List<Book> books = new(Store.Books);

IEnumerable<IGrouping<string, Book>> byGenre = books.GroupBy(b => b.Genre);
ILookup<string, Book> lookup = books.ToLookup(b => b.Genre);

books.Add(new Book("978-0345391803", "The Hitchhiker's Guide to the Galaxy", "Douglas Adams", 9.99m, 3, 1979, "Science Fiction"));

Console.WriteLine($"GroupBy: {byGenre.First(g => g.Key == "Science Fiction").Count()}");
Console.WriteLine($"ToLookup: {lookup["Science Fiction"].Count()}");

GroupBy Is Deferred, ToLookup Runs Now

Output:

GroupBy: 6
ToLookup: 5

Both lines describe the same grouping, but they run at different moments. GroupBy uses deferred execution: the byGenre variable holds no groups, only the instruction "group this list by genre". The instruction ran when First was called, and at that moment the list had six science fiction books. ToLookup runs immediately: on the line where it's called it walks the list, builds the groups and is done. It doesn't see the book added later; for it, the list is frozen as it was at that moment.

Think of a school. GroupBy is the note the principal leaves saying "tomorrow morning, split the students into groups by birth month"; whoever is at school tomorrow gets grouped. ToLookup is a photo taken today and pinned to the board. Both are useful; you just need to know which one you want. In this post we'll first look at the structure GroupBy returns, then what you can do with groups, and finally when to prefer ToLookup.

IGrouping: The Key and the Elements

foreach (IGrouping<string, Book> group in Store.Books.GroupBy(b => b.Genre))
{
    Console.WriteLine($"{group.Key} ({group.Count()}): {string.Join(" | ", group.Select(b => b.Title))}");
}

Science Fiction (5): Dune | Foundation | I, Robot | Neuromancer | The Left Hand of Darkness
Dystopia (2): 1984 | Brave New World
Fantasy (2): The Hobbit | The Fellowship of the Ring
Satire (1): Animal Farm

GroupBy returns an IEnumerable<IGrouping<TKey, TElement>>. Each IGrouping carries a Key property and is itself an IEnumerable<TElement>. That means you can run Count, Select, Sum, in short every LINQ operator, on a group.

Pay attention to the order. The documentation gives a guarantee here: groups come in the order of the element in the source that first produced each key, and the elements inside a group keep their order from the source. Since the first book in the list, Dune, is science fiction, that group comes first; since Animal Farm brought a new genre last, Satire is at the end. If you want the groups alphabetically or by size, you have to say so explicitly with OrderBy.

One Row per Group

The real power of grouping is reducing each group to a single row. Like GROUP BY together with aggregate functions in SQL:

using System.Globalization;

CultureInfo.CurrentCulture = CultureInfo.InvariantCulture;

var authors = Store.Books
    .GroupBy(b => b.Author)
    .Select(g => new
    {
        Author = g.Key,
        Books = g.Count(),
        Stock = g.Sum(b => b.Stock),
        Cheapest = g.MinBy(b => b.Price)!.Title, // a group is never empty
    })
    .Where(x => x.Books > 1)
    .OrderByDescending(x => x.Stock);

foreach (var a in authors)
{
    Console.WriteLine($"{a.Author}: {a.Books} books, stock {a.Stock}, cheapest: {a.Cheapest}");
}

George Orwell: 2 books, stock 16, cheapest: Animal Farm
Isaac Asimov: 2 books, stock 5, cheapest: I, Robot
J.R.R. Tolkien: 2 books, stock 5, cheapest: The Hobbit

The pattern is always the same: split with GroupBy, produce a summary of each group with Select, filter with Where and sort if needed. Here Where(x => x.Books > 1) is the counterpart of HAVING in SQL. The ! after MinBy is there to silence the nullable warning; in the previous post we saw that MinBy returns null on an empty sequence, but a group coming out of GroupBy always contains at least one element. Because we use an anonymous type, var is mandatory here; the type has no name.

GroupBy has two more handy overloads. One puts a part of the element into the group instead of the whole thing, the other turns each group directly into a result. You can also write the same thing in query syntax:

IEnumerable<IGrouping<string, string>> titlesByGenre = Store.Books.GroupBy(b => b.Genre, b => b.Title);
Console.WriteLine(string.Join(" | ", titlesByGenre.Select(g => $"{g.Key}: {g.First()}")));

IEnumerable<string> stockByGenre = Store.Books.GroupBy(
    b => b.Genre,
    (genre, books) => $"{genre}: {books.Sum(b => b.Stock)}");
Console.WriteLine(string.Join(" | ", stockByGenre));

IEnumerable<string> report =
    from b in Store.Books
    group b.Title by b.Genre into g
    orderby g.Count() descending, g.Key
    select $"{g.Key}: {g.Count()}";
Console.WriteLine(string.Join(" | ", report));

Science Fiction: Dune | Dystopia: 1984 | Fantasy: The Hobbit | Satire: Animal Farm
Science Fiction: 15 | Dystopia: 8 | Fantasy: 5 | Satire: 9
Science Fiction: 5 | Dystopia: 2 | Fantasy: 2 | Satire: 1

In the first line, the element selector (b => b.Title) put only the titles into the groups; the type of a group is now IGrouping<string, string>. In the second line, the result selector took the key and the books in the group and produced a single string. In query syntax, group ... by ... into g gives the group a name and lets you continue the query. Without into, the group clause ends the query. Notice that in query syntax the element expression (b.Title) is written before the key (b.Genre); the reverse of the argument order in method syntax.

Composite Keys

Sometimes a single field isn't enough: "each decade of each author", for example. You pass a key that carries more than one field:

var byAuthorAndDecade = Store.Books.GroupBy(b => new { b.Author, Decade = b.Year / 10 * 10 });

foreach (var g in byAuthorAndDecade.Where(g => g.Count() > 1))
{
    Console.WriteLine($"{g.Key.Author}, {g.Key.Decade}s: {string.Join(" | ", g.Select(b => b.Title))}");
}

int tupleGroups = Store.Books.GroupBy(b => (b.Author, b.Year / 10 * 10)).Count();
int classGroups = Store.Books.GroupBy(b => new DecadeKey(b.Author, b.Year / 10 * 10)).Count();
Console.WriteLine($"Anonymous type: {byAuthorAndDecade.Count()}, tuple: {tupleGroups}, class: {classGroups}");

class DecadeKey
{
    public DecadeKey(string author, int decade) => (Author, Decade) = (author, decade);

    public string Author { get; }
    public int Decade { get; }
}

Isaac Asimov, 1950s: Foundation | I, Robot
George Orwell, 1940s: 1984 | Animal Farm
Anonymous type: 8, tuple: 8, class: 10

Anonymous type and tuple keys worked without a problem: Asimov's two books came together in the 1950s, Orwell's two books in the 1940s, for a total of 8 groups. Both are compared by value; two anonymous objects with the same field values are considered equal. Your own DecadeKey class, on the other hand, fell back to reference equality because it doesn't override Equals and GetHashCode, and every book stayed in its own group: 10 groups. The rule we saw in the Dictionary post of the Collections and Generics series applies here too; GroupBy matches keys internally by hashing them. If you're going to write a type for a key, use a record.

ToLookup: An Index Ready for Reading

If you'll access the same groups over and over, building them once with ToLookup makes more sense than running GroupBy again each time:

ILookup<string, Book> byAuthor = Store.Books.ToLookup(b => b.Author);

Console.WriteLine($"Tolkien: {string.Join(", ", byAuthor["J.R.R. Tolkien"].Select(b => b.Title))}");
Console.WriteLine($"Stephen King: {byAuthor["Stephen King"].Count()} books");
Console.WriteLine($"Contains Asimov: {byAuthor.Contains("Isaac Asimov")}, authors: {byAuthor.Count}");

string?[] tags = { "new", null, "sale", null };
Console.WriteLine($"null key: {tags.ToLookup(t => t)[null].Count()}");

Dictionary<string, List<Book>> dictionary = Store.Books
    .GroupBy(b => b.Author)
    .ToDictionary(g => g.Key, g => g.ToList());

try
{
    List<Book> kingBooks = dictionary["Stephen King"];
}
catch (KeyNotFoundException ex)
{
    Console.WriteLine(ex.Message);
}

Tolkien: The Hobbit, The Fellowship of the Ring
Stephen King: 0 books
Contains Asimov: True, authors: 7
null key: 2
The given key 'Stephen King' was not present in the dictionary.

An ILookup<TKey, TElement> resembles a dictionary, but with a few important differences. Its indexer doesn't throw for a missing key, it returns an empty sequence: 0 books for "Stephen King". When we asked a Dictionary<string, List<Book>> the same question, we got a KeyNotFoundException. It also accepts a null key; the two nulls among the tags were collected in their own group. Finally, a lookup can't be changed: it has no Add or Remove method.

In practice you can choose like this: if you group once and walk the results, GroupBy. If you'll access groups by key again and again, say finding each customer's orders inside a loop, ToLookup. If you need a mutable structure you'll add to and remove from later, Dictionary<TKey, List<T>>.

Grouping with a Comparer

In manually entered data, the same genre may be written differently. The default string equality is case-sensitive:

string[] genres = { "Science Fiction", "science fiction", "Fantasy", "SCIENCE FICTION", "fantasy" };

IEnumerable<string> exact = genres.GroupBy(x => x).Select(g => $"{g.Key}: {g.Count()}");
Console.WriteLine(string.Join(" | ", exact));

IEnumerable<string> ignoreCase = genres
    .GroupBy(x => x, StringComparer.OrdinalIgnoreCase)
    .Select(g => $"{g.Key}: {g.Count()}");
Console.WriteLine(string.Join(" | ", ignoreCase));

Science Fiction: 1 | science fiction: 1 | Fantasy: 1 | SCIENCE FICTION: 1 | fantasy: 1
Science Fiction: 3 | Fantasy: 2

The version without a comparer found five separate genres. With StringComparer.OrdinalIgnoreCase, the three science fiction and two fantasy entries came together. A group's Key is the value that opened the group: "Science Fiction". If you want to print a normalized name in a report, normalize it yourself in the key selector, but use culture-independent ToUpperInvariant() instead of ToUpper(); under the Turkish culture the uppercase of "science" becomes "SCİENCE".

How GroupBy Reads the Source

GroupBy is a deferred operator, but it isn't lazy the way Take is. To hand out a group, it must have collected all the elements for that key, and it can only know that once it reaches the end of the source:

int reads = 0;
IEnumerable<IGrouping<string, Book>> groups = Store.Books
    .Where(b =>
    {
        reads++;
        return true;
    })
    .GroupBy(b => b.Genre);

Console.WriteLine($"After defining: {reads}");

IGrouping<string, Book> first = groups.First();
Console.WriteLine($"After First(): {reads} (first group: {first.Key})");

int groupCount = groups.Count();
Console.WriteLine($"After Count(): {reads} ({groupCount} groups)");

After defining: 0
After First(): 10 (first group: Science Fiction)
After Count(): 20 (4 groups)

Nothing was read at definition time. But even when we asked for just the first group, all ten books were read: GroupBy buffers the whole source on the first MoveNext. Then, when Count() was called, the query ran from scratch and the counter went up to 20. This has two consequences. First, on an infinite or very large source, GroupBy pulls all the data into memory. Second, if you'll use the grouped result more than once, pin it down once with ToList() or ToLookup().

In a database things are different: EF Core translates a GroupBy followed by an aggregation into SQL's GROUP BY, and the grouping happens on the server. You'll find the details in the LINQ queries post of the EF Core series. If you're grouping only to count or sum, CountBy and AggregateBy in .NET 9 give the same result without building intermediate groups; we'll see them in the .NET 9 post of the LINQ series.

Common Pitfalls

Keeping a GroupBy result in a variable and assuming the source won't change. The query regroups every time it's enumerated; if you want a fixed snapshot, use ToLookup or ToList.

Using a class without equality as a key. Each instance becomes its own group. Use an anonymous type, a tuple or a record.

Expecting groups to come alphabetically or by size. The order is the order in which keys first appear in the source; if you want another order, add OrderBy.

Forgetting the Dictionary exception when using it like a lookup. GroupBy(...).ToDictionary(...) throws a KeyNotFoundException for a missing key; ILookup returns an empty sequence. Decide which behavior you want.

Grouping manually entered text without a comparer. "Fantasy" and "fantasy" become two separate groups; pass StringComparer.OrdinalIgnoreCase.

Thinking it's cheap to take just the first group. Even GroupBy(...).First() reads the whole source.

A Small Experiment

★☆☆ Group the books by the decade of their publication year and print the number of books for each decade, sorted from oldest to newest.

★★☆ Produce each genre's most expensive book and that genre's total stock value (Price * Stock) with a single query. Then write the same query in query syntax.

★★★ Without using ToLookup, build your own "lookup" with a Dictionary<string, List<Book>> and a foreach. Add a Get method that returns an empty result for a missing key, and check that the order of elements in the groups matches GroupBy.

Grouping split a single sequence into parts. In the next post we move on to the set operations that compare two sequences: Distinct, Union, Intersect, Except, and the *By variants from .NET 6.