Grouping with dicts of lists

Why this matters

Many programs need the same kind of answer. Show all orders for each customer. Show all students in each grade. Show all sales for each day. In each case you start with one flat list of records and you want them sorted into bins, one bin per key.

The tool for this is a dict of lists. A dict stores pairs of a key and a value. Here the value for each key is a list. The key might be a customer name, and the list holds that customer's orders. You can ask the dict for a key and get back all the records that belong to it.

This lesson shows how to build that dict in Python, how to avoid the common bug in the process, and how itertools.groupby does a similar job with one rule that you must follow. The earlier lessons covered lists and dicts on their own, and the lesson on type hints covered the labels such as dict[str, list[str]].

Start with the problem: searching the whole list for each group

Suppose you have five students, each with a name and a grade. You want to print the students in each grade. A first attempt scans the whole list once for each grade:

for grade in ["A", "B", "C"]:
    print("Grade", grade + ":")
    for s in students:
        if s.grade == grade:
            print("  ", s.name)

This works for five students. With 100,000 students and 50 grades, the inner loop runs 50 times over the whole list. The total work grows with the product of the number of groups and the number of records. A better method reads the list once and puts each student straight into the right group.

An everyday picture: labelled folders

Think of a filing cabinet with one folder for each customer. The label on each folder is the key. Each folder holds a stack of papers, which is the list of records for that key.

When you pick up a paper, you read its label. You open the folder with that label and add the paper to the top of the stack. If there is no folder with that label yet, you make a new folder first. You do this once for each paper, so after the last paper the cabinet holds every group.

Three steps happen for every record:

1. Read the key from the record.
2. Find the folder for that key. If it does not exist, create it with an empty list.
3. Add the record to the folder's list.

Every dict of lists is built with these three steps, whatever the records are.

Step one: the manual version with get and set

Start with the most basic version, so you can see each step. Each order in this example is a pair of strings: the customer name and the item bought.

The output is:

ana: ['book', 'lamp']
ben: ['pen']

Here is what happens for each order:

• by_customer.get(customer) looks up the customer. If the customer has no entry, get returns None, which means "no value here".
• The if items is None branch handles the first order for a customer. It creates an empty list and stores it in the dict.
• items.append(item) adds the item to the list. It runs for both new and existing customers, because items now refers to the list in the dict.

The result is correct. The code is longer than it needs to be, though, and it has a trap that a later section covers.

Step two: setdefault does the same work in one call

The dict method setdefault combines the check and the creation. You give it the key and the value to use if the key is missing. If the key is already in the dict, the method returns the existing value and does not change it. If the key is missing, it stores your value under the key and returns it.

by_customer.setdefault(customer, []).append(item)

The call setdefault(customer, []) returns the list for that customer, and .append(item) adds the item to it. The whole group-building loop becomes one line.

The collections module has a second tool for the same job. A defaultdict is a dict that creates a missing value by calling a function you give it. defaultdict(list) creates an empty list the first time you read or change a key, so you can append without any check.

Here is a complete program that groups students by grade. It uses a dataclass for the records, and it prints the grades in sorted order.

The output is:

A: [Student(name='Ravi', grade='A'), Student(name='Kiran', grade='A')]
B: [Student(name='Asha', grade='B'), Student(name='Meera', grade='B')]
C: [Student(name='Dev', grade='C')]
groups: 3

The @dataclass(frozen=True) line defines a small class that holds two values and gives them names. Python writes the constructor (the __init__ method that sets the fields), the attribute names and a readable repr (the text that print shows) for you. The frozen=True option makes the fields read-only, which is a good habit for records that are used as data.

Notice the order of the students inside each group. Ravi comes before Kiran, and Asha before Meera, because the loop adds them in the order they appear in the input list. A list keeps insertion order, so the order inside a group matches the order of the input.

Why sort the keys when you print

A plain dict keeps the order in which its keys were first added. The order of the groups in the dict above is B, A, C, because B came first in the input. The grouping code does not depend on that order. The loop for grade in sorted(by_grade) sorts the keys only for printing. Sort whenever the output must have a fixed order, and keep the dict for the lookups.

Step three: a worked example with two levels of grouping

Real data often needs more than one level. Sales belong to a day, and each day's sales belong to a region. A dict of lists can hold the first level. For the second level, the value in the outer dict is another dict, and that inner dict holds lists.

First, here is the one-level version. It groups sales by date and adds up the amount for each date.

The output is:

2026-01-03 count=2 total=200
2026-01-04 count=2 total=75
2026-01-05 count=1 total=200

The loop over by_date reads each group's list and adds up its amounts. The grouping step and the summing step are separate. This split is the usual shape of a grouping program: group first, then work on each group.

Now the two-level version. The outer dict is keyed by date, and each value is another dict keyed by region. The inner dict's values are lists of sales.

by_date_then_region = {}
for sale in sales:
    by_region = by_date_then_region.setdefault(sale.date, {})
    by_region.setdefault(sale.region, []).append(sale)
 
print(by_date_then_region["2026-01-04"]["north"])

The first setdefault makes the date's inner dict if it does not exist. The second one makes the region's list inside that dict. Then append adds the sale to that list. The output is a list of the two north sales for January 4.

A two-level structure is hard to read. Give it a name, or use one tuple key such as (sale.date, sale.region). The custom keys lesson in this module covers tuples and records as dict keys.

The trap: get, then add to the result

Here is a version that many beginners write. It looks shorter than step one, and it runs without an error until the moment it fails.

broken = {}
broken.get("ana").append("book")

The key "ana" is not in the dict. get returns None, and calling append on None fails. The program stops with this message:

AttributeError: 'NoneType' object has no attribute 'append'

The message names the object that was None, and the method that was called on it. The fix is one of the two patterns from the first two steps: check for None and create the list, or use setdefault. Use setdefault by default, because it keeps the create-and-add logic in one place.

A second trap: assignment replaces the list

Another bug happens when you create a new list for every record, even when the key already has one:

The output is:

bad: {'ana': ['lamp'], 'ben': ['pen']}
good: {'ana': ['book', 'lamp'], 'ben': ['pen']}

Assignment stores the value under the key and removes the value that was there before. So each new order for ana throws away the older orders. The bad dict has lost book. The good dict, built with setdefault, keeps both orders, because the list is created only for a key that is not yet in the dict.

A third trap: reading a defaultdict creates the key

A defaultdict creates a missing value when you read it, not only when you change it. That is useful for grouping, but it can surprise you in a lookup. The next example reads a key that was never added:

The output is:

[]
True

The read of by_customer["zzz"] added the key "zzz" with an empty list. The second line now says True. If you only want to look up a key without creating it, use get on the defaultdict, which does not create anything, or check with in first.

itertools.groupby: a grouping that needs sorted input

The standard library has a function named groupby in the itertools module. It walks a sorted list and yields each run of equal keys as a group. It does not build a dict, and it does not sort for you. Its rule is that equal keys must be next to each other. The example below shows what happens when they are not:

The output is:

groupby unsorted: {'B': ['Meera'], 'A': ['Ravi']}
groupby sorted: {'A': ['Ravi'], 'B': ['Asha', 'Meera']}

In the first run, B appears twice in the input, but the two B records are not next to each other. The function makes two separate groups with the key B. The dict comprehension then keeps only the last one, so Asha is lost. Sorting the input first puts the B records together, and the result is correct.

Use groupby when the data is already sorted, or when the sort is cheap and you need the groups in order. Use the dict of lists when the input is in any order, because it does not need a sort at all.

Comparing the approaches

The table compares the three ways to group records in Python.

There is no multimap in the standard library. The dict of lists is the usual replacement. The C++ std::multimap stores each record as a separate entry with a shared key, and Python has no direct match for that. A dict of lists keeps each group in one place, which is usually easier to work with.

Common mistakes

Mistake one: calling append on the result of get

This was shown above. The message 'NoneType' object has no attribute 'append' means the key was not in the dict. Use setdefault or a defaultdict to create the list.

Mistake two: assignment instead of setdefault

Assigning a new list for each record erases the earlier records for that key. Create the list only when the key is missing.

Mistake three: returning the internal list

When a function returns a group, the caller can change the list and therefore change the dict. Return a tuple, which cannot be changed:

def group_for(by_grade, grade):
    return tuple(by_grade.get(grade, []))

If the caller tries to append to the result, Python raises AttributeError. This is the error for a read-only sequence, and it tells you that the change was refused.

Mistake four: expecting a plain dict to print in sorted order

A dict keeps the order of its first insertions, so its printed order depends on the input. If the output must be sorted, sort the keys when you print, as the grade example did.

Mistake five: using a mutable object (one that can be changed) as a key

If a record is used as a dict key and its fields change after it is added, the dict can no longer find it. Use a key that does not change, such as a string, a tuple of strings or a frozen dataclass. The custom keys lesson in this module explains this in full.

Mistake six: reading a defaultdict to check for a key

A read on a defaultdict creates the key. Use in to check for a key, or get to read without creating one.

Practice

Each exercise is followed by its answer. Try each one before reading the answer.

Exercise 1

Group the words apple, avocado, banana, blueberry, cherry and apricot by their first letter. Use a dict from a letter to a list of words. Print the groups in alphabetical order and the size of each group.

Show the answer

def group_by_first_letter(words):
    groups = {}
    for word in words:
        groups.setdefault(word[0], []).append(word)
    return groups
 
 
groups = group_by_first_letter(
    ["apple", "avocado", "banana", "blueberry", "cherry", "apricot"]
)
for letter in sorted(groups):
    print(letter, "->", len(groups[letter]), groups[letter])

It prints:

a -> 3 ['apple', 'avocado', 'apricot']
b -> 2 ['banana', 'blueberry']
c -> 1 ['cherry']

The key is a one-character string. The setdefault call creates each list the first time its letter appears. The loop sorts the letters only for printing.

Exercise 2

Given a dict from grade to a list of students, write a function that returns the grade whose list is the longest. Use Asha, Ravi, Meera, Dev and Kiran with grades B, A, B, B and A. Print the result.

Show the answer

def largest_group_key(groups):
    best_key = None
    best_size = -1
    for key, members in groups.items():
        if len(members) > best_size:
            best_size = len(members)
            best_key = key
    return best_key
 
 
groups = {}
for s in students:
    groups.setdefault(s.grade, []).append(s.name)
print(largest_group_key(groups))

With those five students, grade B has three students and grade A has two. The function returns B. If two groups tie, the function returns the one that comes first in the dict's order. Sort the items first if you need a fixed rule for ties.

Exercise 3

Write the grouping that stores each student's name under their grade, using setdefault. Then write a line that gives the names of the students in grade B as a single string separated by commas, such as Asha, Meera, Dev.

Show the answer

by_grade = {}
for s in students:
    by_grade.setdefault(s.grade, []).append(s.name)
 
print(", ".join(by_grade["B"]))

With the students from exercise 2, grade B holds Asha, Meera and Dev in that order, and the output is:

Asha, Meera, Dev

The method join takes a list of strings and places the separator between each one.

Exercise 4

Explain in plain words what goes wrong if you write groups[key] = [] inside a loop that adds one record per iteration. Then say which method fixes it.

Show the answer: each assignment replaces the list stored under that key with a new, empty list. The records added before are lost. The earlier list is no longer reachable from the dict, so the program forgets it. The method setdefault fixes the problem. It creates a list only when the key is missing, and it returns the existing list otherwise.

Exercise 5

Write code that keeps a dict from a player name to a list of scores. Add a score of 50 for "Ana", then print the list. Make sure the code does not fail when "Ana" is not yet in the dict.

Show the answer

scores = {}
scores.setdefault("Ana", []).append(50)
print(scores["Ana"])

The output is [50]. The setdefault call creates the list for "Ana" because the key is missing. A second call with another score would add to the same list.

Summary

• A dict of lists groups records by a key. Each key's value is the list of records that share it.
• Use setdefault(key, []).append(record) to create a group's list the first time its key appears, and then add the record to it.
• defaultdict(list) does the same job without the explicit default, but reading a missing key creates it.
• Calling get(key).append(...) without a check fails with AttributeError when the key is missing.
• Assigning a new list with groups[key] = [] replaces the earlier list, which loses records.
• itertools.groupby needs its input sorted by the key. Without the sort, equal keys split into separate groups.