Type hints and generics in Python

Why this matters

A list in Python can hold anything. The list ["Meera", 42] is valid, and nothing stops you from adding a number to a list of names. The mistake shows up later, when the code uses the item as the wrong type. In a large program the gap between the bad append and the crash can be hundreds of lines.

Type hints let you write down what a list or a function is meant to hold, such as list[str]. Python does not enforce the hints when the program runs. A separate tool, a type checker such as mypy, reads the hints and reports the places where the code breaks them. This lesson explains that split, shows how generic code is written with TypeVar and Generic, and says what the hints can and cannot catch.

The lessons on lists and dicts in this module covered the containers themselves. This lesson is about the labels you put on them and the checks that happen before or after the program runs.

Start with the problem: a list with no type

Suppose you want a list of names. Python lets you write this:

The output is:

['Meera', 42]
caught: can only concatenate str (not "int") to str

The list holds a number as well as a name, and Python accepts it. The mistake shows up only when the program adds the two items. The message names the two types, str and int, which helps you find the cause, but the message appears far from the append that caused it.

A type hint moves the warning to the place where the mistake is made. The hint list[str] says what the list should hold. A type checker reads the hint and reports the append of the number as an error before the program runs. The program itself still runs the same way, because Python does not enforce the hint.

Step one: write the type in the annotation

A variable can carry a hint after its name, and a function can declare the types of its inputs and its result:

names: list[str] = []
names.append("Meera")
names.append(42)  # a type checker reports this line

The hint list[str] is written with square brackets, the same syntax as a generic list. The code above still runs. Running it does not raise an error, but a type checker reports the append(42) line. The hint is a promise that the checker enforces, and only the checker.

A function hint works in the same way:

def greet(name: str) -> str:
    return "Hello, " + name

The hint name: str says the argument should be a string, and -> str says the result is a string. Python stores these notes in the function's __annotations__ dictionary, and it does not compare them with the real values.

Step two: write your own generic class

A generic class takes a type as a parameter. A common first example is a box that holds one item. The letter T is a placeholder. It stands for "some type that the user of the class will choose". Programmers usually pick a single capital letter: T for type, K for key and V for value. The letter has no special meaning to Python, but the type checker uses it to link the types in a function or class.

The output is:

Meera 8

The expression Box[str]("Meera") tells the type checker that this box holds strings. The expression Box[int](7) tells it the box holds integers. When the program runs, both objects are plain Box objects. Python keeps the subscript only as a note named __orig_class__, and no check reads it. The get method returns whatever was stored. The checker is the part that knows label.get() is a string and count.get() is an integer, so it can warn if you add a string to the integer result.

Step three: the hints are gone at run time

Here is the part that surprises most people who come from a language with generics that are checked at run time. The type inside the brackets is used only by the type checker. At run time, both boxes are the same class.

The output is:

same runtime class: True Box

Both objects are a Box. The program does not keep a separate copy of the class for each type. The __orig_class__ note is set only when a box is made with brackets, such as Box[str]("Meera"), and nothing checks it, so do not use it to decide what a box holds.

Python also refuses the most direct way to ask. A generic type cannot be used in isinstance, because the type argument does not exist at run time:

The output is:

caught: Subscripted generics cannot be used with class and instance checks
caught: isinstance() argument 2 cannot be a parameterized generic

The first message is about your own generic class. The second is about the built-in list[str]. Both say the same thing: the check isinstance can test the outer class, such as list or Box, but it cannot test what the contents are. If you need to know the contents, check each item yourself, or store the item type in a field when you create the object.

Step four: a bound says what the type must support

Sometimes a generic function needs more than "any type". To find the largest item in a list, the items must support the > comparison. A bound names the kind of type that is allowed. It is set on the TypeVar, as in TypeVar("T", bound=...).

The function below works on any list of items that can be compared. This example sets no bound, so a type checker reports the > line inside the function. At run time, Python does the comparison itself and raises an error when it cannot.

The output is:

9 pear
caught: '>' not supported between instances of 'str' and 'int'

The first line works for numbers and for strings, because both support >. The second call mixes a number and a string. The checker reports the > line once, before the program runs, so it does not find the mixed list by itself. At run time, Python raises the TypeError in the middle of the comparison. The message names the operator and both types, so the cause is easy to find.

Step five: read-only and read-write types

Suppose you write a function that adds up the values in a list of numbers. The list might be a list[int] or a list[float]. A parameter declared as list[float] would refuse a list[int] in a type checker, even though every integer is accepted where a float is expected. This is because a list is a changeable container, so a list[int] cannot be treated as a list[float] without the risk that someone adds a float to the integer list.

The type Sequence from collections.abc solves this. A Sequence can be read but not changed, so a function that only reads can accept any sequence of numbers:

The output is:

6.5
4.0

The function accepts a list and a tuple, because both are sequences. A type checker allows the call because the function promises to read only. Inside the function, nums.append(1) would be reported, since a Sequence has no append.

A common rule of thumb follows from this. Use Sequence or Iterable when a function only reads. Use list when the function changes the list. The rule applies to the parameter type, and the checker enforces it.

Step six: when the hints are wrong

A hint can be wrong, and the program will still run. Python does not check it. This is the main limit of type hints.

The output is:

Asha
7

The second call passes a list of numbers into a function that says it takes strings. A type checker reports that call. Python runs it, and the function returns the number 7 even though the hint says str. The hint did not change the result.

Run the type checker when you write code with hints. Without it, the hints are only notes that Python stores. The type checker is a separate program, so it needs to be installed and run as part of your work, the same way you run tests.

Worked example: a generic pair

Here is a class that holds two values of possibly different types, with a method that swaps them. It has two type variables, so the checker can track each type on its own.

The output is:

('Meera', 91)
(91, 'Meera')
92

The method swap returns a Pair[B, A], so the types are reversed. The type checker follows the change: flipped.first is an integer, so adding 1 is correct. If you wrote wrong: Pair[str, int] = score.swap(), the checker would report that the result is a Pair[int, str], not a Pair[str, int]. The quoted name "Pair[B, A]" is a forward reference, which lets a method name the class it is inside.

Worked example: a generic function that takes a test

The next example passes a rule into a generic function. The rule is a function that returns True or False. The type variable links the list's items to the rule's argument, so the checker can see when a rule does not fit the list.

The output is:

Meera
16
None

The function has one type variable T, and the same T appears in three places: the list, the test and the return value. If you passed lambda n: n.startswith("M") with a list of integers, the checker would report it, because an integer has no startswith method. At run time, Python would raise an AttributeError when the test ran on the first integer.

The function returns None when nothing matches. The hint T | None says so. A caller must check for None before using the result, or it fails with AttributeError or TypeError when the code uses the value as if it were an item.

Where the check happens

Many languages check generic code in the compiler, before the program runs. Python does not. Its type hints are checked only when you run a separate type checker. Until then, a mismatched hint runs like any other code, and the error appears later, when the bad value is used.

The type checker and the run-time behaviour agree on one point: a generic class does not check its type argument at run time, as the Box example showed.

Common mistakes

Writing a hint and never running the checker. The hint is only a note until a checker reads it. Run mypy or a similar checker as part of your work, or the hint protects nothing.

Trusting a hint for the contents of a list. list[str] does not stop a number from being added at run time. The checker must see the code, and the run-time error appears later.

Using isinstance with a type argument. isinstance(x, list[str]) raises TypeError, because the contents are not stored at run time. Check the outer type, and then check the items yourself.

Assuming list[object] accepts any list. A list[str] is a changeable list, and a checker refuses to treat it as a list[object], since a function could then add an object to it. Use Sequence[object] for a function that only reads.

Leaving out None in a return hint. A function that can return None should say so with T | None. Otherwise, a caller may use the result without a check.

Writing an annotation for a value that a function changes. A hint on the parameter does not stop the function from changing the list it receives. The caller's list changes too, as in every other Python function.

Practice

Each exercise is followed by its answer. Try to write the answer yourself before reading it.

Exercise 1

Write the type for a list of prices in cents, where each price is a whole number. Write one line of code that creates an empty list of those prices and adds the number 1999 to it.

Show the answer

prices: list[int] = []
prices.append(1999)

The hint list[int] names the item type directly, and the checker reads it.

Exercise 2

Write a generic function first that takes a list and returns its first item. Assume the list is not empty. Call it with a list of strings and print the result.

Show the answer

from typing import TypeVar
 
T = TypeVar("T")
 
 
def first(items: list[T]) -> T:
    return items[0]
 
 
print(first(["Asha", "Ravi"]))

The output is Asha. The same T appears in the parameter and in the return type, so the checker knows the result has the type of the items.

Exercise 3

Your program has the function def print_all(items: list[object]) -> None. A caller passes a list[str], and a type checker reports an error. Explain why, and write a fixed version that accepts any sequence and only reads from it.

Show the answer

A list[str] is not treated as a list[object], because a list can be changed. If the function added an object to the list, the caller's list would end up holding a value of the wrong type. The fix is to change the parameter to a read-only type:

from collections.abc import Sequence
 
 
def print_all(items: Sequence[object]) -> None:
    for item in items:
        print(item)

A Sequence cannot be changed through its name, so the checker accepts a list[str] for it. At run time, Python would have run the original version without complaint. The difference appears only in the checker.

Exercise 4

Explain in plain words why a list of strings and a list of numbers have the same class at run time. Then say what the type checker does with the type in the brackets.

Show the answer

Both values are plain list objects. The brackets tell the type checker what each list should hold, and the checker uses that to check each append and each read. When the program runs, the brackets are not kept on the object, so type(names) is list for both. The running program cannot tell a list of strings from a list of numbers by its class. Only the checker knew the difference, and only before the run.

Exercise 5

Write a generic function is_sorted(items) that returns True when each item is less than or equal to the next. Use a Sequence parameter, so the function only reads. Test it with [1, 2, 2, 5] and ["b", "a"].

Show the answer

from collections.abc import Sequence
from typing import TypeVar
 
T = TypeVar("T")
 
 
def is_sorted(items: Sequence[T]) -> bool:
    for i in range(1, len(items)):
        if items[i - 1] > items[i]:
            return False
    return True
 
 
print(is_sorted([1, 2, 2, 5]))
print(is_sorted(["b", "a"]))

The output is True for the first list and False for the second. The function compares items with >, so the items must support that comparison. A checker can enforce that with a bound, and Python raises a TypeError at run time if a list mixes types that cannot be compared.

Summary

• Type hints such as list[str] describe what a container should hold. Python does not enforce them when the program runs.
• A separate type checker, such as mypy, reads the hints and reports mismatches before the program runs. Run it as part of your work.
• A generic class uses Generic[T], and a generic function uses a TypeVar, so the checker can link the types of the inputs and the result.
• At run time, Box[str] and Box[int] are the same Box class. isinstance cannot test what the contents are.
• Use Sequence for a parameter that only reads, and list for one that changes the list.
• A hint can be wrong and the program will still run. The check is only as good as the checker that reads it.