Protocols, abstract interfaces, class attributes and constants in Python

Why it matters

Inheritance gives a class one parent. That is not enough for real designs. A duck can fly and swim. A product can be sold and shipped. A list that sorts must know how to compare two items, and a class that prints must know how to describe itself. Python handles these extra abilities in two ways. A class can inherit from several small base classes, and a function can accept any object that has the right methods, with no inheritance at all. Python also has class attributes, which belong to the class instead of one object, and constants, which are values that should not change. This lesson covers all of these tools, and it shows where Python's rules differ from Java's.

The idea from first principles

Suppose a warehouse has a rule: every item that goes on a shelf must tell its weight. The rule does not care what the item is. A box, a barrel and a bag all have to answer the question "what is your weight?". The warehouse manager does not need to know how each item stores its weight. The manager needs only the promise.

An interface is that promise written in code. It lists method names and what they return, without a body, and says what any object must provide if it claims to follow the rule. In Python you can write the promise in two ways:

• A protocol is a promise that any object can meet, with no inheritance. If an object has a weight() method, it follows the rule. The class does not need to name the protocol. The typing.Protocol class lets you write the promise down so that type checkers can check it.
• An abstract base class is a promise that a class names explicitly by inheriting from it. The abc.ABC class checks at run time that the child provides each promised method.

Compare this with an abstract class in the previous lesson. An abstract base class says "here is what a shape is, and here is code that every shape shares". A protocol or an interface says "here is a job you can do". A shape is a thing. Payable is a job. A class is one kind of thing, so it has one main base. A class can have many jobs, so it can follow many protocols and inherit from several small bases.

The everyday comparison is a plug. A kettle and a lamp have nothing else in common, but both have a plug that fits the same wall socket. The socket is the promise, and the plug shape is the interface.

A protocol: any object with the right methods

A protocol names the methods that a function needs. Any class that has those methods works, whether or not it names the protocol. The next program has a function pay_all that needs an object with an amount_due method and a receipt method. Two unrelated classes meet the promise, and neither one inherits from anything.

The output is:

due 120
subscription 3 months
total 270

Trace it:

1. Payable says every payable thing has an amount_due method and a receipt method. The three dots after each def mean "no body here". A protocol lists only the promise.
2. Invoice and Subscription do not inherit from Payable. They have the two methods, so they follow the promise.
3. pay_all prints each receipt and adds each amount. It works with any object that follows the promise.
4. The total is 120 plus 150, which is 270.

The annotation list[Payable] is a type hint. It tells a reader and a type checker what goes in the list. Python does not check hints when the code runs, so the call works the same without them. The type checker mypy reports a mistake if you pass an object that lacks one of the methods.

A protocol protects the function, not the object. The function uses only the methods the protocol names. A Subscription has a months attribute, but pay_all cannot see it through the protocol, because the protocol does not promise it.

Checking a protocol at run time

By default isinstance does not work with a protocol. Add the decorator @runtime_checkable (a line starting with @ that changes the class below it) if you need the check at run time. It then tests whether the object has each promised method.

The output is:

True False

The duck has a fly method, so it counts as a flyer. The rock does not.

An abstract base class: a promise with a shared default

A protocol cannot provide code. If you want a default method that every implementer gets, use an abstract base class instead. A method in the base class that has a body is a default method. The children inherit it, and they may override it.

The next example uses ABC and @abstractmethod. The base class promises amount_due, and it gives a default receipt. The second class overrides receipt, and the first does not.

The output is:

due 120
subscription 3 months
total 270

The result matches the protocol example. The difference is where the rule lives. Here Invoice gets receipt from the base class, so it does not write one. Use a protocol when you only need to describe a shape of object. Use an abstract base class when the children should share code.

The base class refuses to create an object that is missing an abstract method, as the previous lesson showed. A protocol does not, because a protocol is never created.

Several abilities at once

A class can inherit from several abstract base classes, one per ability. Python calls this multiple inheritance. The next example makes a duck that flies and swims, and calls it through each base.

The output is:

duck flies low
duck swims
duck flies low
duck swims
True True

The duck is one object. The variable f is typed as a Flyer, and s is typed as a Swimmer. Both point to the same duck. A function that needs a flyer can take a Flyer, and it does not care that the object also swims.

Python allows a class to inherit from several bases, but it still has one ordered list of places to look for a method. The list is the method resolution order, which the inheritance lesson showed with __mro__. For small designs with unrelated abilities, the order rarely matters. Keep the bases small, so the order is easy to read.

Sortable objects: __lt__ and key=

In Java a class implements Comparable so that a sort knows how to order its items. Python does the same thing with special methods. A class that defines __lt__, which means "less than", can be sorted with sorted and list.sort, because those functions compare items with <.

The next program reads a list of books and sorts them by year, then by title. The rule is written once, in __lt__. The comparison uses a tuple, which Python compares one item at a time from left to right.

Type this into the program's input:

4
Dune 1965
Emma 1815
Ants 1965
Atlas 1980

class Book:
    def __init__(self, title, year):
        self.title = title
        self.year = year
 
    def __lt__(self, other):
        return (self.year, self.title) < (other.year, other.title)
 
    def text(self):
        return f"{self.title} {self.year}"
 
 
n = int(input())
books = []
for _ in range(n):
    title, year = input().split()
    books.append(Book(title, int(year)))
 
for b in sorted(books):
    print(b.text())

The output is:

Emma 1815
Ants 1965
Dune 1965
Atlas 1980

The method __lt__ returns True when self comes before other. The tuples (1965, "Ants") and (1965, "Dune") are compared the same way the sort would order them: first by year, and when the years are equal, by title. Two books from 1965 come out as Ants and then Dune, because A comes before D.

In Java you would write year - other.year, and you would have to watch for overflow. Python integers have no fixed size, so a subtraction cannot overflow. The tuple comparison is still the clearest way to write a rule with several parts.

Python also has functools.total_ordering, which fills in the other comparison methods when you write one of them, such as __lt__. This is a convenience, and the tuple form above is enough for this lesson.

A key function: a rule outside the class

Sometimes you want to sort by a different rule, without changing the class. Java uses a Comparator object for this. Python uses a key function, which returns the value to compare. The key argument takes any function that accepts one item.

The output is:

['Ants', 'Dune', 'Emma']

The lambda is a short function with no name. It takes one book and returns its title. The sort compares the titles, and the class Book did not change. The natural order stays with __lt__, if you wrote it, and callers add other orders when they need them.

Functions as values: lambdas and callables

In Java a functional interface is an interface with one method, and a lambda fills it in. Python does not need an interface for this. Functions are values. You can store one in a variable, pass it to another function, and call it later. A lambda is a short way to write a function with no name.

The next program uses three lambdas for three different operations. A function takes one of them as an argument and calls it.

The output is:

7
12
4
42

Each lambda is a complete rule for the operation. add returns a + b, which for 3 and 4 is 7. mul returns a * b, which is 12. mx returns the larger value, which is 4. The run function receives mul as a value and calls it with 6 and 7, so the answer is 42. The call op(x, y) works for any function that takes two arguments.

A lambda body is a single expression. Use a def for anything longer. The lambda above writes the conditional form a if a > b else b, which is the same idea as Java's ?: operator.

The function run does not check the type of op. It needs only that op can be called with two arguments. This is the same as the protocol idea: the promise is the call, not a class name.

Class attributes: values that belong to the class

Until now every attribute you have written belonged to an object. A Dog object has its own name. Sometimes a value belongs to the whole class instead. In Python you write such a value in the class body, outside __init__. It is stored once, on the class, and every object reads the same value.

A useful everyday picture: a company has one office address, and every employee shares it. A class attribute is like that address. An instance attribute is like an employee's personal phone number, which each employee has separately.

The next program gives every ticket a unique number. The counter that makes the numbers is one class attribute, shared by all tickets. A class method reports how many were issued. A static method holds a helper that needs no state at all.

The output is:

1 Ana
3 Cy
issued 3
10
0
7

Here is what happens:

1. _next_id is written in the class body, so it is stored once, on Ticket. It is not part of any ticket.
2. Ticket("Ana") reads Ticket._next_id (1) into the new ticket's id, then adds 1 to it, which becomes 2. Ana gets number 1.
3. Ticket("Ben") gets 2, and the class attribute becomes 3. Ticket("Cy") gets 3, and it becomes 4.
4. Ticket.issued() is a class method. It receives the class as cls and returns cls._next_id - 1, which is 3.
5. MathUtil.clamp is a static method. It needs no object and no class, so it is called through the class name. The first call returns 10 because 15 is above the high limit. The second returns 0 because -4 is below the low limit. The third returns 7, which is inside the range.

The underscore in _next_id is only a convention, as the encapsulation lesson showed. It tells other code that the counter is internal.

A class with only static methods, such as MathUtil, is often better written as a plain module with functions. The class version is shown here because Java programmers expect it. In Python a module is a file, and the functions in it are called as mathutil.clamp(...) after an import.

Rules for class attributes, static and class methods

A static method receives no self, so it cannot read or change an object's attributes. A class method receives the class as cls, so it can read and change class attributes. An instance method receives the object as self. Use the first kind that gives you the access you need, and no more.

A class attribute is shared by every object of the class. Changing it through the class changes it for all objects. This is right for counters and constants, and dangerous for anything that should be separate per object.

The next example shows the most common trap. It is a counter that looks like it works, and does not.

The output is:

0 1 1

The line self.count += 1 reads the class attribute, adds 1, and then stores the result on the object as a new instance attribute. The class attribute is never changed, so Bad.count stays 0. Each object has its own count, which is 1. To change the class attribute, write the class name: Bad.count += 1. The Ticket example above does this with Ticket._next_id += 1.

You can also call a class method through an object, such as t.issued(). It works, but it is misleading, because the object is not used. Write the class name instead.

Constants: values that should not change

Python has no keyword that makes a variable constant. The usual convention is a name in capital letters, such as MAX_USERS, which tells readers the value should not change. The type checker mypy can check the convention for you if you add the annotation Final.

The output is:

100 5
5

The program runs, and the final assignment changes the value, because Python does not stop it. A type checker reports the line MAX_USERS = 5 as an error, because it sees the Final annotation. This is the same honest limit as the other conventions in this module. Python trusts the programmer and the tools around the code.

For a value that truly cannot change, use an immutable type such as a tuple, a string or a number. A tuple cannot be changed item by item, so t[0] = 9 raises an error. The next example shows it:

The output is:

TypeError: 'tuple' object does not support item assignment

A Final name still points to the object it was given. If the object is a list, the list can change, even though the name cannot be reassigned. Final protects the name, not the object behind it. This is the same point the encapsulation lesson made about final lists in Java.

Choosing between a protocol, an abstract base class and a plain class

With all these tools, a decision comes up often. Here is a simple way to think about it.

• Use a protocol when you want to describe the methods a function needs, and the objects that come to it may have no common parent. A function that prints any document, from any source, is a good example.
• Use an abstract base class when several related classes share code and a common identity. Shapes, animals and reports fit this pattern. Use several small abstract bases when a class has several separate jobs, such as flying and swimming.
• Use a plain class when the thing has one clear meaning and no shared promise. A ticket, a book or a bank account often needs no base class.

Protocols are usually the better default for designs that will change. They keep classes loosely connected. An abstract base class ties its children to one family tree. Many experienced Python developers say to depend on the behaviour an object has, not on the class it comes from.

Keeping an interface small

An interface should describe one job. A Payable protocol with an amount and a receipt is a clear job. A protocol with twelve methods covering payment, refund, printing, tax and email is hard to satisfy, because every class must provide all twelve, even those that need only two. Programmers call this an interface that is too fat.

A good habit is to split a large protocol into several small ones. A class that needs printing follows Printable, and a class that needs saving follows Savable. A class can follow both. A function that prints a document takes a Printable, and it does not need to know about saving at all. This also makes tests simpler, because a test can pass a small fake object with only the one method under test.

The same idea applies to the methods you put in a class. A class with many public methods exposes many promises. Keep the public part small, as the encapsulation lesson said. A protocol is a public promise too, so it deserves the same care.

Choosing a class attribute, an instance attribute or a static method

A quick guide helps. If the value or method depends on one object's data, make it an instance attribute or an instance method. A dog's name and a dog's bark belong to one dog. If the value is the same for every object, such as a fixed limit, make it a class attribute. If the function needs no object and no class, make it a static method, or better, a plain function in the module.

Be careful with a class attribute that tracks individual objects. The counter is shared, so every object sees the same value. That is right for counting how many objects exist, and wrong for data that belongs to one object, such as a score. If you write a class attribute and then find that every object shows the same value, check this first.

Static methods are also harder to replace. Code that calls Temperature.to_fahrenheit by name is tied to that one function. If the behaviour might change, or must be replaced in tests, an instance method on an object that you pass around is more flexible. Use static methods for pure helpers with no state, and use objects for everything with changing or swappable behaviour.

Common mistakes

Writing self.count += 1 for a class attribute. The line creates a new attribute on the object, and the class attribute never changes. Write Bad.count += 1 or cls.count += 1. The example in this lesson shows the result.

Forgetting to implement a promised method. An abstract base class refuses to create an object that is missing an abstract method, with a TypeError that names the method. A protocol does not check this when the code runs. A type checker reports it.

Using year - other.year in __lt__. This works in Python, because integers never overflow, but it does not handle the second rule. Use a tuple such as (self.year, self.title) < (other.year, other.title).

Expecting isinstance to work with a protocol. It works only when the protocol is marked @runtime_checkable. Without the decorator, Python raises TypeError.

Calling an instance method from a static method. A static method has no self. Calling self.something() inside it raises NameError, because self is not defined there. Make the method a class method or an instance method, or create an object first and call the method on it.

Thinking Final makes an object unchangeable. It makes the name unchangeable, as far as a type checker is concerned. The object behind the name can still change if it is a list. Use a tuple for a value that must not change.

Confusing __lt__ and key=. __lt__ is one natural order built into the class. key= is a separate rule you pass in when you want a different order.

Debugging walkthrough: a sort that does not sort

Suppose you add __lt__ to a class, call sorted, and the list comes out unchanged or in an odd order. Check these things in order.

1. Check that __lt__ is defined on the class, with that exact name and two underscores on each side. A misspelled name such as __lessthan__ is ignored, and sorted then fails with TypeError: '<' not supported between instances of 'Book' and 'Book'.
2. Read the body of __lt__. It must return True when self should come first. A common error is to return True for the reverse order, which sorts the list backwards.
3. Test the method with two objects by hand. Print a < b for a pair you know the order of. True means a comes first.
4. If you pass key=, print the key for each item. A key that returns the same value for every item leaves the order unchanged.

When the answer is still unclear, print the list before and after the sort, and print the key or the comparison for each pair that matters.

Variations and edge cases

A protocol can extend another protocol, in the same way a class extends a class. The child protocol has all the methods of the parent, and it may add more. A class that meets the child must provide all of them.

A base class can have a @staticmethod or @classmethod that belongs to the interface. This is a good place for helper code that works on any object of the family.

If two bases define a default method with the same name, the class that inherits from both gets the one that comes first in the method resolution order. The class can choose one by calling it explicitly, as in Flyer.describe(self), or by writing its own version.

A nested class is a class written inside another class. It does not need an object of the outer class. You will see this in the data structures modules, where a Node class sits inside a list class.

Edge case: a None object passed to a function that expects an interface. Calling a method on None raises AttributeError. Check for None if the caller might pass one.

Edge case: sorting with None values. None does not support < with other values, so the sort raises TypeError. Filter out None values first, or handle them in the key function.

Interview-style questions

What is an interface in Python? A set of method promises. In Python it is written as a typing.Protocol, which any object can meet without inheritance, or as an abc.ABC class, which a child inherits from. A class can follow several.

How is a protocol different from an abstract base class? A protocol needs no inheritance, and it provides no code. An abstract base class is inherited, can provide default methods, and refuses to create a child that is missing an abstract method.

What is a default method? A method in a base class with a body, which children inherit and may override.

What is a class attribute? A value stored on the class, shared by every object of the class. An instance attribute is stored on one object.

What is the difference between a class method and a static method? A class method receives the class as its first argument and can read and change class attributes. A static method receives nothing extra, so it cannot read the class or the object.

What does Final do at run time? Nothing. It is read by type checkers, which report a reassignment. The name is only a promise to the reader.

Why is __lt__ useful? It lets sorted, min, max and sorted containers order your objects without extra code.

Real-world uses

Python's standard library uses protocols and base classes everywhere. A file object and a string buffer (io.StringIO) both provide read, so a function that reads data works with either. Web frameworks use base classes to let you plug in your own logger, your own data source or your own security check. A game might use a protocol such as Damageable, so that a wall, a monster and a crate can all take damage through one call.

Class attributes appear in configuration constants, in counters, and in the registries that frameworks keep. Static methods and module-level functions hold utility code such as unit conversions. Constants with capital letters, and Final annotations, mark values that the rest of the code relies on.

Practice

Attempt each exercise before opening its answer.

Exercise 1: a Speaker protocol (easy)

Write a protocol Speaker with one method speak() that returns text. Write two classes, Dog and Robot, that have the method, without inheriting from Speaker. Dog returns woof and Robot returns beep. Put one of each in a list and print what each says.

Expected output:

woof
beep

Show the answer

The two classes share no parent. They share only the promise, which is enough for the list and the loop.

Exercise 2: a default method (easy)

Write an abstract base class Speaker with an abstract method speak() and a default method loud() that returns the result of speak() in capital letters, using upper(). Write Dog as a child of Speaker. Call loud() on a Dog object and print it.

Expected output:

WOOF

Show the answer

loud is written once in the base class. Every child gets it, and it calls that child's own speak.

Exercise 3: why does it fail? (easy)

The code below stops with an error. Read the message and fix it.

from abc import ABC, abstractmethod
 
 
class Greeter(ABC):
    @abstractmethod
    def greet(self):
        ...
 
 
class Friendly(Greeter):
    def greet_people(self):
        print("hello")
 
 
f = Friendly()

Show the answer

Friendly inherits the abstract method greet and does not write it. The message is TypeError: Can't instantiate abstract class Friendly without an implementation for abstract method 'greet'. The method in Friendly has the wrong name, greet_people. Rename it to greet, so the class provides the promised method. This replaces the Friendly class from the question, and Greeter stays as it was:

# Greeter is the abstract base class from the question.
class Friendly(Greeter):
    def greet(self):
        print("hello")

Exercise 4: what does it print? (easy)

Predict the output, then check.

class Counter:
    total = 0
 
    def __init__(self):
        Counter.total = Counter.total + 1
 
 
a = Counter()
b = Counter()
print(Counter.total)
print(a is b)

Show the answer

It prints 2, then False. total is a class attribute, so both constructor calls add to the same value. a and b are different objects, so a is b is false.

Exercise 5: sort words by length (medium)

Read n and then n words. Order the words by length first, and by alphabetical order when the lengths match. Print the sorted words, one per line. Put the rule in a class Word with a __lt__ method.

Input:

4
pear
fig
apple
kiwi

Expected output:

fig
kiwi
pear
apple

Show the answer

class Word:
    def __init__(self, text):
        self.text = text
 
    def __lt__(self, other):
        return (len(self.text), self.text) < (len(other.text), other.text)
 
 
n = int(input())
words = [Word(input()) for _ in range(n)]
for w in sorted(words):
    print(w.text)

The lengths are 4, 3, 5 and 4. Sorted by length: fig (3), then kiwi and pear (both 4, alphabetical order keeps kiwi first), then apple (5). The tuple (len(text), text) puts the length first and the text second.

Exercise 6: a key function (medium)

Using the same words as exercise 5, but with no __lt__ method, sort them alphabetically with key=. Print them.

Input is the same as exercise 5. Expected output:

apple
fig
kiwi
pear

Show the answer

class Word:
    def __init__(self, text):
        self.text = text
 
 
n = int(input())
words = [Word(input()) for _ in range(n)]
words.sort(key=lambda w: w.text)
for w in words:
    print(w.text)

The key function is separate from the class, so Word has no natural order at all. The lambda supplies the rule at the point where the sort happens.

Exercise 7: count instances (medium)

Write a class Player with a class attribute count that goes up by one in __init__, and a class method how_many() that returns it. Create three players, then print the count.

Expected output:

3

Show the answer

The counter is a class attribute, so it belongs to the class. Each __init__ adds to the same number. The method reads cls.count, which is the same value. The line Player.count += 1 changes the class attribute. Writing self.count += 1 would create a new attribute on each object instead.

Exercise 8: a constant and a limit (medium)

Write a class Cart with a class constant LIMIT = 3. Add a method add() that returns True and adds one item if the count is below the limit, and returns False otherwise. Call add() five times and print how many returned True.

Expected output:

3

Show the answer

The constant is read through the class name inside the method. The first three calls return True, and the last two return False.

Exercise 9: a static helper with no state (medium)

Write a class Temperature with a static method to_fahrenheit(celsius) that returns celsius * 9 / 5 + 32. Print the Fahrenheit value for 100 Celsius.

Expected output:

212.0

Show the answer

The method is static, so it is called on the class name and needs no object. Python's / always gives a float, so the result prints as 212.0. Use // if you need a whole number.

Exercise 10: a lambda rule (hard)

Write a function count_passing(check, numbers) that returns how many numbers pass check. In the main part of the program, pass a lambda that accepts even numbers, and a second lambda that accepts numbers above 10. Count how many numbers from 1 to 15 pass each check, and print both counts on separate lines.

Expected output:

7
5

Show the answer

The even numbers from 1 to 15 are 2, 4, 6, 8, 10, 12, 14, which is 7. The numbers above 10 are 11, 12, 13, 14, 15, which is 5. The function takes behaviour as a value, which is the same idea as the key function.

Exercise 11: a dangerous comparison (hard)

The class below is supposed to sort scores from low to high. Explain what goes wrong when the scores are compared with subtraction in another language with fixed-size integers, and say whether the same problem exists here. Then write a __lt__ that compares the scores.

class Score:
    def __init__(self, value):
        self.value = value

Show the answer

In a language with fixed-size integers, the subtraction a - b can overflow when the numbers are very large, and the result can wrap around to a negative number. The sort would then think the larger score comes first. Python integers have no fixed size, so subtraction never overflows here. The problem still matters when a subtraction is used to compare objects with several parts, because the result is hard to read. The clearest version compares the values directly:

class Score:
    def __init__(self, value):
        self.value = value
 
    def __lt__(self, other):
        return self.value < other.value

With this method, sorted orders the scores from low to high.

Exercise 12: a shared field versus a per-object field (hard)

A student writes a Player class whose __init__ stores the score with Player.score = score, which writes to the class instead of the object. They create two players, a = Player(5) and b = Player(9), and then print a.score. The output is 9, not 5. Explain why, and fix the class so that each player keeps its own score.

Show the answer

Player.score = score stores the value on the class, so there is only one score, shared by every player. The second call overwrites it with 9, and a.score reads that same shared value. The fix is to write self.score = score, which stores the value on the object:

class Player:
    def __init__(self, score):
        self.score = score
 
 
a = Player(5)
b = Player(9)
print(a.score)

This prints 5. Keep a class attribute for values that belong to the whole class, such as a counter of players.

Summary

• A protocol describes the methods a function needs, and any object with those methods meets it, without inheritance.
• An abstract base class is inherited, can provide default methods, and refuses to create a child that is missing an abstract method.
• A class can inherit from several small bases, one per ability.
• A class with __lt__ can be sorted, and key= sorts by any rule you pass in.
• A lambda is a short unnamed function, and any function can be passed as a value.
• A class attribute is shared by every object. Change it through the class name, not through self.
• Final and capital-letter names mark constants for readers and type checkers. Python does not stop a reassignment at run time.