Interface design with protocols and small base classes

Why this matters

An interface is a promise. It says, "any object that follows me will have these methods." Every class that follows the promise must keep it, and every piece of code that receives the object depends on it. If the promise is too big, every class that follows it has to write methods it does not need. If the promise has a method nobody calls, you still have to write and test that method, and a future change to it can break code that was never related to the problem. Good interface design is the habit of writing a promise that is small, clear, and true for the people who depend on it.

In Python an interface is usually a typing.Protocol (a class that lists the methods an object must have) or an abstract base class from abc. The lessons on polymorphism and on protocols showed how to write each one. This lesson is about choosing what goes inside one.

The idea: start from the caller

A caller is the code that uses an object. When you design an interface, the best starting point is to look at the caller first and ask: what does this code actually need from the other object?

Here is a small example. A shop has a Checkout class. When an order is paid, it should tell the customer. The first version of Checkout sends an email directly.

class EmailSender:
    def send_email(self, to, text):
        print(f"Email to {to}: {text}")
 
 
class Checkout:
    def __init__(self):
        self._sender = EmailSender()
 
    def complete_order(self, customer, total):
        self._sender.send_email(customer, f"Order paid, total {total}")

This works, but it has a problem. Checkout creates its own EmailSender. A test of complete_order will really try to send email. You cannot run the checkout logic without the email part. If you later want to send an SMS instead, you must edit Checkout.

The fix is to ask a different question. Checkout does not care about email. It only needs to tell a customer something. So the protocol describes that need and nothing more:

from typing import Protocol
 
 
class Notifier(Protocol):
    def notify_customer(self, customer: str, message: str) -> None: ...

Now Checkout receives a notifier from outside, and it no longer knows how the message is delivered. Here is the full program. It has two implementations: one that records messages and one that does nothing.

The output is:

Email to Asha: Order paid, total 450
Email to Ravi: Order paid, total 120
Messages sent: 2
Silent notifier ran without output.

Look at what Checkout can see. It can call one method, notify_customer. It cannot see sent_count, and it does not need to. The protocol is exactly the set of things Checkout uses. A test can pass in the silent notifier, and the checkout logic runs with no email at all. A future SMS class only has to write one method.

The question to ask for every interface is this: "If I remove this method, which caller breaks?" If no caller breaks, the method probably should not be in the interface.

Keep the interface small

A fat interface is an interface with many methods. Fat interfaces grow when one person adds a method for each new feature, and nobody removes methods later. Here is a storage base class that someone wrote to cover everything a database might do.

from abc import ABC, abstractmethod
 
 
class FatStorage(ABC):
    @abstractmethod
    def save(self, key, value): ...
 
    @abstractmethod
    def load(self, key): ...
 
    @abstractmethod
    def delete(self, key): ...
 
    @abstractmethod
    def exists(self, key): ...
 
    @abstractmethod
    def count(self): ...
 
    @abstractmethod
    def rename(self, old_key, new_key): ...
 
    @abstractmethod
    def backup(self, path): ...
 
    @abstractmethod
    def compact(self): ...

Now suppose a score board only needs to store a score under a name and read it back. It needs two methods. The fat base asks for eight. Look at what happens when a test wants a test double, which is a fake object that stands in for the real one. A test double for FatStorage must write every abstract method before Python lets you create it. Here is an attempt that writes only two of them:

The program stops with an error. The last line of the output is:

TypeError: Can't instantiate abstract class TestStorage without an implementation for abstract methods 'backup', 'compact', 'count', 'delete', 'exists', 'rename'

Python refuses to create the object, and the message lists the six methods that are still missing. The test double must be complete before Python creates it, even though this test uses only two of the methods. The same test double written against a protocol would need no inheritance at all, and it would only need the methods the code under test calls. The next example shows that.

A smaller protocol solves the problem. The score board needs a put and a get, so the protocol has exactly those two:

The output is:

Asha: 55
Ravi: 0

MapStore has two methods, and each one is a line long. The score board works, and a test can write a second implementation in minutes. The method score_of returns 0 for a player who has no score yet. The check value is not None handles the missing case, because get returns None when a key is not stored.

Notice that ScoreBoard also does not know it is talking to a dictionary. A later version could store scores in a file or a database. The protocol stays the same, so ScoreBoard does not change.

Do not add a method nobody calls

The fat storage base had a compact method. Compacting means rearranging storage to free space. Maybe it was added because someone thought storage might need it one day. Nothing calls it.

This kind of method costs more than it looks like it does. Every class that inherits from the base must write it. Every test double must write it. Someone reading the code has to wonder what it is for. If the method has a bug, nobody notices, because nothing runs it.

The rule is simple: add a method to an interface when a caller needs it today, not when you think it might be useful. Adding a method later is easy when you have a real caller. Removing one that everyone implements is hard.

You can check whether a method has callers. In an editor, the "find usages" command lists every place the name is used. On the command line, grep searches text in files and prints each line that contains the words you give it. Here is a search for compact( in a small file, main.py, where the method is declared on line 9:

$ grep -rn "compact(" main.py
9:    def compact(self): ...

Only one line matched, and it is the declaration. No caller exists in that file. In a real project you would search the whole source folder, and the same result would tell you the method is unused. The -r flag means search folders recursively, and -n prints line numbers.

Use the standard interfaces first

Python already has many small interfaces in its standard library. Before you write a new one, check whether a standard one fits. The most common are:

• typing.Callable, for anything that can be called with arguments and returns a value. A plain function fits it, and so does a lambda.
• collections.abc.Iterable, for anything you can loop over with a for loop.
• collections.abc.Sized, for anything that len works on.
• Comparison methods such as __lt__, for objects that have a natural order, as the operator overloading lesson showed.
• A function that returns True or False for one value. Java calls this a predicate. In Python it needs no name, because any function that returns a boolean works where a predicate is expected.

Reusing a standard interface has a big advantage. Other programmers already know it, and many library functions accept it without extra work. The built-in filter accepts any function that returns a boolean, for example.

The example below builds a small rule system. Java interfaces can have default methods, which are method bodies that come with the interface. This lesson keeps the combining methods in an ordinary class, because the protocol only needs passes.

Worked example 3: a rule with combinable parts

A shop wants to pick orders by rules. A rule is a yes or no test on one order. A rule could be "the amount is 500 or more" or "the order is express." The shop also wants to combine rules: "large and express" or "not large."

The rule needs one method, passes, that answers yes or no for one item. Python lets a rule be any object with that method. The class below wraps a plain function, and it adds the three ways to combine rules. The operators &, | and ~ mean "and", "or" and "not", and each one builds a new rule. Each combination is written once, in this class.

The output is:

Large: ['B2', 'D4']
Large and express: ['D4']
Not large: ['A1', 'C3']
Large or express: ['A1', 'B2', 'C3', 'D4']

Look at pick. It takes a rule and calls exactly one method on it, passes. It does not know whether the rule is a simple test or a combination of two rules. The caller needs one method, so the protocol that pick depends on has one method.

The class Order is a frozen dataclass, a short way to write a class that only holds data. It was covered in the immutable objects lesson of this module.

Compare this with a version without the combining methods. Each place that wants to combine rules would write its own loop or its own if, and the logic "both must pass" would be repeated. Putting & in the class means the combining logic is written once, next to the rule it belongs to.

The three combining methods work for any test, so they fit the rule class. A method that needs to know about one kind of order, such as its express flag, does not belong on the general rule. Keep shared behaviour to what every rule can follow without extra knowledge.

Common mistakes

Writing a fat interface. The TestStorage example above shows the usual symptom. A class that inherits from an abstract base must write every abstract method, or Python refuses to create it. A test double must write them too. The fix is to split the interface by what each caller needs, or to pass each caller only the part it uses.

Calling a method that the declared type does not have. A type checker reports a call to a method that the declared type does not promise. Python itself runs the call if the object has the method, so the mistake appears only in the type checker, not at run time. The example below shows it:

The output is 1. Python runs the program, because the object has a count method. A type checker such as mypy reports an error on the line s.count(), because the variable is declared as Storage, and Storage does not promise count. You have two choices. If callers need count, add it to the base class or the protocol. If they do not, you do not need it in the base class at all.

Writing a method with the wrong parameters. A subclass that changes the parameter list of an inherited method does not fail when the class is written. The failure appears when someone calls the method with the old arguments:

The program stops with an error. The last line of the output is:

TypeError: Square.area() missing 1 required positional argument: 'scale'

The message names the method and the missing parameter. A type checker reports the mismatch earlier, when it reads the class. Keep the parameters the same as the base class when you override a method.

Inheriting two methods with the same name. When a class inherits from two bases that both define a method with the same name, Python does not refuse to build the class. It uses the first one in the method resolution order, which the inheritance lesson explained. The next example shows it:

The output is left. The result comes from the order of the bases in the class line, not from a rule about the method. If the choice matters, write a method in the class that chooses, for example def name(self): return Right.name(self). A clear design avoids the question, by keeping the names of the bases different.

Adding a method to a base class that many classes already inherit. Every subclass gets the new method at once, and any subclass that defines a method with the same name can change behaviour without warning. A method with a default body avoids the missing-method error, but only if the default behaviour is safe for every subclass. When you are not sure, leave the base class alone and add a new small protocol for the new need.

Practice

Exercise 1

A Printer base class has five methods: print_line, print_duplex, print_color, scan and fax. A class ReceiptPrinter prints one line of text at a time, and it is the only caller that uses this base class. Which methods should the base class keep?

Show the answer

Keep only print_line. ReceiptPrinter only prints lines, so the other four methods are not needed by the caller. Remove print_duplex, print_color, scan and fax from the base class. A printer that can scan can have its own small protocol, used by callers that scan.

Exercise 2

A ReportService calls find_all() on a repository to build a monthly report. The repository class also has find_by_id, delete and count_by_status, which nothing calls yet. Write the smallest protocol ReportService can use.

Show the answer

The protocol has one method, the one the caller uses. The other three methods are left out. They can be added when a caller needs them. The fake repository needs only find_all, which is all the test uses.

Exercise 3

Use the | operator of the Rule class from the worked example, where a | b passes when either rule passes, to keep the numbers from the list 3, 12, 15, 8 where the number is below 10 or even, and print the result.

Show the answer

The number 15 is not below 10 and is odd, so it fails both rules. The program prints [3, 12, 8].

Exercise 4

A class MemoryStorage has a method count() that is not in the Storage base class. Code declares s: Storage = MemoryStorage() and then calls s.count(). A type checker reports Storage has no attribute count. Give two ways to fix it, and say when to pick each.

Show the answer

The first way is to add count to the Storage base class. Pick this when callers that use Storage need the count. Every subclass then has to write count, so keep the base class small with this in mind.

The second way is to declare the variable as s: MemoryStorage = MemoryStorage(). Pick this when only this one caller needs the count and the code does not need to work with other storage types.

Exercise 5

A Logger base class has two methods: log(message) and flush(). Callers only use log. Nothing in the project calls flush. Should flush stay? Explain how you would check, and what you would do if it has no callers.

Show the answer

No, flush should not stay if nothing calls it. Check by searching the source folder for the word flush, for example with grep -rn "flush(" ., or by using the editor's find usages command on the method. If only the definition and the subclass versions show up, no caller exists. Remove the method from the base class and from every subclass. If the method is needed later, add it back at that time, with a caller that uses it.

Summary

• An interface is a promise, so it should contain only the methods that its callers use.
• Start from the caller: ask which methods the calling code needs, and write the protocol from those answers.
• A fat base class forces every subclass and every test double to write methods they do not need.
• Do not add a method for a future need that has no caller today. Check for callers with an editor search or grep.
• Look at standard interfaces such as Callable, Iterable and the comparison methods before writing your own.
• Put shared combining behaviour in a base class only when every subclass can follow it without extra knowledge.