Static locals and counters in Python

Why this matters

Many programs need to remember something between two calls to the same function. A login system counts failed attempts. A ticket machine hands out the next number. A function that is slow to compute can save its answer, so the next call returns at once.

A function that forgets everything when it ends cannot do these jobs. A function that remembers too much, or remembers in the wrong place, causes bugs that are hard to find. This lesson shows the tools Python gives you for this job, and the mistakes beginners make when they first use them.

Readers who know Java or C++ will look for a static keyword here. Python does not have one. The section on static locals shows what to use instead.

Words used in this lesson

A local variable is a name created inside a function. It is created when the function starts and thrown away when the function ends.

A module-level variable is a name assigned at the top of a file, outside every function. It lives as long as the program runs. Other languages call this a global variable.

The global statement is the word global followed by one or more names, written inside a function. It tells Python that a name refers to the module-level variable, so an assignment changes that variable instead of creating a new local.

An attribute is a value stored on an object or on a class, reached with a dot, such as counter.count.

A closure is a function defined inside another function that uses a name from the outer function. The inner function keeps access to that name after the outer function has returned.

A counter is a variable that goes up by one each time something happens.

The problem: a local variable forgets

Start with a small example that shows what goes wrong. The function below is meant to count how many times it has been called. It has a local variable called local_count, which starts at zero.

Output:

Hello Ana (local count 1)
Hello Ravi (local count 1)
Hello Mei (local count 1)

The count is always 1. Each call starts a new run of the function. The line local_count = 0 creates a new variable with the value zero. The next line adds one. Then the function ends, and the variable is thrown away. The next call creates a brand new variable, starts it at zero again, and throws it away again.

The variable works as Python designs it to work. It cannot keep anything from one call to the next. The fix is to move the counter out of the function, into a place that lives as long as the program does.

A module-level variable keeps its value

A module-level variable is written at the top of the file, outside every function. It is created once, when the file runs. Here is the same counter written at the top of the file:

Output:

Hello Ana (call 1)
Hello Ravi (call 2)
Hello Mei (call 3)
Total calls: 3

Read the program line by line. The line call_count = 0 creates one variable at the top of the file. The function greet adds one to it on every call. The value is not thrown away when greet ends, because the variable does not belong to the function.

The line global call_count is needed. An assignment to a name inside a function makes that name local, unless a global line says otherwise. The mistakes section shows what happens without it.

Think of two boxes. A local variable is a box that a function brings out when it starts and puts back when it ends. A module-level variable is a box that stays on the shelf for the whole run. Each call to greet takes the same box off the shelf, changes the number inside, and puts it back. The number is still there the next time.

Python has no static local variable

Some languages, such as C++ and Java, let you write static inside a function, so that a variable inside one function keeps its value between calls. Python has no such keyword. If you try, Python reports a syntax error. A syntax error is a mistake in the grammar of the code, found before the program runs.

def next_number():
    static n = 0
    n = n + 1
    return n

Output:

  File "main.py", line 2
    static n = 0
           ^
SyntaxError: invalid syntax

Python offers four ways to get the same effect. The first two are the closest match. The third is simple and common. The fourth is a trap, and it is shown so that you can recognise it.

A closure with nonlocal

A closure keeps a variable that is hidden inside a function. Each time the outer function runs, it creates a new variable. The inner function keeps access to it.

Output:

1 2 3
1

The function make_counter creates count and returns increment. The name count is not visible outside make_counter, so no other code can change it by name. The line nonlocal count lets increment change the variable in make_counter, instead of creating a new local. The two counters, counter_a and counter_b, each have their own count, so they do not share a value.

This is the closest match to a static local, because the variable is hidden inside the function that uses it.

A function attribute

Functions are objects in Python, and you can attach a value to a function as an attribute. The value stays on the function object, so it survives between calls.

Output:

1
2

The line next_number.n = 0 sets the starting value. Inside the function, next_number.n refers to the attribute. This works, but the attribute can be read and changed from anywhere in the file, so it is less private than a closure.

A class attribute

A class attribute is a value stored on a class. Every object of that class, and every call to a method of the class, shares the same value. This is the usual Python tool for a shared counter, and it is the nearest match to a private static field in other languages.

Output:

1
2

A method marked with @classmethod receives the class itself as its first argument, which is named cls by convention. The line cls.count += 1 changes the class attribute. A line that starts with @ above a function is called a decorator. It changes how the function behaves, and here it tells Python that increment works on the class, not on one object.

A default argument is a trap

A default value is a value a parameter uses when the caller does not pass one, such as items=[] below. Python creates a default value once, when the function is defined. Every call that does not pass the argument then uses the same object.

Output:

['pen']
['pen', 'cup']

The second call remembers the first. The list keeps its contents between calls, which looks like a counter, but nobody meant it to work that way. Do not use a default argument to keep state. Use a closure or a class instead. The example is here so that you can recognise the surprise when you see it.

How a module-level variable differs from a global in other languages

In some languages, a global variable is visible everywhere in the program. In Python, a module-level variable is visible to every function in the same file. Another file can read it only through the module that holds it, such as settings.limit, and any file that does this can change it too.

The table below compares the places where a value can be kept between calls.

A closure variable is the safest choice when one function needs a value between calls. It keeps the shared state inside one place, so you can check every line that touches it by reading one function.

The next example shows the risk that comes with any shared value. A score is kept in a module-level variable. Two functions change it. One adds points, and the other resets it to a wrong value. Nothing in the main code looks wrong, but the score changes in a way the caller did not ask for.

Output:

Score after two adds: 17
Score after another function changed it: -5
Score after one more add: -2

Nothing in this program is an error, and each function does what its code says. The bug is in the design: the score is shared, and any function can overwrite it. When a value like this goes wrong, you must search every function that writes to it. The fewer places that write to the name, the faster you find the cause.

Two rules keep shared values manageable. First, use a closure or a class when only a few functions need the value. Second, write to a shared value in one or two functions only, and let the rest of the code call those functions. The second rule is the one most beginners skip. In the example above, reset_by_another_part writes to the score directly. A better design would have one function that changes the score, and that function would be the only place where the score changes.

A worked example: a memory of past answers

The Fibonacci numbers are a list that starts with 0 and 1. Each later number is the sum of the two numbers before it, so the list goes 0, 1, 1, 2, 3, 5, 8, and so on. The simple method for fib(30) calls itself many times and repeats the same work over and over. Saving each answer avoids the repeats.

A dictionary is a collection of key and value pairs. Each pair has a key, which is used to look things up, and a value, which is the answer stored under that key. Here the key is the position n, and the value is the Fibonacci number at that position. Python integers have no size limit, so fib(100) is exact and no special type is needed.

Output:

fib(10) = 55
computations so far = 9
fib(30) = 832040
computations so far = 29
fib(30) again = 832040
computations so far = 29

Walk through the pieces. The dictionary memo is never assigned to a new object, so it needs no global line. The line memo[n] = result changes an entry inside the dictionary, not the name memo. Only computations is assigned as a name inside the function, so only it needs global.

The function first handles the two base cases, where n is 0 or 1. Those answers are just n. Then it checks the dictionary. If the answer is already there, it returns it without further work. If not, it counts one new computation, works out the answer from the two earlier numbers, stores it, and returns it.

The first line of output shows that fib(10) needed nine new computations, one for each position from 2 to 10. The second call, fib(30), computed only the positions from 11 to 30, which is 20 more, so the total rose from 9 to 29. The third call found the answer in the dictionary and did no new work. The counter stayed at 29.

Python's standard library has a ready-made version of this memory. The functools module provides cache, which is a decorator. A decorator is a line starting with @ above a function, and cache replaces the function with a version that stores its answers.

Output:

fib(30) = 832040
CacheInfo(hits=28, misses=31, maxsize=None, currsize=31)
fib(100) = 354224848179261915075
CacheInfo(hits=99, misses=101, maxsize=None, currsize=101)

The method cache_info() reports three numbers. A hit is a call whose answer was already stored. A miss is a call that did new work. The 31 misses for fib(30) are the positions from 0 to 30. The manual version counted only the positions from 2 to 30, which is 29, because it did not count the two base cases. The cache keeps all answers, so fib(100) needed only 70 more misses, and the other calls were hits.

A worked example: handing out ids

The last example is a common real task. A program needs a new id for each order it creates. The first id is 100, the next is 101, and so on. Each call must give a number that no earlier call gave.

Output:

ids: 100 101 102
next free id: 103

The function keeps two values on purpose. The local variable current holds the value to hand out. The module-level variable next_id holds the value for the next call. The function returns the local copy, then moves the module-level value on. If the function had returned next_id after adding one, the first id would be 101, and the caller would never see 100. The order of the lines matters.

The standard library has a ready-made id source. The itertools module provides count, which gives a new number each time you ask for one with next:

Output:

100 101 102

The name id_source is never assigned again, so the function needs no global line. The call next(id_source) changes the object that id_source refers to, not the name itself.

This pattern appears in real systems. A database hands out row numbers. A web service hands out request numbers. In a program that runs one thing at a time, a value like this is enough. Programs that run several threads at once need extra care, because two threads can read the same value at the same time. That topic is outside this lesson.

Common mistakes

Mistake 1: writing static inside a function. Python reports a SyntaxError at the word static, as shown above. Remove the word. Then choose one of the four tools in the section on static locals. Mistake 2: expecting a local counter to grow. A counter declared inside a function starts at zero on every call. The output is always 1, as the first example showed. If your counter stays at one, check whether the variable is declared inside the function. If it is, move it to the module level, or use one of the other tools.

Mistake 3: assigning to a module-level name without global. Python treats any assigned name as local to the function. The read on the right side then fails, because the local has no value yet:

call_count = 0
 
def greet(name):
    call_count = call_count + 1
    print(f"Hello {name} (call {call_count})")
 
greet("Ana")

Output:

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    greet("Ana")
    ~~~~~^^^^^^^
  File "main.py", line 4, in greet
    call_count = call_count + 1
                 ^^^^^^^^^^
UnboundLocalError: cannot access local variable 'call_count' where it is not associated with a value

Add global call_count as the first line of greet. The same error appears when a closure assigns to an outer name without nonlocal.

Mistake 4: reading a class attribute by its bare name inside a method. A name written in the class body is not visible to methods by its bare name. Python reports a NameError:

class Counter:
    count = 0
 
    def show(self):
        print(count)
 
Counter().show()

Output:

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    Counter().show()
    ~~~~~~~~~~~~~~^^
  File "main.py", line 5, in show
    print(count)
          ^^^^^
NameError: name 'count' is not defined. Did you mean: 'self.count'?

Use Counter.count or self.count instead. Both forms read the class attribute.

Mistake 5: forgetting that the value is shared. A module-level value keeps its value for the whole run. If one part of the program changes the counter, and a later part expects it to start at zero, the later part sees the old value. When you test a function that uses a shared value, reset the value to its starting point before each test, or the tests will affect each other.

Mistake 6: treating an underscore as protection. A name that starts with an underscore, such as _count, tells other programmers that the name is for internal use. Python does not enforce this:

Output:

0
99

The underscore is a convention. A value that must not change from outside should be changed only through a function or method that the class provides.

Practice

Exercise 1. Write a module-level variable visits that starts at zero, and a function visit that adds one to visits and prints Visit number followed by the new value. Call visit four times.

Show the answer

The output is four lines, numbered 1 to 4.

Exercise 2. Write a function next_ticket that returns 1 the first time it is called, 2 the second time, and so on. Keep the next number in a module-level variable. Call it three times and print each result.

Show the answer

The output is ticket 1, ticket 2 and ticket 3, one per line.

Exercise 3. Explain in your own words why count = 0 inside a function cannot count across calls, even though the function is called many times.

Show the answer

The variable is created when the function starts and thrown away when it ends. Each call starts with a new variable that is set to zero again. The count is lost before the next call can use it. A module-level variable, a closure variable or a class attribute lives for longer than one call, so it keeps its value between calls.

Exercise 4. A colleague wrote a function with the line static total = 0 inside its body, and Python reports a SyntaxError. Show how to fix it with the smallest change.

Show the answer

Move the assignment out of the function, to the top of the file, and remove the word static. Then add a global total line to the function. The fixed code is below, and it prints 12.

Exercise 5. A module-level variable limit is changed by three different functions. A bug appears where the limit is sometimes 5 and sometimes 50. What would you change first, and why?

Show the answer

Put all the writes into one function, such as set_limit, and make that function the only place where the value changes. Then search for the bug in one function, instead of three. The example below keeps the name limit to stay short. A leading underscore, such as _limit, is a naming hint that tells other programmers not to change the value directly.

The output is 50.

Summary

• A local variable is created when a function starts and thrown away when it ends, so it cannot count across calls.
• A module-level variable keeps its value for the whole run. A function that changes it must declare global first.
• Python has no static keyword. A closure with nonlocal, a function attribute, or a class attribute gives the same effect.
• functools.cache stores the answers of a function, and cache_info() reports hits and misses.
• Keep writes to a shared value in one or two functions, so a wrong value is easy to trace.