Guards with short-circuit in Python

Why this matters

A list holds a fixed number of items, and each item has a position called an index. The first item is at index 0, the second at index 1, and so on. The last valid index is the length of the list minus 1. Reading a position that does not exist makes Python stop the program with an IndexError. It is one of the most common crashes in Python code.

A guard is a check that runs before a risky step. For a list read, the guard asks whether the index is inside the list. If it is not, the program skips the read. Writing the guard is easy. Making sure the read really does not run is the hard part, and that depends on how Python evaluates and and or.

This lesson is about the two logical words and and or. It uses the bool type and the if statement from the earlier lessons, and it does not repeat the comparison operators.

Two words that combine tests

A boolean expression is a test that gives True or False. Each test on its own is an operand, which is a value on one side of an operator. The word and joins two operands and gives True only when both are True. The word or joins two operands and gives True when at least one is True.

Here is a small program that prints the results for each combination:

The output is:

True and True = True
True and False = False
True or False = True
False or False = False

Each and needs both sides to be True. Each or needs at least one side to be True. A beginner often remembers this part and forgets the next one.

Short-circuit: the right side may never run

Python does not always compute both operands. It reads the left side first. If the left side already decides the answer, Python stops and does not compute the right side at all. This is called short-circuit evaluation. The word "short-circuit" means that the normal path is cut short.

The rules are these:

• For and, if the left side is False, the whole answer is False, so the right side is skipped.
• For or, if the left side is True, the whole answer is True, so the right side is skipped.

The right side is only computed when the left side has not yet decided the answer. This is what makes a guard work. Put the bounds check on the left, and put the list read on the right. When the check fails, the read is never done.

The output is:

False

The left side 3 < 3 is False, so and returns False straight away. The list [1, 2, 3] is never indexed at position 3, which would fail.

Worked example 1: a read after the end of a list

Here is a list with three scores. The index 5 is past the end, because the valid indexes are 0, 1 and 2. The guard is index < len(scores), which means the index is smaller than the length. The guard comes first, then the read.

The output is:

no score at index 5

The left side index < len(scores) is 5 < 3, which is False. Because the word is and, the answer is already False. Python skips scores[index] > 50, so the read at index 5 never happens and there is no crash.

Worked example 2: a negative index

A negative index is a special case in Python. It counts from the end of the list, so scores[-1] is the last item. That is a valid read, and it gives no error. The guard for a negative index therefore protects against a wrong answer, not against a crash. The guard checks that the index is zero or more, then checks the top end.

The output is:

index -1 was skipped
index 1 scored 35 not above 50
scores[-1] without the guard = 90

The first if stops at index >= 0, which is False, so the read scores[-1] never runs. Without that guard, the code would have read 90, which is the last score and not the score at index -1 that the program meant to check. The second if has three operands joined by and. Python reads them from left to right. Each one must be True for the read on the right to run, and here the index 1 passes all checks, so the read at index 1 runs and gives 35.

Joining several tests with and works in the same way. Python checks them from left to right and stops at the first one that is False.

Worked example 3: a loop that stops at the end

A search loop needs a guard too. A while loop repeats its body as long as the test is True. The loop should keep going while the index is inside the list and the value at that index is not the one it wants. Without a guard, the loop reads past the end when the value is not in the list.

The guard goes on the left of and, so it is checked before the read in each step of the loop:

The output is:

values[0] = 4
stopped at i = 1

In the first step i is 0. The guard is 0 < 3, which is True, so Python also checks values[0] != 0, which is 4 != 0, and that is True. The body prints 4 and i becomes 1. In the second step the value at index 1 is 0, so values[1] != 0 is False, and the loop ends with i equal to 1. The guard protects the read, and the read is the reason the loop stops.

The same loop can be traced one check at a time. A trace table writes down each check and what it returned, which shows whether the read ran. The table has one row for each time the loop test is evaluated:

A third row never appears, because the loop stopped. If the loop had kept going, the guard i < 3 would have turned False at i equal to 3, and the read would have been skipped on that row.

Worked example 4: the same idea with or

The or word uses the same trick in the other direction. When the left side is True, Python skips the right side. This suits a check that says a position is not usable. The position is bad if the index is below 0, or if it is past the end, or if the value there is zero. Any one of those is enough, so the tests are joined with or.

The output is:

target 3 is not usable

The first test target < 0 is False. The second test target >= len(values) is 3 >= 3, which is True. Because the answer is now True, Python skips values[target] == 0. If that read had run, it would fail, since index 3 is past the end of the list.

Notice that the check uses >= and not >. The valid indexes stop at len(values) - 1, so any index that is equal to the length or larger is bad. Using > would let index 3 through, and that is the off-by-one mistake covered below.

Worked example 5: a None guard

The value None means that a name refers to nothing. Calling a method on None, such as name.lower(), fails with an AttributeError. A None check is a guard in the same way as a bounds check. It goes first, and the method call goes second.

The output is:

name is missing, no crash

The test name is not None is False, so len(name) is never run. Without the guard, len(None) would fail with a TypeError.

Worked example 6: a call that is skipped

Short-circuit also skips function calls on the right side, not only list reads. This matters when a function has an effect, such as printing or counting. The next program counts how many times the function runs. A function is a named block of code that starts with def and gives a value back with return. The function check_and_count adds 1 to a counter and returns the value it was given.

The output is:

first = False second = True calls = 2
list has 1 item, index 1 skipped

In first, the left call returns False, so and skips the right call. In second, the left call returns True, so or skips the right call. Only two calls happen in total, and the counter shows 2. The list example works the same way: lst[1] would fail on a list with one item, but the guard idx < len(lst) is False and skips it.

The keyword global in the function tells Python that calls refers to the name outside the function. Functions are covered in a later module.

Worked example 7: neighbours in a grid

A grid is a list of rows, and each row is a list of cells. The first index chooses the row and the second chooses the column. A cell on the edge of a grid has fewer neighbours, because the cells beyond the edge do not exist. The function below counts the cells that hold 1 directly above, below, left and right of a given cell. It needs four bounds checks, one for each side of the grid.

The output is:

top-left: 1
centre: 2
bottom-right: 1

Each of the four steps gives a new row nr and a new column nc. The condition is a chain of five tests joined by and. The first four check the four edges of the grid, and the last one reads the cell. Take the top-left cell, at row 0 and column 0. The step upward gives row -1. The first test nr >= 0 is False, so Python skips the rest of the condition and never reads grid[-1][0]. Without that guard, Python would silently read the last row, because a negative index counts from the end. The step to the right gives column 1 and row 0, which passes every test, and the cell at that position holds 1, so the count goes up by one. The result is 1.

The read is the only step in the condition that can fail, so it goes last. Each comparison before it only looks at numbers, and none of them can fail.

Guards at the top of a function

A guard does not always sit inside an if in the middle of a loop. Often it sits at the top of a function, and the function returns early when the input is bad. The function then never reaches the read.

The output is:

valueAt 1: 0
valueAt 3: -1
valueAt -2: -1

The guard uses or. If either test is True, the index is bad, and the function returns -1 without reading the list. Only a valid index reaches the last line. The value -1 is a signal that the caller must check for. A function that returns a special value like this must say so in its name or its comment, because the caller can otherwise mistake -1 for a real value. Returning None is the other common choice in Python, and it is clearer when no real value could be confused with it.

What happens with & and |

Python also has single-character versions of these words. & and | do the same job when both sides are bool, but they do not short-circuit. They always compute both sides. That is the trap for a guard written with the wrong character:

The output is:

Traceback (most recent call last):
  File "/path/to/main.py", line 3, in <module>
    if (index < len(scores)) & (scores[index] > 50):
                                ~~~~~~^^^^^^^
IndexError: list index out of range

The guard index < len(scores) is False, but & still computes scores[index] > 50. The read at index 5 runs and fails. The path on the File line depends on where you save the program. The first worked example used and and did not fail. The only change is one character, and the result is an error. Use and and or for guards, and use the single-character forms only when you really want both sides to run.

Common mistakes

Putting the read before the check. The order matters. In values[i] != 0 and i < len(values), the list is read first, so the check comes too late when i is already past the end. This program, with i = 3 and the list [4, 0, 9], fails:

The output is:

Traceback (most recent call last):
  File "/path/to/main.py", line 3, in <module>
    if values[i] != 0 and i < len(values):
       ~~~~~~^^^
IndexError: list index out of range

Using & instead of and. As shown above, & runs both sides, so the guard does not protect the read. Check every guard you write for the single character.

Off-by-one in the check. The valid indexes run from 0 to len(values) - 1. The test index > len(values) lets index == len(values) through. Reading that position fails. This program uses the wrong test and then reads index 3, which is one past the end of a three-item list:

The output is:

Traceback (most recent call last):
  File "/path/to/main.py", line 3, in <module>
    if index < 0 or index > len(values) or values[index] == 0:
                                           ~~~~~~^^^^^^^
IndexError: list index out of range

The fix is index >= len(values).

Mixing up and and or. A check for "is the index valid" uses and, because both parts must be true. A check for "is the index bad" uses or, because one bad part is enough. Writing one for the other gives an answer that is the reverse of what you meant, and often fails. This check is meant to test that the index is inside the list, but or makes it True for every index that is 0 or more:

The output is:

in range
Traceback (most recent call last):
  File "/path/to/main.py", line 5, in <module>
    print(values[index])
          ~~~~~~^^^^^^^
IndexError: list index out of range

The first test 5 >= 0 is True, so the or is already True and the check says "in range". The check passed for the wrong reason. The fix is index >= 0 and index < len(values).

Guarding a different variable. The guard must test the same index that the read uses. A guard that checks i and a read that uses j protects nothing.

Practice

Exercise 1. Predict the output of this program, then check it.

data = [3, 8, 1]
i = 3
if i < len(data) and data[i] == 1:
    print("one")
else:
    print("other")

Show the answer

The output is other. The guard 3 < 3 is False, so the and is already False, and Python skips data[3] == 1. The read at index 3 never happens.

Exercise 2. Write a function is_odd(values, index) that returns True when the index is valid and the value there is odd. It must return False for an invalid index and must never fail. Test it with the list [5, 4, 7] and the indexes -1, 0, 1 and 3.

Show the answer

The output is:

isOdd -1: False
isOdd 0: True
isOdd 1: False
isOdd 3: False

The function checks the lower bound, then the upper bound, then reads the value. Each read happens only after the checks before it are True.

Exercise 3. Predict the output of this program. Explain why it does not fail.

s = None
if s is None or len(s) == 0:
    print("blank")

Show the answer

The output is blank. The left test s is None is True, so the or is already True. Python skips len(s), which would fail on None.

Exercise 4. Predict what this program does. It uses & on purpose.

a = [1, 2]
k = 2
if (k < len(a)) & (a[k] > 0):
    print("yes")

Show the answer

The program fails with IndexError: list index out of range. The & operator computes both sides, so a[2] is read even though k < len(a) is False. Changing & to and stops the failure, and the program prints nothing.

Exercise 5. This linear search fails when the target is not in the list. Find the bug, then fix it. Test it with the target 7, which is in the list, and the target 9, which is not.

def find_index(arr, target):
    i = 0
    while arr[i] != target and i < len(arr):
        i += 1
    return i

Show the answer

The bug is the order. The read arr[i] runs before the check i < len(arr). When the target is not in the list, i reaches the length, and the read fails. The fixed version puts the bounds check first:

The output is:

findIndex 7: 2
findIndex 9: 3

The value 9 is not in the list, so the function returns 3, which is the length of the list. A returned index equal to the length means "not found", because no valid index has that value. A caller must check for this result before using it as a position.

Summary

• and and or skip the right side when the left side already decides the answer. This is short-circuit evaluation.
• Put a bounds or None check on the left and the risky read on the right, so the read runs only when the check is True.
• The single-character & and | do not short-circuit, so they run the read even when the guard is False.
• Check the order: a read placed before its check still fails.
• Valid list indexes run from 0 to len(list) - 1, so the upper test is index >= len(list).
• A negative index does not fail in Python. It counts from the end, so the lower-bound guard prevents a wrong answer.