Spaced repetition with flashcards

Why this matters

You have learned many commands and ideas in this course. A week later most of the exact details are gone unless you practised them. A flashcard is a small tool for practice: one question on the front, the answer on the back. You try to answer from memory, then you check. Doing this again on a schedule is what keeps the answer in memory.

Lesson 05 covers when to review, at 1, 3 and 7 days. Lesson 01 covers why recall works better than re-reading. This lesson covers the two parts those lessons leave out. The first is how to write a card that is worth reviewing. The second is how to rate each review honestly, because your rating decides when the card comes back.

Words used in this lesson

Flashcard: a card with a question on the front and an answer on the back.
Deck: a set of cards kept together. Here the deck is one text file.
Review: one attempt to answer a card from memory.
Rating: your own grade for one review. This lesson uses four: Again, Hard, Good and Easy.
Gap: the number of days until a card comes back.
Due: a card is due on the day it should come back.
CSV: "comma-separated values", a plain text table. Each line is one row, and commas separate the columns. A field that contains a comma goes inside double quotes.

Step 1: one fact on each card

A card is only useful if one attempt tells you one thing. Here is a card a beginner might write after the terminal lessons:

Front: What are the terminal flags?
Back: ls -a shows hidden files, mkdir -p makes parent folders, grep -c counts lines, and wc -l counts lines too.

When you read the front, you cannot tell what a correct answer looks like. If you remember two of the four facts, did you pass? The card gives no clear result. The back also holds four facts, so you will probably recall one and forget three.

Three rules fix this:

One fact on the back. If the back joins two facts with "and", split the card into two.
The front is a question with a short answer. A question that needs a yes or no is too weak, and so is one that asks you to "explain" a whole topic.
Write the back in your own words. A copied sentence teaches you to recognise the wording, not to recall the idea.

Example 1: splitting a lesson section

The flags section of the terminal lessons says that a lone -- means "no more flags after this point", and gives rm -- -weird-name.txt as the example. That is one idea, so it makes one card:

Front: What does a lone -- do in a command?
Back: It ends the flags. Words after it are treated as names, so a file called -weird-name.txt can be removed with rm -- -weird-name.txt.

Example 2: a question that is too vague

"What is a pipe?" can be answered in many ways, so you cannot tell whether your answer counts. A sharper card names the exact thing:

Front: What does cmd1 | cmd2 do?
Back: It sends the output of cmd1 into cmd2 as its input.

Example 3: a card with a command you must check

Some cards hold a command. Before you write one, run the command in your scratch folder and copy the real output into the card. Here is a file with seven lines, checked with two commands:

$ printf 'a\nb\nc\nd\ne\nf\ng\n' > lines.txt
$ wc -l lines.txt
7 lines.txt
$ wc -l < lines.txt
7

The card that tests the difference between them:

Front: What does wc -l file.txt print, compared with wc -l < file.txt?
Back: The first prints the count and the file name, such as 7 lines.txt. The second prints only the count, 7, because the shell feeds the data to wc, and wc then has no file name to show.

The < form sends the file's contents into the command as input, which is covered in the terminal lessons. The card is harder than the first two because the answer has two parts that must both be right.

Step 2: keep the deck as a file

A deck can be a plain CSV file with a header row. Each line after the header is one card, and each card has three columns: id, front and back. The id is a short name you use to refer to the card in a schedule. Here are six cards from the terminal lessons:

id,front,back
ls-a,"What does ls -a show that plain ls does not?","Names that start with a dot, such as .bashrc."
mkdir-p,"What does mkdir -p do?","It creates missing parent folders, and does not fail if the folder already exists."
grep-c,"What does grep -c print?","The number of matching lines."
wc-l,"What does wc -l count?","The number of lines."
exit-code,"What does echo $? print right after a command?","The exit code of that command. 0 means success."
pipe,"Which symbol sends one command's output into another command?","The pipe, |."

The mkdir card has a comma in its back, so that field sits in double quotes. If a field needs a double quote mark inside it, write two double quote marks in a row. You can edit the file in any text editor. Keep the deck in one folder, and add cards as you go.

Step 3: a program that quizzes you and sets the next gap

The program below reads the deck, shows each card that is due, asks you to press Enter before it shows the answer, then asks for your rating. From the rating it works out the next gap. It keeps everything in memory, so the schedule is lost when the program stops. A later step would save it to a file.

Python · editable

import csv
import io

DECK = """id,front,back
ls-a,"What does ls -a show that plain ls does not?","Names that start with a dot, such as .bashrc."
mkdir-p,"What does mkdir -p do?","It creates missing parent folders, and does not fail if the folder already exists."
grep-c,"What does grep -c print?","The number of matching lines."
wc-l,"What does wc -l count?","The number of lines."
exit-code,"What does echo $? print right after a command?","The exit code of that command. 0 means success."
pipe,"Which symbol sends one command's output into another command?","The pipe, |."
"""

RATINGS = {"1": "again", "2": "hard", "3": "good", "4": "easy"}

def next_interval(interval, rating):
    """Return the new gap in days after one honest rating."""
    if rating == "again":
        return 1
    if rating == "hard":
        return max(1, round(interval * 1.2))
    if rating == "good":
        return max(1, interval * 2)
    if rating == "easy":
        return max(1, interval * 3)
    raise ValueError("unknown rating: " + rating)

def ask_rating():
    while True:
        key = input("Rate 1 Again, 2 Hard, 3 Good, 4 Easy: ").strip()
        if key in RATINGS:
            return RATINGS[key]
        print("Type 1, 2, 3 or 4.")

def review_day(cards, today):
    for card in cards:
        if card["due"] > today:
            continue
        print("Q:", card["front"])
        input("Press Enter to see the answer. ")
        print("A:", card["back"])
        rating = ask_rating()
        card["interval"] = next_interval(card["interval"], rating)
        card["due"] = today + card["interval"]
        card["rating"] = rating
        print("Next review on day", card["due"])
        print()

cards = []
for row in csv.DictReader(io.StringIO(DECK)):
    row["interval"] = 1
    row["due"] = 0
    cards.append(row)

for day in range(3):
    print("Day", day)
    review_day(cards, day)

print("Schedule after day 2")
for card in cards:
    print(f"{card['id']:<10} {card['rating']:<6} gap={card['interval']} due=day {card['due']}")

Four lines do most of the work. csv.DictReader reads each row into a dictionary whose keys are the header names, so card["front"] is the question. next_interval holds the rule: a new card starts with a gap of 1, Again resets the gap to 1, and the other ratings multiply it. if card["due"] > today: continue skips every card that is not due yet. And ask_rating loops until you type 1, 2, 3 or 4.

To run it in the lesson box, type the input the program asks for. For each card shown, give an empty line for the Enter prompt, then the rating. The run below was made with the input piped in from a file, with one wrong key (9) typed on purpose. When input is piped, the Enter presses leave no mark, so the answer text appears on the same line as the prompt.

Day 0
Q: What does ls -a show that plain ls does not?
Press Enter to see the answer. A: Names that start with a dot, such as .bashrc.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 2
 
Q: What does mkdir -p do?
Press Enter to see the answer. A: It creates missing parent folders, and does not fail if the folder already exists.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 1
 
Q: What does grep -c print?
Press Enter to see the answer. A: The number of matching lines.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 3
 
Q: What does wc -l count?
Press Enter to see the answer. A: The number of lines.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 1
 
Q: What does echo $? print right after a command?
Press Enter to see the answer. A: The exit code of that command. 0 means success.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 2
 
Q: Which symbol sends one command's output into another command?
Press Enter to see the answer. A: The pipe, |.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Type 1, 2, 3 or 4.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 2
 
Day 1
Q: What does mkdir -p do?
Press Enter to see the answer. A: It creates missing parent folders, and does not fail if the folder already exists.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 3
 
Q: What does wc -l count?
Press Enter to see the answer. A: The number of lines.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 2
 
Day 2
Q: What does ls -a show that plain ls does not?
Press Enter to see the answer. A: Names that start with a dot, such as .bashrc.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 8
 
Q: What does wc -l count?
Press Enter to see the answer. A: The number of lines.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 4
 
Q: What does echo $? print right after a command?
Press Enter to see the answer. A: The exit code of that command. 0 means success.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 4
 
Q: Which symbol sends one command's output into another command?
Press Enter to see the answer. A: The pipe, |.
Rate 1 Again, 2 Hard, 3 Good, 4 Easy: Next review on day 3
 
Schedule after day 2
ls-a       easy   gap=6 due=day 8
mkdir-p    good   gap=2 due=day 3
grep-c     easy   gap=3 due=day 3
wc-l       good   gap=2 due=day 4
exit-code  hard   gap=2 due=day 4
pipe       again  gap=1 due=day 3

Read the day 0 block. The ls-a card was rated Good, so its gap became 2 and it is due on day 2. The mkdir-p card was rated Again, so its gap is 1. The card for grep-c was rated Easy, so its gap is 3. Day 1 shows only the two cards that were due, so grep-c and ls-a are skipped. The pipe card received the wrong key first, and the program asked again until it got a valid rating.

Step 4: rate by what actually happened

The program trusts the number you type, so the rating has to be honest. These four ratings are the ones this lesson uses:

1, Again. You could not produce the answer, or you produced a wrong one.
2, Hard. You produced the answer, but only after long thought, or with part of it missing.
3, Good. You produced the answer with a little effort.
4, Easy. You produced the answer at once, with no doubt.

The test is simple. Before you press Enter to see the answer, say the answer aloud or write it on paper. Then compare. If you looked first and then think "I knew that", you did not test your memory, so the rating is not Easy. Rate it Again, because tomorrow you will not know whether you knew it.

Example 3: the same card, rated two ways

The program below copies the gap rule from the deck program and prints what each rating does. It also shows a short sequence of gaps.

Python · editable

new card: [1, 1, 2, 3]
after a gap of 2: [1, 2, 4, 6]
hard from 4: 5
gaps for good, good, again, easy: [2, 4, 1, 3]

Look at the first two lines. A new card rated Easy comes back after 3 days, and rated Good after 2 days. Now suppose you rate Easy only because you read the answer first. The card returns on day 3 and you cannot answer it. The next review is an Again, and the gap drops to 1. In this program, one false Easy costs you a few days of reviews on that card. In a real deck that holds hundreds of cards, a habit of false Easy ratings means many cards come back too late, and you spend the extra time relearning them.

Step 5: a card that keeps failing

Some cards will be rated Again again and again. The gap stays at 1, so the card comes back every day. This is the program telling you the card is too big. Take this card:

Front: What does $? hold?
Back: The exit code of the last command, plus the name of the shell, plus the time it took to run.

The back has three facts, and one of them is wrong. Split the card into two that each have one fact:

Front: What is $? in the shell?
Back: A special variable that holds the exit code of the last command.
 
Front: Which exit code means success?
Back: 0. Any other number means the command failed.

Each new card starts again at a gap of 1. After a few honest reviews, both should move to longer gaps.

Common mistakes

Flipping before you try

The most common error is reading the back first and then rating Good. The card looks finished, but you have not tested anything. Fix it by saying the answer aloud or writing it before you press Enter for the answer.

Rating by feel

Rating Easy because the card looks familiar is the same mistake in a different form. Familiar and known are not the same. If you could not give the full answer without looking, the rating is not Easy.

Adding too many new cards at once

Every new card needs reviews on several days. Forty new cards on one day create reviews that continue for weeks. A small number of new cards, such as five to ten a day, is easier to keep up with.

Cards that copy the lesson

A card whose front says "Read the section on flags" is not a question, and a back that holds a full paragraph will be hard to recall in one attempt. Rewrite the card as one question with a short answer.

A rating typo with no check

The RATINGS dictionary holds only the keys 1, 2, 3 and 4. Without a check, a key that is not in the dictionary stops the program with an error. This is what Python prints for the key 5:

Traceback (most recent call last):
  File "<string>", line 2, in <module>
    RATINGS["5"]
    ~~~~~~~^^^^^
KeyError: '5'

The last line is the one to read. KeyError: '5' means the dictionary has no key called '5'. The ask_rating function in the program avoids this crash by checking the key first and printing Type 1, 2, 3 or 4. instead.

Running out of input

If the program asks for input and the input has ended, Python stops with this error:

Traceback (most recent call last):
  File "/path/to/deck_quiz.py", line 60, in <module>
    review_day(cards, day)
    ~~~~~~~~~~^^^^^^^^^^^^
  File "/path/to/deck_quiz.py", line 42, in review_day
    input("Press Enter to see the answer. ")
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
EOFError: EOF when reading a line

EOF means "end of file", the point where no more input can come. In a terminal, pressing Ctrl+D at a prompt ends the input in the same way. The traceback shows the line that asked for input, so you can see which prompt got no answer.

Practice

Exercise 1. The card What do ls, ls -a and ls -l show? holds three facts. Split it into three cards with one answer each.

Show the answer

Front: What does plain ls show?
Back: The names of the files and folders in the current folder, except names that start with a dot.
 
Front: What does ls -a show that plain ls does not?
Back: Names that start with a dot, such as .bashrc.
 
Front: What does ls -l show for each file?
Back: One line per file with its permissions, owner, size and date.

Exercise 2. A card is at a gap of 4 days and you rate it Hard. What is the new gap? Check it with the program.

Show the answer

The new gap is 5 days. Hard multiplies the gap by 1.2 and rounds the result to a whole number, so 4 times 1.2 is 4.8, which rounds to 5. Running print(next_interval(4, "hard")) from Example 3 prints 5.

Exercise 3. Start with a new card. Rate it Good, then Good, then Again, then Easy. List the gap after each rating.

Show the answer

The gaps are 2, 4, 1 and 3. The first Good gives 2 (1 times 2). The second Good doubles it to 4. Again resets the gap to 1. Easy from 1 gives 3. The last line of the output in Example 3 shows the same list: [2, 4, 1, 3].

Exercise 4. Rewrite this card so that it tests one fact and has a question on the front:

Front: Explain how the terminal works.
Back: The terminal is a text program in which you type commands, and a shell reads them, runs the programs you name, and prints their output, and you can use flags, pipes and files to control what happens.

Show the answer

The card is too large to be useful, and it asks for an explanation with no clear answer. Several smaller cards would do the job, for example:

Front: What does the shell do with a command you type?
Back: It reads the command, runs the program named, and prints the program's output.
 
Front: What does a pipe do?
Back: It sends the output of one command into the next command as input.

Exercise 5. The program has no check on the rating key. Someone types 5 at the rating prompt. What error appears, and what does the last line tell you?

Show the answer

The error is KeyError: '5'. The dictionary RATINGS has no key '5', so the lookup fails. The fix is the check in ask_rating, which asks again until the key is 1, 2, 3 or 4.

Summary

A card holds one fact, and its front is a question with a short answer.
Keep the deck as a CSV file with id, front and back columns.
Say or write the answer before you look, then rate what you did.
The four ratings (Again, Hard, Good, Easy) set the next gap, so a false Easy brings the card back too late.
A card that keeps getting Again is too big. Split it.
Add new cards slowly enough that you can keep up with the reviews.

Spaced repetition with flashcards

Why this matters

Words used in this lesson

Step 1: one fact on each card

Example 1: splitting a lesson section

Example 2: a question that is too vague

Example 3: a card with a command you must check

Step 2: keep the deck as a file

Step 3: a program that quizzes you and sets the next gap

Step 4: rate by what actually happened

Example 3: the same card, rated two ways

Step 5: a card that keeps failing

Common mistakes

Flipping before you try

Rating by feel

Adding too many new cards at once

Cards that copy the lesson

A rating typo with no check

Running out of input

Practice

Summary

Stuck? Ask the tutor