Regular expressions with grep and sed

Why this matters

A log file can hold thousands of lines. You often need only the lines that match a rule, such as "starts with a date in November" or "contains an email address". You also need to change text in many places at once, such as replacing an old name in a list of files.

grep finds lines. sed rewrites lines. Both take a regular expression, usually shortened to regex. A regex is a pattern written with ordinary letters plus a few special symbols. Each symbol has a meaning, such as "start of line" or "any digit". Once you know the symbols, one short pattern can match many different lines.

This lesson covers the patterns and the sed substitution form. It assumes you already know how to run grep PATTERN FILE, which prints each line that contains the pattern. Shell wildcards such as *.txt from the earlier lessons are a different, simpler system. Do not mix the two: * in a regex means "repeat the previous character", not "any file name".

Setting up the practice files

Create these files in a folder called regex. Every example below uses them.

$ mkdir -p ~/forge-lab/regex && cd ~/forge-lab/regex
$ cat > server.log <<'EOF'
2026-11-04 09:00:01 INFO server started
2026-11-04 09:01:10 WARN slow query 1200ms
2026-11-04 09:02:44 ERROR payment failed user=ravi
2026-11-04 09:03:30 ERROR timeout calling db
2026-11-04 09:04:12 INFO user=asha logout
EOF
$ printf 'asha.k@example.com\nbad-address\nravi@mail.co.in\nx@y\n' > emails.txt
$ printf 'ha\nhaha\nhahaha\nhi\n' > laugh.txt
$ printf 'to to be\nis is no\n' > repeats.txt
$ printf 'call\ncell\nc ll\n' > words.txt
$ printf 'aaa\nab\n' > plus.txt
$ printf 'cat sat\ndog\nbird cat\nhotdog\n' > pets.txt
$ printf 'user=ravi\nuser=asha\n' > users.txt

The text inside <<'EOF' and EOF is written into server.log exactly as typed. Keep the quotes around 'EOF', so that bash does not change anything inside the text.

Plain text first

Start with a pattern that has no special symbols. Any letter or digit matches itself:

$ grep 'ERROR' server.log
2026-11-04 09:02:44 ERROR payment failed user=ravi
2026-11-04 09:03:30 ERROR timeout calling db

Quotes matter. Put the pattern in single quotes so that the shell passes it to grep unchanged. Without quotes, characters such as * may be replaced by file names before grep runs.

Special symbols

A regex uses a few characters with special meaning. The most common ones are listed here, and each one is used in the examples that follow.

• . matches any single character.
• ^ matches the start of a line.
• $ matches the end of a line.
• [...] matches one character from a set, such as [0-9] for any digit.
• [^...] matches one character that is not in the set.
• * means "zero or more of the thing before it".
• +, ?, |, ( ) and { } are special only in extended mode, which the next section turns on.

The dot is useful and also easy to misuse. Here is what it does:

$ grep 'c.ll' words.txt
call
cell
c ll

c.ll matches c, then any one character, then ll. The space in c ll counts as that one character, so the line matches. The dot matches anything, including a letter you did not mean to allow. That is why the pattern can be too loose.

Extended mode with grep -E

grep has two pattern dialects. The default one treats +, ?, |, ( and { as plain characters. Add -E to get the extended dialect, where they have their special meaning. Most people use -E for every regex they write, so you do not have to remember which symbols need a backslash.

Compare the two. The plus sign matches one or more of the letter before it:

$ grep 'a+' plus.txt
$ grep -E 'a+' plus.txt
aaa
ab

The first command finds nothing. In the default dialect, a+ means the letters a and +, and the file has no plus signs. The second command finds both lines, because a+ means "one or more a".

Other extended symbols follow the same pattern:

• ? means "zero or one of the thing before it".
• a|b means "a or b".
• {2,3} means "between 2 and 3 of the thing before it".

Anchors: matching the start or end of a line

An anchor does not match a character. It matches a position. ^ matches the position at the start of the line, and $ matches the position at the end.

$ grep -E '^2026-11-04 09:0[12]' server.log
2026-11-04 09:01:10 WARN slow query 1200ms
2026-11-04 09:02:44 ERROR payment failed user=ravi

The pattern says: the line must begin with the date and a time that starts with 09:0, followed by 1 or 2. The [12] is a character set, so it matches a 1 or a 2 in that position. Lines that have the same text in the middle but not at the start do not match.

The end anchor works the same way:

$ grep -E 'db$' server.log
2026-11-04 09:03:30 ERROR timeout calling db

Only the line that ends in db matches. The word db appears in no other place.

Anchors are worth using in scripts. Suppose a list of file names should be checked for the ending .txt. The pattern \.txt$ matches only names that end that way. The pattern \.txt also matches a name such as notes.txt.bak, which is not what you want.

Character sets and repeats

Put a set in square brackets. A range inside a set uses a dash: [a-z] is any lowercase letter, and [0-9] is any digit.

A count after an item says how many times it may repeat. {4} means exactly four, {2,} means two or more, and {2,3} means two or three.

$ grep -E '[0-9]{4}-[0-9]{2}' server.log | head -1
2026-11-04 09:00:01 INFO server started

This pattern matches a four-digit year, a dash, and a two-digit month. The | head -1 part sends the output into head, which keeps only the first line. Pipes were covered in an earlier lesson.

A set that starts with ^ means "anything except these". The pattern [^ ]+@ matches one or more characters that are not spaces, followed by an @. That finds the start of an email address:

$ grep -E '[^ ]+@' emails.txt
asha.k@example.com
ravi@mail.co.in
x@y

Now a stricter check. This pattern accepts a simple email address: letters and dots before the @, then a domain name, a dot, and a top-level part of at least two letters. An optional two-letter part, such as .in in mail.co.in, is allowed.

$ grep -nE '^[a-z.]+@[a-z]+\.[a-z]{2,}(\.[a-z]{2})?$' emails.txt
1:asha.k@example.com
3:ravi@mail.co.in

The -n flag adds line numbers. The line bad-address has no @, and x@y has no dot after the domain, so both are rejected. This is a simple check that catches common typos. It is not a full email validator, because the rules for valid addresses are wider than one pattern.

Groups and alternatives

Parentheses group part of a pattern, so a symbol can apply to the whole group. The pattern (ha)+ means one or more copies of ha, not one or more a after h.

$ grep -E '^(ha)+$' laugh.txt
ha
haha
hahaha

Without the parentheses, ha+ would mean h followed by one or more a. With the anchors, ^ha+$ would match haa but not hahaha.

A count can apply to a group as well:

$ grep -E '^(ha){2,3}$' laugh.txt
haha
hahaha

The bar | inside a group chooses between alternatives:

$ grep -E 'WARN|ERROR' server.log
2026-11-04 09:01:10 WARN slow query 1200ms
2026-11-04 09:02:44 ERROR payment failed user=ravi
2026-11-04 09:03:30 ERROR timeout calling db

A group can also be referred to later in the same pattern. \1 means "the text that group 1 matched". This finds a word that appears twice in a row:

$ grep -E '\b([a-z]+) \1\b' repeats.txt
to to be
is is no

The \b symbol is a word boundary, so the match starts and ends on whole words. The group captures to, then the space, then \1 requires the same word again. This feature is available in GNU grep, the version used on most Linux systems.

Groups are also the way sed picks text to reuse, as the next part shows.

Useful grep options

These options combine with any pattern:

$ grep -c 'ERROR' server.log
2
$ grep -v 'INFO' server.log
2026-11-04 09:01:10 WARN slow query 1200ms
2026-11-04 09:02:44 ERROR payment failed user=ravi
2026-11-04 09:03:30 ERROR timeout calling db
$ grep -on 'user=[a-z]*' server.log
3:user=ravi
5:user=asha

The -o output shows just the text the pattern matched, which is useful for pulling out one field. The -n flag before -o in the last command adds the line number to each match.

Changing text with sed

sed stands for "stream editor". It reads a file one line at a time, applies rules, and prints the result. By default it prints to the screen and leaves the file as it was. The most common rule is the substitute command, which looks like this:

s/OLD/NEW/

s means substitute, OLD is a regex, and NEW is the replacement text. Without -E, sed uses the default dialect, so write -E as you did with grep.

$ echo 'cat cat dog' | sed 's/cat/bird/'
bird cat dog
$ echo 'cat cat dog' | sed 's/cat/bird/g'
bird bird dog

Only the first match on each line changes, unless you add the g flag, which means "global": change every match on the line.

Each line of a file goes through the same rule, so one s command can change many lines:

$ sed -E 's/(ERROR|WARN) /[\1] /' server.log
2026-11-04 09:00:01 INFO server started
2026-11-04 09:01:10 [WARN] slow query 1200ms
2026-11-04 09:02:44 [ERROR] payment failed user=ravi
2026-11-04 09:03:30 [ERROR] timeout calling db
2026-11-04 09:04:12 INFO user=asha logout

The pattern (ERROR|WARN) captures the word and the space after it into group 1. The replacement [\1] puts that word back inside square brackets. The \1 in the replacement is the same group reference as in the grep example, but now it means "insert the text the group matched".

Groups can also reorder text. Here a name is written as first last, and the replacement writes it as last, first:

$ printf 'Asha Patel\nRavi Shah\n' | sed -E 's/^([A-Za-z]+) ([A-Za-z]+)$/\2, \1/'
Patel, Asha
Shah, Ravi

The replacement \2, \1 writes group 2, then a comma and a space, then group 1.

Selecting lines with sed

sed can also choose lines. A pattern between slashes selects lines that match it. Two commands are common:

• /PATTERN/p prints the matching lines. Combine it with -n, which turns off the default printing of every line.
• /PATTERN/d deletes the matching lines.

$ sed -n '/ERROR/p' server.log
2026-11-04 09:02:44 ERROR payment failed user=ravi
2026-11-04 09:03:30 ERROR timeout calling db
$ sed '/INFO/d' server.log
2026-11-04 09:01:10 WARN slow query 1200ms
2026-11-04 09:02:44 ERROR payment failed user=ravi
2026-11-04 09:03:30 ERROR timeout calling db

A number in front of the command picks a line by its position, and two numbers separated by a comma pick a range. 3,4s/ERROR/FAIL/ applies the substitution only to lines 3 and 4:

$ sed '3,4s/ERROR/FAIL/' server.log
2026-11-04 09:00:01 INFO server started
2026-11-04 09:01:10 WARN slow query 1200ms
2026-11-04 09:02:44 FAIL payment failed user=ravi
2026-11-04 09:03:30 FAIL timeout calling db
2026-11-04 09:04:12 INFO user=asha logout

Dry runs: preview before you edit

sed can edit a file in place with the -i flag. That is fast, and it is also how a wrong pattern can damage a file you cannot easily rebuild. A dry run is a check that shows the change without making it. Here is the habit to use every time:

1. Run the sed command without -i, so the result prints to the screen.
2. Compare the result with the original using diff.
3. Only when the change is right, make it in place, and keep a backup.

Step 1 and 2 look like this for users.txt:

$ sed -E 's/user=([a-z]+)/login: \1/' users.txt | diff users.txt -
1,2c1,2
< user=ravi
< user=asha
---
> login: ravi
> login: asha

The diff command prints the differences. Lines that start with < are from the first file, and lines that start with > come from the second. Here the second file is -, which means the output of the pipe. The 1,2c1,2 line says lines 1 and 2 of the first input were changed into lines 1 and 2 of the second input. If the output had been wrong, for example because the pattern matched nothing, diff would print nothing, and you would know the command did not do what you meant.

Step 3 uses -i with a suffix. The suffix names a backup copy that sed saves before it writes the new file:

$ cp users.txt work.txt
$ sed -i.bak 's/user=/member: /' work.txt
$ cat work.txt
member: ravi
member: asha
$ ls work.txt*
work.txt
work.txt.bak

work.txt.bak holds the original text. If the change was wrong, copy the backup back over the file. The .bak suffix is a name you choose. Some people use .orig, and the important part is that the name is different from the original.

Worked example: find failed requests and rename a path

A web server writes one line per request: the date, the time, the method, the path, and the status code. A status of 400 or above means the request failed. Create visits.txt with four lines:

$ cat > visits.txt <<'EOF'
2026-11-04 09:00:01 GET /home 200
2026-11-04 09:00:05 POST /login 500
2026-11-04 09:01:10 GET /cart 404
2026-11-04 09:02:00 GET /login 200
EOF

The first task is to print the failed requests. The pattern needs a space, then a digit of 4 or 5, then two more digits, then the end of the line:

$ grep -E ' [45][0-9]{2}$' visits.txt
2026-11-04 09:00:05 POST /login 500
2026-11-04 09:01:10 GET /cart 404

The second task is to list only the paths of those requests. Pipe the result into a second grep, this time with -o, which prints just the match. The path is a slash followed by lowercase letters:

$ grep -E ' [45][0-9]{2}$' visits.txt | grep -oE '/[a-z]+'
/login
/cart

The third task is to rename the path /login to /signin. This is a change to the file, so it gets a dry run first. The | character is used as the separator here, so the slash in the path needs no backslash:

$ sed -E 's|/login|/signin|g' visits.txt | diff visits.txt -
2c2
< 2026-11-04 09:00:05 POST /login 500
---
> 2026-11-04 09:00:05 POST /signin 500
4c4
< 2026-11-04 09:02:00 GET /login 200
---
> 2026-11-04 09:02:00 GET /signin 200

The diff shows exactly two changed lines and no others. Lines 1 and 3 do not contain /login, so they are not in the output of the diff. Only after this check would you run sed -i.bak on the real file.

Common mistakes

Forgetting -E. A pattern such as (cat|dog) without -E looks for the literal characters (, | and ). It finds nothing, and the command still exits with code 1 and no message. If a pattern you expect to match returns nothing, check the flag first.

A dot that matches too much. The dot matches any character, so v1.2 also matches v1x2. Escape a dot with a backslash when you mean a real dot, as in v1\.2:

$ echo 'v1.2' | sed -E 's/v1.2/OK/'
OK
$ echo 'v1x2' | sed -E 's/v1.2/OK/'
OK
$ echo 'v1x2' | sed -E 's/v1\.2/OK/'
v1x2

A greedy star that eats too much. The star takes as many characters as it can, so <b>.*<\/b> runs from the first <b> to the last </b> on the line. A pattern that says "any character except <" stops at the first match:

$ echo '<b>one</b> and <b>two</b>' | sed 's/<b>.*<\/b>/X/'
X
$ echo '<b>one</b> and <b>two</b>' | sed 's/<b>[^<]*<\/b>/X/g'
X and X

The first command replaces everything from the first <b> to the last </b>, so the words and and the second tag are lost. The second command replaces each tag pair on its own.

A slash inside the pattern. The / character ends the pattern in s/OLD/NEW/. A path such as /home needs a backslash before each slash, or a different separator. The | character works well for paths:

$ echo '/home/a/b' | sed 's/\/home/\/data/'
/data/a/b
$ echo '/home/a/b' | sed 's|/home|/data|'
/data/a/b

A bad pattern. An unmatched opening parenthesis is an error, and grep prints it:

$ grep -E '(ERROR' server.log
grep: Unmatched ( or \(
$ echo $?
2

A missing file or a broken command. sed reports both problems and exits with a non-zero code:

$ sed 's/a/b/' nothere.txt
sed: can't read nothere.txt: No such file or directory
$ sed 's/a/b' users.txt
sed: -e expression #1, char 5: unterminated `s' command

The second error is a missing final /. The s command needs three slashes: one before the old text, one before the new text, and one at the end.

Changing a file before checking the result. Running sed -i without a preview is the most costly mistake in this lesson. Use the dry run described above every time.

Practice

Use the practice files from this lesson. Each answer is below its exercise.

Exercise 1. Print the lines of server.log that contain user= followed by a name of four or more letters.

Show the answer

$ grep -E 'user=[a-z]{4,}' server.log
2026-11-04 09:02:44 ERROR payment failed user=ravi
2026-11-04 09:04:12 INFO user=asha logout

The name ravi and the name asha each have four letters, so both lines match.

Exercise 2. Print only the lines of emails.txt whose domain ends in .in.

Show the answer

$ grep -E '@[a-z.]+\.in$' emails.txt
ravi@mail.co.in

The $ anchor makes the pattern require .in at the very end of the line.

Exercise 3. Change INFO to OK on lines 1 and 2 of server.log only. Print the result without changing the file.

Show the answer

$ sed '1,2s/INFO/OK/' server.log
2026-11-04 09:00:01 OK server started
2026-11-04 09:01:10 WARN slow query 1200ms
2026-11-04 09:02:44 ERROR payment failed user=ravi
2026-11-04 09:03:30 ERROR timeout calling db
2026-11-04 09:04:12 INFO user=asha logout

Line 2 contains WARN, not INFO, so nothing changes there. Line 5 is outside the range, so it keeps INFO.

Exercise 4. Write the users.txt lines as login: ravi and login: asha into a new file called users-new.txt, leaving users.txt unchanged. Then compare the two files with diff.

Show the answer

$ sed -E 's/user=([a-z]+)/login: \1/' users.txt > users-new.txt
$ diff users.txt users-new.txt
1,2c1,2
< user=ravi
< user=asha
---
> login: ravi
> login: asha

Writing to users-new.txt leaves users.txt unchanged. The diff output confirms the two lines were changed and nothing else.

Exercise 5. Print the lines of pets.txt that start with cat or dog.

Show the answer

$ grep -E '^(cat|dog)' pets.txt
cat sat
dog

The line bird cat does not start with either word, and hotdog contains dog but not at the start.

Summary

• grep -E turns on extended patterns, where +, ?, |, ( ) and { } have their special meanings.
• ^ and $ match the start and end of a line, and they match a position rather than a character.
• Character sets such as [0-9] and groups such as (ha)+ describe repeated or varied text. Use \1 to refer back to a group.
• sed -E 's/OLD/NEW/g' replaces text on every line. Use \1 in the new text to reuse a group.
• Run sed without -i first, and compare the result with diff before you edit a file.
• Use sed -i.bak when you edit in place, so the original text stays in the backup file.