Archives and downloads: tar, gzip and curl

Why this matters

Software and course projects usually reach you as one file. An author puts many files and folders into a single package, then publishes it on a web page or a server. You download it, unpack it, and run what is inside.

That leaves two questions every time. How do I pack and unpack the package? And how do I know that the file I received is the file the author published? A damaged download can look normal until a program fails with an error that makes no sense.

This lesson covers three tools. tar packs many files into one file and unpacks them again. gzip makes a file smaller. curl downloads a file from a web address. A checksum, a short fingerprint of a file, answers the second question.

The commands use paths, flags and pipes from the earlier lessons in this module. Scripts that repeat steps like these are covered in Loops and conditionals in scripts, so the last example here stays short.

Archives and compression in plain words

An archive is one file that holds other files and folders inside it. Think of a box with items packed inside. One box is easier to copy, upload and check than two hundred loose items.

Compression makes a file smaller by writing repeated patterns in a shorter form. Text compresses well because words repeat. A photo that is already a JPEG, or a file that is already a .zip, barely gets smaller.

The two jobs are done by two tools:

• tar joins files and folders into one file. It does not make anything smaller. The name comes from "tape archive", an old format for magnetic tape.
• gzip makes one file smaller. It does not know about folders.

A .tar.gz file is a tar archive that has been compressed with gzip. People often call it a tarball. The ending tells you there are two layers: the outer gzip layer and the inner tar layer. The tar command handles both layers in one step.

Making an archive with tar

Start with a small project to practise on. These commands create a folder called greeter-1.0 with three files in it:

mkdir -p greeter-1.0/src
printf 'Greeter 1.0\nPrints a greeting.\n' > greeter-1.0/README.txt
printf 'def greet(name):\n    return "Hello, " + name\n' > greeter-1.0/src/greet.py
printf 'print("hi")\n' > greeter-1.0/src/main.py

mkdir -p creates a folder, and also any folder above it that is missing. printf prints text exactly as you write it, where \n means "start a new line". The > symbol sends the text into a file, replacing anything the file held before.

Now pack the folder into an archive:

tar -czf greeter-1.0.tar.gz greeter-1.0

The command prints nothing when it works. Check that the archive exists and see its size. The sizes, times and checksums below come from one test run, so yours will differ:

ls -l greeter-1.0.tar.gz

-rw-r--r-- 1 chiranjeet chiranjeet 304 Nov  5 10:31 greeter-1.0.tar.gz

Each letter in -czf is a flag, and each flag does one job:

• c means create an archive.
• z means compress the result with gzip.
• f means the next word is the name of the archive file.

The flags can be written together, as -czf, or separately, as -c -z -f. The word after f must be the archive name, and the words after that are the things to pack. In this command, the archive name is greeter-1.0.tar.gz and the thing to pack is the folder greeter-1.0.

Add v (as in -czvf) to print each file as it is added. Use it when you want to see progress.

Packing a folder stores the folder name inside the archive, so unpacking later recreates greeter-1.0/ with everything under it. This is usually what you want, because the files stay grouped under one name.

What gzip reports for a tar.gz

gzip -l shows the compressed and uncompressed sizes of a gzip file:

gzip -l greeter-1.0.tar.gz

         compressed        uncompressed  ratio uncompressed_name
                304               10240  97.2% greeter-1.0.tar

The uncompressed size is 10240 bytes, not the size of the three files, and the name ends in .tar. The reason is that tar pads the archive to a multiple of 10240 bytes, and gzip only sees the padded tar file. Use ls -l on the files themselves to see their real sizes.

Looking inside before you extract

Before you unpack an archive from someone else, list what it holds. The t flag means list:

tar -tzf greeter-1.0.tar.gz

greeter-1.0/
greeter-1.0/README.txt
greeter-1.0/src/
greeter-1.0/src/main.py
greeter-1.0/src/greet.py

The listing shows whether everything sits under one folder, or whether files are loose at the top level. A loose file lands in the folder you extract into, mixed in with your own files.

Add v to see owners, sizes and dates:

tar -tvzf greeter-1.0.tar.gz

drwxr-xr-x chiranjeet/chiranjeet 0 2026-11-05 10:31 greeter-1.0/
-rw-r--r-- chiranjeet/chiranjeet 31 2026-11-05 10:31 greeter-1.0/README.txt
drwxr-xr-x chiranjeet/chiranjeet  0 2026-11-05 10:31 greeter-1.0/src/
-rw-r--r-- chiranjeet/chiranjeet 12 2026-11-05 10:31 greeter-1.0/src/main.py
-rw-r--r-- chiranjeet/chiranjeet 45 2026-11-05 10:31 greeter-1.0/src/greet.py

In the first column, the leading d marks a folder and - marks a file. The next nine characters are the permissions, which the environment variables and permissions lesson explains. The number after the owner and group is the size in bytes.

The -tf form also works on a .tar.gz file in GNU tar, the version of tar on most Linux systems. The examples here were run with GNU tar 1.35. Still, write z when the archive is compressed, so the command means the same thing on every system.

Extracting an archive

The x flag means extract. Use -C to choose the folder to extract into. Create that folder first, and keep it empty:

mkdir -p extracted
tar -xzf greeter-1.0.tar.gz -C extracted
find extracted | sort

extracted
extracted/greeter-1.0
extracted/greeter-1.0/README.txt
extracted/greeter-1.0/src
extracted/greeter-1.0/src/greet.py
extracted/greeter-1.0/src/main.py

find prints every path under a folder. The | pipe sends that list into sort, which puts it in alphabetical order.

Removing the top folder

Sometimes you want the files to land directly in a folder, without the greeter-1.0 folder in between. The --strip-components option removes a number of leading folder names from each path:

mkdir -p flat
tar -xzf greeter-1.0.tar.gz -C flat --strip-components=1
find flat | sort

flat
flat/README.txt
flat/src
flat/src/greet.py
flat/src/main.py

The number 1 removes one folder name, the greeter-1.0/ part, from every path. Read the listing first when you use this option, because it changes where every file lands.

Extraction overwrites files

Extraction does not ask before it replaces a file. This example puts a different README into the folder first:

mkdir -p ow/greeter-1.0
printf 'old text\n' > ow/greeter-1.0/README.txt
tar -xzf greeter-1.0.tar.gz -C ow
cat ow/greeter-1.0/README.txt

Greeter 1.0
Prints a greeting.

The text old text is gone. Extracting into an empty folder avoids this problem.

Checking a file with SHA-256

A checksum is a short code computed from the bytes of a file. The same bytes always give the same code. If one byte changes, the code changes as well. This makes a checksum a good way to find a file that was damaged in transfer.

SHA-256 is a method for computing such a code. Its output is 64 characters long, using the digits 0 to 9 and the letters a to f. The letters are part of hexadecimal, a way of writing numbers with sixteen symbols. You do not need the details. You only need to know that the code is a fingerprint.

The author usually publishes the checksum next to the download. You can make one yourself to check your own copies, or to check a file after you move it. Create a checksum file:

sha256sum greeter-1.0.tar.gz > greeter-1.0.tar.gz.sha256
cat greeter-1.0.tar.gz.sha256

1f75cf299c16a51efee611461e2d29016c1f573ae13cd41078698a0be198ee2a  greeter-1.0.tar.gz

The > sends the output into a new file. The checksum file holds the code, two spaces, and the file name. The archive records file times, so your code will differ from the one shown.

Checking a file against its checksum

Use -c to check. The command reads the checksum file and computes the code for each file it names:

sha256sum -c greeter-1.0.tar.gz.sha256
echo "exit $?"

greeter-1.0.tar.gz: OK
exit 0

echo "exit $?" prints the exit status of the last command. An exit status of 0 means the command succeeded. Any other number means it failed. Programs and scripts use this number to decide what to do next.

Now simulate a damaged download. Copy the archive and the checksum file into a new folder, then add one letter to the end of the copy:

mkdir -p broken
cp greeter-1.0.tar.gz greeter-1.0.tar.gz.sha256 broken/
printf 'x' >> broken/greeter-1.0.tar.gz
(cd broken && sha256sum -c greeter-1.0.tar.gz.sha256)
echo "exit $?"

greeter-1.0.tar.gz: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match
exit 1

The >> adds text to the end of a file, without erasing it. The parentheses run cd inside a subshell, a short-lived copy of the shell. The folder you are in stays the same after the command, so the echo runs from where you started.

A FAILED result means the file is not the one the checksum describes. Do not unpack it. Download it again, or ask the sender for a new copy.

A checksum needs the right folder

The checksum file stores the file name as a path relative to where you ran the command, so you must run the check from the folder that holds the file:

mkdir -p elsewhere
(cd elsewhere && sha256sum -c ../greeter-1.0.tar.gz.sha256)
echo "exit $?"

sha256sum: greeter-1.0.tar.gz: No such file or directory
greeter-1.0.tar.gz: FAILED open or read
sha256sum: WARNING: 1 listed file could not be read
exit 1

The message says the file was not found, which is different from a damaged file. Check the folder first.

What a checksum does not prove

A checksum shows that the file matches the code it was compared with. If someone replaces the file and the checksum on the same web page, the check still passes. For this reason, a checksum published on the same page as the download only protects against accidental damage. When a project publishes signed releases, a signature is the stronger check. Signatures are not covered in this lesson.

Downloading with curl

A URL is the address of a file, such as https://example.com/files/tool.tar.gz. Browsers and curl both ask a web server for a URL using HTTP, the protocol that web pages use. The HTTP requests and responses lesson in the internet module covers that protocol. Here you only need to know that curl sends the request and writes the answer somewhere.

The examples below use a web server on your own machine, so every output comes out the same for you. A port is a number that picks one program on a machine. localhost means this machine, so localhost:8000 is the server started below. In a second terminal, go to the folder that holds greeter-1.0.tar.gz and greeter-1.0.tar.gz.sha256, copy README.txt into it with cp greeter-1.0/README.txt ., then run:

python3 -m http.server 8000 --bind 127.0.0.1

Leave it running. The server answers requests for the files in its folder at http://localhost:8000/. Press Ctrl+C in that terminal to stop it. Run the commands below in your first terminal.

When curl saves a file, it also prints a progress meter with numbers that change as the file arrives. The outputs below leave those lines out.

Saving the response to a file

The -o option names the file to write:

mkdir -p downloads
cd downloads
curl -o downloaded.tar.gz http://localhost:8000/greeter-1.0.tar.gz
ls -l downloaded.tar.gz

-rw-r--r-- 1 chiranjeet chiranjeet 304 Nov  5 10:31 downloaded.tar.gz

The -O option, with a capital letter, uses the file name from the URL instead:

curl -O http://localhost:8000/greeter-1.0.tar.gz
ls -l greeter-1.0.tar.gz

-rw-r--r-- 1 chiranjeet chiranjeet 304 Nov  5 10:31 greeter-1.0.tar.gz

Without either option, curl prints the response to the screen. That works for text:

curl http://localhost:8000/README.txt

Greeter 1.0
Prints a greeting.

Do not print a binary file such as a .tar.gz to the screen. The terminal shows unreadable characters and may stop responding to normal input. Always use -o or -O for archives.

Checking the status code

Every HTTP response has a status code, a three-digit number. 200 means the request worked. 404 means the server could not find the file. The -w option prints information after the transfer. %{http_code} is the status code:

curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8000/greeter-1.0.tar.gz

200

The -s option hides the progress meter. -o /dev/null throws away the file contents, because only the status code is needed. /dev/null is a place that discards anything written to it.

Making curl fail on an error

By default, curl treats a 404 page as a successful download. It saves the error page and exits with status 0:

curl -o page.html http://localhost:8000/missing.tar.gz
echo "exit $?"
head -n 3 page.html

exit 0
<!DOCTYPE HTML>
<html lang="en">
    <head>

The file page.html now holds an error page instead of the file you wanted. If you had named it tool.tar.gz, you would find out later, when unpacking fails.

The -f option makes curl stop on an error, and it does not create the output file:

curl -f -o nothing.tar.gz http://localhost:8000/missing.tar.gz
echo "exit $?"
ls nothing.tar.gz

curl: (22) The requested URL returned error: 404
exit 22
ls: cannot access 'nothing.tar.gz': No such file or directory

Exit status 22 is how curl reports a 404 with -f.

Silent mode and errors

The -s option stops curl from printing anything, errors included. The -S option turns error messages back on. When you combine them as -sS, the progress meter stays hidden and errors still show. Try a server that is not running. Port 9 on this machine has no server behind it:

curl -s http://127.0.0.1:9/
echo "exit $?"
curl -sS http://127.0.0.1:9/
echo "exit $?"

exit 7
curl: (7) Failed to connect to 127.0.0.1 port 9 after 0 ms: Could not connect to server
exit 7

With -s alone, the failure is silent, and the only clue is the exit status. Adding -S shows the reason. Exit status 7 means that curl could not connect to the server at all.

For scripts, the usual choice is -fsSL. The letters mean: -f stop on an HTTP error, -s hide the meter, -S show errors, and -L follow redirects, which are responses that tell curl the file has moved to another address.

Piping a download into a checksum

You can send the download straight into sha256sum and skip saving the file:

curl -fsSL http://localhost:8000/greeter-1.0.tar.gz | sha256sum

1f75cf299c16a51efee611461e2d29016c1f573ae13cd41078698a0be198ee2a  -

The - at the end means "the input came from standard input", the stream that the pipe sends. The code matches the checksum file made earlier, so this copy is the same file. Still, save the file when you plan to keep it, because the pipe leaves nothing on disk.

Resuming a partial download

If a download stops in the middle, -C - asks curl to continue from the point where the local file ends. The following commands simulate a partial download by keeping the first 100 bytes of the file:

head -c 100 greeter-1.0.tar.gz > part.tar.gz
curl -C - -o part.tar.gz http://localhost:8000/greeter-1.0.tar.gz
echo "exit $?"

** Resuming transfer from byte position 100
curl: (33) HTTP server does not seem to support byte ranges. Cannot resume.
exit 33

head -c 100 keeps the first 100 bytes. Resuming needs the server to send only part of a file on request, which is called support for byte ranges. Python's simple server does not support it, so the command stops with exit status 33. The same resume worked when the source was a local file, written as file:/// followed by the full path, so the limit is in the server and not in the command. Many production download servers do support byte ranges.

A worked example: download, check, unpack

The three tools work well together. The script below downloads an archive and its checksum file, checks the archive, and unpacks it only when the check passes. Save it as fetch-and-check.sh:

#!/bin/bash
# usage: fetch-and-check.sh BASE_URL FILE DEST_DIR
set -eu
base="$1"
file="$2"
dest="$3"
curl -fsSL -o "$file" "$base/$file"
curl -fsSL -o "$file.sha256" "$base/$file.sha256"
sha256sum -c "$file.sha256"
mkdir -p "$dest"
tar -xzf "$file" -C "$dest" --strip-components=1
echo "unpacked into $dest"

Each line does one thing:

• The first line tells the system to run the file with bash.
• set -eu makes the script stop at the first command that fails, which is what -e does, and treats an unset variable as an error, which is what -u does.
• $1, $2 and $3 are the three words given after the script name. The lines base="$1" and the lines after it copy them into named variables, so the rest of the script reads clearly.
• The two curl lines fetch the archive and its checksum file. -fsSL makes a failed download stop the script.
• sha256sum -c checks the archive. If the check fails, the script stops here, because of set -e, and nothing is unpacked.
• The last two lines create the destination folder and unpack the archive into it, dropping the top folder name.

Run it against the local server. The folder the server serves holds the archive and a matching checksum file:

bash fetch-and-check.sh http://localhost:8000 greeter-1.0.tar.gz unpacked
find unpacked | sort

greeter-1.0.tar.gz: OK
unpacked into unpacked
unpacked
unpacked/README.txt
unpacked/src
unpacked/src/greet.py
unpacked/src/main.py

To see a mismatch, make a second folder with the archive and a checksum file that holds the wrong code, here 64 zeros typed by hand. Serve that folder on port 8001 in a third terminal, then run the script against it:

mkdir -p bad-server
cp greeter-1.0.tar.gz bad-server/
printf '0000000000000000000000000000000000000000000000000000000000000000  greeter-1.0.tar.gz\n' > bad-server/greeter-1.0.tar.gz.sha256
python3 -m http.server 8001 --bind 127.0.0.1 --directory bad-server

Then, in the first terminal:

bash fetch-and-check.sh http://localhost:8001 greeter-1.0.tar.gz unpacked

greeter-1.0.tar.gz: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match

The script stopped at the check, and no unpacked folder was created. That is the behaviour you want: a file that fails its check is never unpacked.

Common mistakes

The archive name is wrong or the file is missing. Tar reports:

tar (child): nope.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

Run ls to see the name and the folder. Names are case sensitive, so Greeter-1.0.tar.gz is not greeter-1.0.tar.gz.

The file is not an archive. Listing a plain text file with -tzf gives:

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

Run file on the name first. It prints the type of a file from its first bytes:

file greeter-1.0.tar.gz

greeter-1.0.tar.gz: gzip compressed data, from Unix, original size modulo 2^32 10240

If the answer does not mention gzip compressed data or tar archive, the file is something else, such as an HTML page saved by mistake.

A flag takes the wrong word. The f flag takes the word after it as the archive name. Putting -C directly after -xzf makes tar look for an archive called -C:

tar (child): -C: Cannot open: No such file or directory

Write the archive name right after -xzf, then -C and the folder: tar -xzf greeter-1.0.tar.gz -C extracted.

The checksum check runs from the wrong folder. The message is No such file or directory, followed by FAILED open or read. Run the check from the folder that holds the file, or use a cd in a subshell as shown above.

Extraction replaces files silently. Unpacking into a folder that already holds a file with the same name replaces that file. Extract into an empty folder unless you mean to replace what is there.

A checksum failure is treated as a warning. The FAILED line means the file is not the one the publisher described. Download it again. If the second copy also fails, ask the publisher, because the published file or its checksum may be wrong.

curl saves an error page as your file. Without -f, a 404 page is saved as the output file and the exit status is 0. Use -f in any command whose output you will use.

Silent mode hides the reason for a failure. -s alone prints nothing when the connection fails. Use -sS or -fsSL so the error message still appears.

Resume is requested from a server that cannot do it. The message HTTP server does not seem to support byte ranges means the download must start again from the beginning.

curl runs before the server starts. The first attempt fails with:

curl: (7) Failed to connect to localhost port 8000 after 0 ms: Could not connect to server

Start the server in its own terminal, check that it prints its startup line, and then run curl.

Practice

Try each exercise in your scratch folder before you read the answer. Every command below uses the greeter-1.0.tar.gz file made in this lesson, except where the exercise says otherwise.

1. You have a folder called notes with several text files in it. Write the command that makes an archive called notes.tar.gz from it.

Show the answer

tar -czf notes.tar.gz notes. The c creates the archive, z compresses it, and f names the file that follows. The folder is listed last, as the thing to pack.

2. Write the command that lists the files inside greeter-1.0.tar.gz without unpacking anything.

Show the answer

tar -tzf greeter-1.0.tar.gz. The t flag lists the contents and does not write any file.

3. Unpack greeter-1.0.tar.gz into a folder called out, so that the files land in out/greeter-1.0. The folder out may not exist yet.

Show the answer

mkdir -p out followed by tar -xzf greeter-1.0.tar.gz -C out. The -p option creates out only when it is missing, and -C out sets the folder that tar extracts into.

4. Unpack the same archive so that README.txt and src/ land directly inside out, with no greeter-1.0 folder in between.

Show the answer

mkdir -p out followed by tar -xzf greeter-1.0.tar.gz -C out --strip-components=1. The option removes the first folder name from every path, which is greeter-1.0/.

5. A colleague sends you tool.tar.gz and a checksum file. You run sha256sum -c tool.tar.gz.sha256 and see tool.tar.gz: FAILED. Explain what this means, and what you should do before you unpack the file.

Show the answer

The file you received is not the file the checksum describes. It may have been damaged in transfer, or the sender may have published a different file. Do not unpack it. Download the file again and run the check a second time. If it still fails, ask the sender, and check whether the checksum they sent is the right one for that file.

6. Write a curl command that downloads https://example.com/files/tool.tar.gz and saves it as tool.tar.gz. It should stop with an error on a 404 response, follow redirects, and still show any error message. The address is a placeholder, so do not expect it to download.

Show the answer

curl -fsSL -o tool.tar.gz https://example.com/files/tool.tar.gz. The flags are -f for stop on an HTTP error, -s to hide the progress meter, -S to show errors, -L to follow redirects, and -o to name the saved file.

Summary

• tar -czf name.tar.gz folder packs a folder into one compressed archive, tar -tzf name.tar.gz lists it, and tar -xzf name.tar.gz -C folder unpacks it into folder.
• Unpack into an empty folder, because extraction replaces files with the same names without asking.
• sha256sum file > file.sha256 records a fingerprint, and sha256sum -c file.sha256 checks the file against it. FAILED means the file is not the one described, so do not unpack it.
• A checksum published on the same page as the download only protects against accidental damage, not against someone changing both.
• curl -o name URL saves a response under a name you choose, and curl -O URL uses the name from the URL.
• Use -f so an HTTP error stops the command, and use -sS instead of -s when you need to see the reason for a failure.