Latency, bandwidth and round trips

Why this matters

A web page can feel slow on a fast connection. Many beginners see this and assume they need a faster internet plan. Often the real cause is not the speed of the data but the number of times the browser has to stop and wait for an answer.

This lesson explains the two numbers that describe a network, latency and bandwidth. It shows how they combine into the time a page takes to load. Lesson 04 showed one request and one response. This lesson shows what happens when a page needs dozens of them.

Two numbers that describe a network

Latency is the time one piece of data takes to travel from one end of the network to the other. It is measured in milliseconds, written ms. One millisecond is one thousandth of a second.

Bandwidth is how much data can travel in one second. It is measured in bits per second. A bit is a single 0 or 1. Eight bits make one byte. Bandwidth is usually written as Mbps, which means millions of bits per second. A 10 Mbps connection can move 10,000,000 bits each second, which is 1.25 million bytes.

A round trip is one trip from the client to the server and back again. The time for one round trip is the round-trip time, written RTT. Because a reply has to travel back, one round trip costs about twice the one-way latency.

Here is a plain example with a water pipe. The width of the pipe is the bandwidth. It decides how much water can flow each second. The time a drop takes to travel from one end of the pipe to the other is the latency. A wide pipe lets more water through each second, but the first drop still takes the same time to arrive. Two different things, two different numbers.

The same holds for a network. A request is a question. The answer cannot start until the question arrives, and the client cannot ask the next question until it has the answer to the last one. Each question and answer costs one round trip, however small the answer is.

The rule to remember

Latency decides how long each wait is. Bandwidth decides how fast the data flows once it starts moving. Small files are mostly waiting. Big files are mostly moving.

Where latency comes from

Signals in a fibre-optic cable travel at about 200 km per millisecond. This is an approximate figure. The signal is slower than light in empty space, which is about 300 km per millisecond. Distance is therefore the first cause of latency. A server 5,000 km away needs at least about 25 ms to receive a message, and about 50 ms for a full round trip. This is the best case, with no delays inside routers or computers. Real round trips are longer.

Other causes add time on top of distance. Each router along the path must read a packet and pass it on. A busy network can hold packets in a queue. A server that is busy can take longer to answer. So latency is not one fixed number for one pair of computers. It changes from one moment to the next.

Measuring round-trip time

The simplest way to measure RTT is to time how long it takes to open a TCP connection. You met TCP in the first lesson. Before any data is sent, the client and the server exchange three short messages to agree that they are ready. The client's first message travels to the server, and the server's reply travels back. That is one round trip. The client's connect call returns once the reply has arrived.

Many people use the ping command for the same purpose. It sends a small message and reports how long the reply took. Many networks block ping, so the method below works in more places.

The code below times a TCP connection to a real website. It resolves the name first, so the DNS lookup does not count toward the connect time. DNS, the system that turns a name like example.com into an address, is covered in the second lesson of this module.

import socket
import statistics
import time
 
 
def timed_lookup(host):
    start = time.perf_counter()
    ip = socket.gethostbyname(host)
    return ip, (time.perf_counter() - start) * 1000
 
 
def connect_time(ip, port):
    start = time.perf_counter()
    s = socket.create_connection((ip, port), timeout=10)
    elapsed = (time.perf_counter() - start) * 1000
    s.close()
    return elapsed
 
 
ip, dns_ms = timed_lookup("example.com")
print(f"DNS lookup: {dns_ms:.1f} ms (resolved once)")
samples = [connect_time(ip, 443) for _ in range(10)]
print(f"TCP connect to {ip}:443, median of 10: {statistics.median(samples):.1f} ms")
print(f"fastest: {min(samples):.1f} ms, slowest: {max(samples):.1f} ms")

One run on the author's machine printed this:

DNS lookup: 6.9 ms (resolved once)
TCP connect to 172.66.147.243:443, median of 10: 39.3 ms
fastest: 33.0 ms, slowest: 46.2 ms

Your numbers will differ. The address can change, and the distance between your computer and the server decides most of the value. Take the median of several samples, not one. A single sample can be unlucky. The median is the middle value when the samples are sorted, so one slow sample does not move it much.

The curl command gives the same information for one request. Its -w option prints timing values after the transfer. This run used curl 8.18.0, and your times will differ:

curl -s -o /dev/null -w "namelookup=%{time_namelookup} connect=%{time_connect}\n" https://example.com/

namelookup=0.004908 connect=0.050242

Here namelookup is the DNS time in seconds, and connect is the time in seconds from the start until the TCP connection was open. The connect value includes the lookup, so the connection itself took about 0.050242 minus 0.004908, which is about 45 ms.

Worked example 1: one request that waits on the server

Imagine a server that takes 50 ms to prepare its answer. The network is fast. How long does the client wait? The client waits for the round trip and for the server's work. The server's 50 ms is added to the travel time.

The next code starts a small local server that waits for a set number of milliseconds before it replies. The client times the request from start to finish.

import statistics
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import parse_qs, urlparse
from urllib.request import urlopen
 
 
class DelayHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        ms = int(parse_qs(urlparse(self.path).query)["ms"][0])
        time.sleep(ms / 1000)
        body = b"done"
        self.send_response(200)
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)
 
    def log_message(self, *args):
        pass
 
 
server = ThreadingHTTPServer(("127.0.0.1", 8765), DelayHandler)
threading.Thread(target=server.serve_forever, daemon=True).start()
base = "http://127.0.0.1:8765"
 
 
def fetch(url):
    start = time.perf_counter()
    with urlopen(url) as resp:
        resp.read()
    return (time.perf_counter() - start) * 1000
 
 
times = [fetch(base + "/delay?ms=50") for _ in range(5)]
print(f"median total time: {statistics.median(times):.1f} ms")

The variable base holds the address of the server above, http://127.0.0.1:8765. That server reads the number after ms= and waits that many milliseconds before it replies. It answers each request in its own thread, so the requests in worked example 2 can run in parallel. A server that answers one request at a time would make the parallel run no faster. Any server that does this will give the same kind of result.

median total time: 51.5 ms

The total is 51.5 ms in this run, and yours will differ. The server's wait is 50 ms. The other 1.5 ms is the trip over the loopback network and the work of the Python code. On a real network, the trip would be longer. The lesson is that the client cannot finish before the server finishes, and the server's wait is part of the total.

Worked example 2: many small requests

A web page is rarely one file. A page can ask for an HTML file, a stylesheet, a script, and many images, icons and fonts. Each one is a separate request with its own round trip.

The next experiment makes 20 requests. Each one waits 50 ms on the server, like a small image that takes time to prepare. First the client asks one at a time. Then it asks six at a time. Six is a common limit. Browsers open only a few connections to one website at once, and the exact number depends on the browser and the protocol version. The code below uses six workers as an example.

from concurrent.futures import ThreadPoolExecutor
 
paths = [base + "/delay?ms=50"] * 20
 
start = time.perf_counter()
for p in paths:
    fetch(p)
sequential = (time.perf_counter() - start) * 1000
 
start = time.perf_counter()
with ThreadPoolExecutor(max_workers=6) as pool:
    list(pool.map(fetch, paths))
parallel = (time.perf_counter() - start) * 1000
 
print(f"one at a time: {sequential:.0f} ms")
print(f"six at a time: {parallel:.0f} ms")

one at a time: 1029 ms
six at a time: 209 ms

One at a time, the waits add up: 20 waits of 50 ms is 1000 ms, and the rest is overhead. Six at a time, the 20 requests fit in four rounds: six, six, six and then two. Four rounds of 50 ms is about 200 ms, and the measured 209 ms is close to that.

The key idea is that the total time depends on how many waits happen one after another. Making the same 20 requests faster, or on a faster line, would not remove the 50 ms server wait. Sending them at the same time removes most of the waiting.

Requests that wait for each other

Parallel requests only help when the requests are independent. Many pages need one answer before they can ask the next question. A profile page may first ask for the user's id, and only then ask for that user's posts. The second request cannot be sent until the first reply has arrived.

Using the formula from worked example 3 below, with an RTT of 100 ms and no transfer time, the arithmetic is simple. A chain of three dependent requests needs three round trips, so 300 ms. If the same three requests could be sent together, the total would be one round trip, 100 ms. The chain costs three times as much. A developer can often shorten a chain by asking for both answers in one request, if the API offers that. The lesson is to look for dependencies before you add parallel requests.

Worked example 3: the bandwidth formula

The time to load a file can be estimated with a simple formula. It is not exact, because real networks have extra effects covered later in this lesson, but it shows which number matters.

total time = round trips × RTT + bits ÷ bandwidth

The first part is the waiting. The second part is the moving. To use the formula, convert everything to the same units. This lesson uses milliseconds for time and bits for data.

The code below applies the formula to three cases. The RTT is 100 ms in every case. Each case is run at two or three bandwidths.

one 2 KB file, 10 Mbps, RTT 100 ms: 101.6 ms
one 2 KB file, 1000 Mbps, RTT 100 ms: 100.0 ms
one 5 MB file, 10 Mbps, RTT 100 ms: 4294.3 ms
one 5 MB file, 1000 Mbps, RTT 100 ms: 141.9 ms
100 files of 2 KB, one after another, 10 Mbps, RTT 100 ms: 10163.8 ms
100 files of 2 KB, one after another, 1000 Mbps, RTT 100 ms: 10001.6 ms
100 files of 2 KB, one after another, 10000 Mbps, RTT 100 ms: 10000.2 ms

Read the results with the formula in mind.

The first case is one 2 KB file. Its transfer part is 2,048 bytes times 8 bits, divided by 10 million bits per second, which is 1.6 ms. The waiting part is 100 ms. Almost all of the time is waiting. Going from 10 Mbps to 1,000 Mbps saves 1.6 ms out of about 100 ms.

The second case is one 5 MB file. Here the transfer part is large. At 10 Mbps it takes about 4.2 seconds, and the 100 ms wait is a small part of the total. Bandwidth matters a lot for big files. Going from 10 Mbps to 1,000 Mbps cuts the total from about 4.3 seconds to about 0.14 seconds.

The third case is the one that matters for web pages. One hundred small files, requested one after another, need 100 round trips. That is 10,000 ms of waiting before any moving is counted. The transfer part is tiny. Going from 10 Mbps to 10,000 Mbps saves only about 163 ms out of about 10,164 ms.

The table shows the pattern. Bandwidth helps when the data is big. Latency and the count of round trips decide the rest.

Why more bandwidth does not fix a slow page

A common belief is that a page is slow because the line is slow. The formula shows why that belief often fails. A page of many small files spends most of its time waiting for answers. A wider line does not shorten a wait. It only makes the moving part shorter, and for small files the moving part was already tiny.

Here are the three ways to make such a page faster, and what each one does.

• Make fewer requests. Combine small files into one, or remove files the page does not need. Each request removed saves one round trip.
• Send requests in parallel. When the requests do not depend on each other, sending them together removes most of the waiting, as worked example 2 showed.
• Move the server closer to the user. A server nearby has a shorter RTT. A content delivery network, which keeps copies of files in many cities, does this for popular files.

Bandwidth is still useful. A page with large images, video or big downloads needs it. The point is that for many pages, the number of round trips is the limit, and a faster line does not change that number.

Real connections add more effects than the formula shows. A new TCP connection starts slowly and speeds up as it proves it can carry data, so a short transfer can take longer than the formula predicts. Keeping connections open for reuse avoids this cost for later requests. The details belong to a networking course, and the formula is enough for the decisions in this lesson.

Common mistakes

Reading Mbps as ms. Mbps is a rate: how many bits move each second. ms is a time. A 100 Mbps line does not add 100 ms of delay. It means 100 million bits can move in one second. The delay of a request depends on RTT, not on the rate.

Measuring only on the same computer. A connection from a program to a server on the same machine uses the loopback network, which is the computer talking to itself. The connect time is well under 1 ms on most machines. Those numbers say nothing about a real website. Measure against the server you care about, and run the test several times at different hours.

Counting DNS as network latency by accident. socket.create_connection accepts a name such as example.com, and it looks up the address inside the call. The lookup time is then part of your measurement. The code in this lesson resolves the name first, then measures the connection to the address alone.

Setting a timeout shorter than the server's work. A client that gives up too early sees an error even though the server is working. This is the output when a client waits 0.1 seconds for a server that needs 0.5 seconds:

TimeoutError: timed out

The message does not say the server is slow. It says the client stopped waiting. The fix is to give the server enough time, or to make the server faster, and the choice depends on the app.

Confusing a closed port with a slow network. When nothing listens on the port, the connection fails at once. Python prints this:

URLError: <urlopen error [Errno 111] Connection refused>

curl prints a similar message:

curl: (7) Failed to connect to 127.0.0.1 port 1 after 0 ms: Could not connect to server

Both messages mean the connection was refused, and latency is not the cause. Check that the address and port are right, and that a server is running on that port.

Writing a loop of requests without thinking about order. A for loop that calls a fetch function waits for each call to finish before starting the next. That is the one-at-a-time case in worked example 2. It is correct when the next request needs the last answer. It is slow when the requests are independent.

Practice

Exercise 1. A page loads 30 files one after another. The RTT is 80 ms. Ignore the time to move the data. How long does the waiting take?

Show the answer

30 round trips times 80 ms each is 2,400 ms, which is 2.4 seconds.

Exercise 2. A file is 1,000,000 bytes. The connection is 8 Mbps. Ignore the round trip. How long does the moving take?

Show the answer

1,000,000 bytes is 8,000,000 bits. At 8,000,000 bits per second, that takes 1 second.

Exercise 3. A page needs 200 files of 2 KB each, requested one after another. The RTT is 120 ms. Compare the total time on a 50 Mbps line and a 500 Mbps line.

Show the answer

The waiting is 200 times 120 ms, which is 24,000 ms. The moving is tiny. On 50 Mbps the total is about 24,066 ms, and on 500 Mbps it is about 24,007 ms. Both are about 24 seconds, so the faster line saves about 59 ms.

Exercise 4. The same 10 files with an RTT of 150 ms are requested in two different ways. Requested one after another, they take 10 round trips. Requested in batches of five, they take two rounds. Ignore the moving. What are the two totals?

Show the answer

One after another: 10 times 150 ms is 1,500 ms. In batches of five: 2 times 150 ms is 300 ms. The batched version is five times faster, and the bandwidth did not change.

Exercise 5. A client sets a timeout of 2 seconds. The server needs 3 seconds to answer. What does the client see, and what is the cause?

Show the answer

The client raises a timeout error, such as TimeoutError: timed out, after 2 seconds. The cause is the client giving up, not a network fault. The server may still finish its work after the client has left.

Summary

• Latency is the time one message takes to travel. Bandwidth is how many bits can move each second.
• One round trip costs at least the RTT, and every request that waits for an answer pays it.
• Bandwidth decides the moving time, which matters for large files. Latency and the number of round trips decide the waiting time, which matters for many small files.
• A page of many small requests is slow mainly because of waiting. Fewer requests and parallel requests help more than a faster line.
• Time a TCP connection to measure RTT, and resolve the name first so DNS is not counted.