Buffered output

Why this matters

A program that prints a few lines finishes before you notice anything. A program that prints hundreds of thousands of lines can take several times longer than it needs to, and the slow part is often the printing, not the arithmetic. Contest judges, data tools and test scripts all print large outputs, so the way you write output affects how long the program takes.

This lesson covers two things. The first is how to print many lines fast, which mostly means using '\n' instead of std::endl. The second is what happens to output that has not been written yet when a program stops early. Input speed and reading large files are not covered here.

What a buffer is

Writing output one line at a time to the screen is like carrying parcels to a shop one at a time. Each trip has a fixed cost, even when the parcel is small. A better way is to fill a bag and make one trip.

A buffer is a block of memory where output waits before it is sent on. When your program writes "line 1\n", the text goes into the buffer. The buffer is sent to the operating system later, in one piece. Sending a block of text to the operating system is called a write, and each write has a fixed cost.

A flush means sending whatever is waiting in the buffer right away. There are two ways to ask for a flush:

• std::endl writes a newline and then flushes.
• '\n' writes only the newline character. It does not flush.

Without std::endl, the buffer is sent when it is full, when the program ends normally, or when the program reads from std::cin. The last case happens because cin is tied to cout by default. Reading from cin flushes cout first, so a prompt such as "Enter a number: " appears before the program waits for you to type.

The difference between endl and '\n' is one flush per line. For a short program that is nothing. For a program that prints 300,000 lines, it is 300,000 flushes, which means about 300,000 writes.

First example: the same output two ways

The program below prints the same 300,000 lines. The mode is chosen with a command-line argument. The line count is the same in every mode, so the only difference is how the output is written. The program prints the time to cerr, which is the error output stream, so the timing message is not mixed into the output file.

#include <chrono>
#include <iostream>
#include <string>
using namespace std;
 
int main(int argc, char* argv[]) {
    string mode = argc > 1 ? argv[1] : "newline";
    const int n = 300000;
 
    if (mode == "fast") {
        ios::sync_with_stdio(false);
        cin.tie(nullptr);
    }
 
    auto t0 = chrono::steady_clock::now();
    if (mode == "endl") {
        for (int i = 0; i < n; ++i) cout << "line " << i << endl;
    } else if (mode == "string") {
        string out;
        out.reserve(n * 16);
        for (int i = 0; i < n; ++i) {
            out += "line ";
            out += to_string(i);
            out += '\n';
        }
        cout << out;
    } else {
        for (int i = 0; i < n; ++i) cout << "line " << i << '\n';
    }
    auto t1 = chrono::steady_clock::now();
 
    cerr << mode << ": " << chrono::duration_cast<chrono::milliseconds>(t1 - t0).count() << " ms\n";
    return 0;
}

Compile it with g++ -std=c++20 -Wall -O2 -o buf1 buf1.cpp, then run each mode with the output sent to a file:

./buf1 endl > out_endl.txt
./buf1 newline > out_newline.txt
wc -l out_endl.txt

The > symbol sends the output into a file instead of the screen. The wc -l command counts the lines in the file. Its output is:

300000 out_endl.txt

The four modes were run several times on the same machine, which used g++ 15.2.0. The times below are the ranges seen across those runs. Your numbers will differ, but the order should be similar.

The endl mode is about 15 to 24 times slower than newline on this run. The gap is the cost of the 300,000 flushes. The newline mode already gets almost all of the speed, so for most programs '\n' is the change to make.

The string mode is fastest because it makes one large write at the end. It costs memory, because the whole output must fit in the string before anything is printed. For output that is many gigabytes, that is not practical. For a few megabytes, it is a simple and fast choice.

Where the time goes

The time command runs a program and reports how long it took. The times it reports are split into two parts. User time is the time the program spent running its own code. System time is the time the operating system spent on the program's behalf, and for output that is mostly the writes.

The output below is from one run of each mode, with the output sent to a file. The format is zsh's. Bash prints real, user and sys lines instead.

./buf1 endl > t_endl.txt  0.10s user 0.22s system 99% cpu 0.319 total
./buf1 newline > t_nl.txt  0.02s user 0.00s system 98% cpu 0.022 total

The user time differs by only about 0.08 seconds. The system time differs by about 0.22 seconds, which is almost all of the gap. Building the text is cheap. Asking the operating system to write the text 300,000 times is the expensive part, and endl makes the program do that once per line.

Turning off synchronisation

By default, C++ keeps cout in step with the older C output function printf. The two share one buffer, so text from both appears in the order the program wrote it. That step costs some time on every write.

ios::sync_with_stdio(false) turns that step off. The C++ streams then keep their own buffer, separate from printf. The standard says the call should come before any input or output. cin.tie(nullptr) stops cin from flushing cout before each read. Both settings are used in the fast mode above, and they save a little more time on large outputs.

The cost is that the two buffers are sent at different times. The next program prints one line with cout, one with printf, and one more with cout. It is built with and without the sync setting.

#include <cstdio>
#include <iostream>
using namespace std;
 
int main(int argc, char* argv[]) {
    if (argc > 1) ios::sync_with_stdio(false);
    cout << "first from cout\n";
    printf("second from printf\n");
    cout << "third from cout\n";
    return 0;
}

The first run keeps the sync setting on. The second run turns it off by giving an argument:

./buf2
./buf2 off

The output of the first run is:

first from cout
second from printf
third from cout

The output of the second run is:

first from cout
third from cout
second from printf

The printf line moved to the end. The C++ buffer and the C buffer were sent at different times, so the order changed. Running the second case several times gave the same order each time, but the standard does not promise a particular order when the two are mixed.

The rule is simple. If a program uses both printf and cout, leave the sync setting on. If it uses only cout, the setting is safe to turn off.

Building the whole output first

The string mode in the first example shows a third approach. The program adds each line to a std::string, then writes the string once. The function to_string turns the number i into text. The += operator adds text to the end of a string.

This approach is useful when a program needs to compute a whole answer before printing anything, or when many small pieces of text are joined into one line. It is also a good way to see that the cost of a write is per call, not per character. Many small writes cost more than one large write that holds the same characters.

A buffer with a size limit

The string mode uses too much memory for very large output, and endl writes too often. A middle path is a small buffer that the program controls. It collects lines and writes them out whenever the pending text reaches a set size. The class below uses 64 KB as the limit. Text is written in blocks of about that size, so the number of writes stays small, and memory use stays fixed no matter how many lines the program prints.

#include <cstdio>
#include <iostream>
#include <string>
using namespace std;
 
// A small output buffer that the program owns. It writes to cout only when
// the pending text reaches the limit, and once more at the end.
class Writer {
public:
    explicit Writer(size_t limit) : limit_(limit) { pending_.reserve(limit + 64); }
    void line(const string& text) {
        pending_ += text;
        pending_ += '\n';
        if (pending_.size() >= limit_) flush();
    }
    void flush() {
        cout << pending_;
        pending_.clear();
    }
private:
    size_t limit_;
    string pending_;
};
 
int main() {
    Writer w(64 * 1024);
    for (int i = 0; i < 300000; ++i) {
        w.line("line " + to_string(i));
    }
    w.flush();
    return 0;
}

The line function adds the text and a newline to pending_. The flush function sends everything in pending_ to cout, then empties the string. The last flush call in main sends the final partial block, which is needed because the last block is usually smaller than the limit.

This program was run once with the output sent to a file. The file had 300,000 lines and 3,488,890 bytes. It was byte for byte the same as the file from the newline mode, which was checked with cmp. The run took 0.009 to 0.012 seconds in total, against 0.017 to 0.022 seconds for the newline mode in the same session. That gap is small, but the approach also keeps memory fixed, so it still works when the output is many gigabytes.

The limit is a trade-off. A larger limit means fewer writes but more memory. A smaller limit uses less memory and writes more often.

Choosing how to print

The choice depends on how much output there is and what must survive a crash. These rules cover most programs:

• For a few hundred lines, use '\n' and stop there. The time difference is too small to notice.
• For a finished program that prints many lines, use '\n'. The buffer is sent when the program returns from main, so nothing is lost.
• For output too large to hold comfortably in memory, use a buffer with a size limit, as shown above.
• Before a step that might crash, or before a prompt that the program waits on, flush. cout << flush; sends the buffer without writing a newline, and std::endl does the same thing with a newline added.

The third and fourth rules are the ones beginners miss. A progress message that is printed before a risky call is only useful if it reaches the file, and a flush is the only way to make sure it does.

What happens to unsent output when a program stops early

A buffer is sent when the program ends normally, when it returns from main. It is not sent if the program stops in another way. A program can stop because it crashes, because it calls abort (a standard function that stops the program at once), or because the operating system kills it. In those cases, whatever is still in the buffer is lost.

The next program prints three steps and then calls abort. It has two modes. In newline mode, each step ends with '\n'. In endl mode, each step ends with std::endl.

#include <cstdlib>
#include <iostream>
#include <string>
using namespace std;
 
int main(int argc, char* argv[]) {
    string mode = argc > 1 ? argv[1] : "newline";
    for (int i = 1; i <= 3; ++i) {
        cout << "step " << i;
        if (mode == "endl") cout << endl;
        else cout << '\n';
    }
    abort();
}

The output was sent to a file in both runs:

./buf3 > b3a.txt; echo "exit=$?"; wc -l < b3a.txt
./buf3 endl > b3b.txt; echo "exit=$?"; cat b3b.txt

The results were:

exit=134
0
exit=134
step 1
step 2
step 3

The exit code 134 means the program stopped because of abort. In newline mode, the file has no lines at all. The three '\n' lines were still in the buffer when the program stopped, so they were never written. In endl mode, each line was flushed as it was written, so all three lines reached the file before the abort.

This is the reason to use std::endl, or an explicit std::flush, in one specific place: right before code that might stop the program. Printing a progress message before a risky step is a common case. For ordinary output in a finished program, '\n' is correct.

Common mistakes

Using endl in a loop that prints many lines

Many beginners write cout << value << endl; out of habit. In a loop of 300,000 lines, that is 300,000 flushes. The first example measured this at about 320 ms against about 15 ms for '\n' on the same machine. Write '\n' in loops, and keep endl for the few places where a flush is needed.

Calling flush inside the loop

cout << flush; has the same cost as endl. Writing cout << i << '\n' << flush; in a loop of 300,000 lines puts the flushes back, and the program is as slow as the endl version. Flush once, at the point where it matters, and not once per line.

Mixing printf with cout after turning sync off

The second example showed the order change. The fix is to keep the sync setting on when a program uses both, or to use only one output method. Choosing one style for a whole program avoids the problem.

Losing output when the program crashes

The third example showed that three lines went missing when the program called abort. A crash in the middle of a long run can hide the messages that would have explained it. If you print progress lines to find a crash, use endl or cerr for those lines. The cerr stream is not buffered in the same way, so its text usually appears right away.

Turning off sync too late

The standard says the sync setting should be changed before any input or output happens. Changing it after some output has already been written can cause surprises. Put the two lines at the very start of main, before any other cout or cin use.

Expecting a prompt to appear when cin is untied

Setting cin.tie(nullptr) stops cin from flushing cout before each read. If a program then prints a prompt with '\n' and waits for input, the prompt may still be in the buffer. The user sees nothing and the program looks stuck. Use cout << "Enter a number: " << flush; in that case, or keep cin tied.

Practice

Exercise 1. A loop prints 1000 numbers with cout << i << endl;. What single change makes it flush only at the end?

Show the answer

Replace endl with '\n'. The line becomes cout << i << '\n';, which writes the newline without flushing. The buffer is still sent when the program ends normally.

Exercise 2. Write a program that prints the three lines row 1, row 2 and row 3, but makes only one call to cout.

Show the answer

Build the text in a string and print it once:

#include <iostream>
#include <string>
using namespace std;
 
int main() {
    string out;
    for (int i = 1; i <= 3; ++i) {
        out += "row " + to_string(i) + '\n';
    }
    cout << out;
    return 0;
}

Exercise 3. A program runs cout << "saving file" ; and then calls abort(). The output goes to a file. Is the text saving file in the file? Why?

Show the answer

No. The text has no newline and no flush, so it is still in the buffer when abort stops the program. This is the same effect the newline mode showed in the abort example. Adding << endl or << flush before the abort call would write it out.

Exercise 4. In the printf example, the line from printf came last with sync turned off. A classmate says, "So printf is always slower." Is that right?

Show the answer

No. The example says nothing about speed. The order changed because the C++ stream and the C output had separate buffers that were sent at different times. The lesson is about the order of output, not about which function is faster.

Exercise 5. Why does endl cost more than '\n' when the output goes to a file, even though both write the same characters?

Show the answer

endl asks for a flush on every line. A flush sends the buffer to the operating system right away, so each line costs one write request. With '\n', the buffer is sent only when it is full, so many lines share one write. The characters are the same, but the number of write requests is much smaller with '\n'.

Summary

• A buffer holds output until it is sent in one piece, and a flush sends it right away.
• Write '\n' in loops. std::endl flushes every time, which is slow for large output.
• Turning off sync with ios::sync_with_stdio(false) is safe only when the program does not mix cout and printf.
• Output still in the buffer is lost if the program stops through a crash or abort.
• Use endl or flush only where a line must reach the file or screen before a risky step.