Floating-point comparisons

Why this matters

A program that checks two decimal numbers with == can report "not equal" for two numbers that a person would call the same. A shop program might check whether a bill equals 0.3 and fail on a total that looks correct on the screen. The bug is silent. Nothing crashes, and the printed number looks right, so the fault can stay hidden for a long time.

This lesson explains why the bug happens, how to compare decimal numbers safely, and what the special value NaN does when it reaches a comparison. The lesson on division by zero covers what happens when whole numbers are divided by zero. Here we only look at decimal numbers.

Decimal numbers are stored in binary

A computer stores every value as bits. A bit is a single digit that is either 0 or 1. A double is a number with a decimal point, stored in 64 bits. The layout of those bits follows a standard called IEEE 754. You do not need to know the layout to use the type. You need one fact: the bits store numbers as sums of powers of two (1, 2, 4, 8 and so on, and also 1/2, 1/4, 1/8), not as powers of ten.

Powers of two can hit 0.5, 0.25 and 0.75 exactly. They cannot hit 0.1 exactly. The same thing happens with a third in ordinary decimal writing. Writing 1/3 as 0.333 is close, but it is not exact, because the threes never stop. In binary, 0.1 is 0.0001100110011 with the same pattern repeating forever. The computer keeps only 53 significant bits, so it cuts the pattern off and rounds. The stored value is very close to 0.1, but it is not 0.1.

A first look

Here is a complete program. It adds 0.1 and 0.2, then prints the result in two ways.

Press Run. The output is:

0.1 + 0.2 printed normally: 0.3
0.1 + 0.2 with 17 digits: 0.30000000000000004
0.3 with 17 digits:       0.29999999999999999
a == 0.3 ? false
0.1 exact value:  0.10000000000000001
0.1 * 3 = 0.30000000000000004, equals 0.3? false
ten times 0.1 summed: 0.99999999999999989
equals 1.0? false
1e16 + 1.0 == 1e16 ? true
1.0 / 3.0 = 0.33333333333333331

The first line prints 0.3 because cout shows only six significant digits by default. A significant digit is a digit that counts, starting from the first non-zero digit. The default rounding hides the error. The setting setprecision(17), from the <iomanip> header, asks cout to show 17 significant digits. That is enough to show the exact stored value of a double.

With 17 digits, the first two lines differ. 0.1 + 0.2 is stored as 0.30000000000000004, and the literal 0.3 is stored as 0.29999999999999999. These are two different stored values. So a == 0.3 is false.

The fourth block of output shows the same thing for a multiplication. 0.1 * 3 gives 0.30000000000000004, which is not the same as 0.3.

The loop adds 0.1 ten times and gets 0.99999999999999989, not 1.0. Each addition rounds its result to the nearest stored value, so a small error is added each time. The error stays small here, about 1e-16, but == does not allow any error at all.

The line with 1e16 + 1.0 shows another effect. At 1e16, the gap between one stored value and the next one is larger than 1. Adding 1.0 moves the number to a value that cannot be stored, so the result rounds back to 1e16. The comparison is true. Large numbers lose their small parts.

The last line shows that 1.0 / 3.0 cannot be stored exactly either. The printed digits stop at the 17th place, but the stored value is still only an approximation.

Why == is only safe for some values

Some decimal values are stored exactly. Those are the values that are sums of powers of two, such as 0.5, 0.25 and 0.75. Comparing them with == gives the answer you expect:

double half_plus_quarter = 0.5 + 0.25;
cout << (half_plus_quarter == 0.75) << endl;   // prints 1, which means true

This is why a test can pass for simple numbers and then fail for numbers from a calculation. The program was not wrong for the easy cases. It was lucky with them.

The safe rule is this: never use == to check whether two double values are equal, unless you know they were stored exactly. Use a tolerance instead.

Tolerance comparisons

A tolerance is an allowed gap. Two numbers count as equal when they differ by less than the gap. The gap is also called epsilon, written eps in code. A tolerance is a decision you make, based on how big the errors in your numbers can get.

An absolute tolerance

The first kind of tolerance uses a fixed gap. The check takes the difference between the two numbers, makes it positive with std::abs, and tests whether it is smaller than the gap. std::abs is in the <cmath> header.

#include <cmath>
 
bool almost_equal(double a, double b) {
    return std::abs(a - b) < 1e-9;
}

The gap here is 1e-9, which means 0.000000001. Here is a small program that uses it on the values from the first example:

The output is:

0.5 + 0.25 == 0.75 ? true
almost_equal(0.1 + 0.2, 0.3) ? true
ten 0.1s close to 1.0 ? true
fmod(7.5, 2.0) = 1.5
x != x ? true

The first line is true because both sides are stored exactly. The second and third lines are true only because of the tolerance. The last two lines are explained below: fmod in the mistakes section and x != x in the NaN section.

A fixed gap has a weakness. It works for numbers near 1, but it can be too big for very small numbers and too small for very large ones. The next example shows both problems.

The output is:

expected = 1000000000000
computed = 1000000000000.0002
computed == expected? false
absolute 1e-9 says equal? false
relative 1e-9 says equal? true
tiny: absolute 1e-9 says equal? true
tiny: relative 1e-9 says equal? false

Look at the first block. The number 1e12 is one trillion. The value computed is the same number plus 2e-4, which is 0.0002. That difference is a normal rounding error for a number this large, because the stored values near one trillion are only about 0.0001 apart. The fixed gap of 1e-9 says the two numbers are different. That answer is wrong for this purpose.

The second block shows the other problem. The values 1e-12 and 2e-12 differ by one part in two, which is a large difference. The fixed gap of 1e-9 says they are equal, because their difference, 1e-12, is smaller than 1e-9.

A relative tolerance

A relative tolerance sets the allowed gap as a fraction of the size of the numbers. The function close_relative takes the bigger of the two sizes, multiplies it by rel, and compares the difference with that result. With rel set to 1e-9, the allowed gap for numbers near one trillion is about 1000, so the rounding error of 0.0002 is accepted. For numbers near 1e-12, the allowed gap is about 2e-21, so the two tiny values are correctly seen as different.

Many programs use both checks together, so that values near zero and values far from zero are both handled. Which check to use depends on the values. Ask what size of error you expect. The tolerance should be bigger than that error, and small enough that real differences still show up.

NaN

NaN stands for "Not a Number". It is a special value that a double can hold. A calculation produces NaN when its result has no meaning as a number. Two common examples are 0.0 / 0.0 and the square root of -1.0.

NaN is not an error that stops the program. It is a value that moves through the program. Each comparison with NaN gives false, except for the != test. That makes NaN easy to miss. The example below shows the rules.

The output is:

nan == nan ? false
nan != nan ? true
nan < 1.0 ? false
isnan(nan) ? true
0.0 / 0.0 is nan? true
1.0 / 0.0 = inf
nan + 5.0 is nan? true

These are the rules to take from the output.

NaN is not equal to anything, including itself. nan == nan is false. A test that asks "is this value equal to NaN" always fails, so it cannot find NaN.

Every ordering check with NaN gives false. In the program, nan < 1.0 is false. The test nan > 1.0 is also false. Because of this, a check like if (temperature > 100.0) does not fire for a NaN temperature. The alarm stays silent.

Adding a number to NaN gives NaN. Adding 5.0 to NaN returns NaN, so one bad value can spread to every result that uses it.

The function std::isnan, from <cmath>, is the reliable test. It returns true only for NaN. Use it to check a value before you compare it.

Infinity is another special value. The output shows 1.0 / 0.0 printed as inf. A double can hold infinity, and it compares as larger than every normal number. Infinity can also produce NaN. For example, inf - inf is NaN. The check std::isinf tests for infinity, in the same way that std::isnan tests for NaN.

The sign of a NaN can show in the printed output. On the g++ 15.2 build used to test this lesson, 0.0 / 0.0 printed as -nan. Other systems may print nan. Do not write code that depends on the sign or on the printed text. Use std::isnan.

Common mistakes

Mistake 1: comparing decimal results with ==

This program compiles without warnings, but it prints the wrong answer for what the author intended:

The output is:

not equal

The fix is to use a tolerance, such as almost_equal from the earlier section, in place of ==.

Mistake 2: using % on a decimal number

The % operator works only on whole numbers. Using a double on its left side gives a compile error. Here is a program that fails:

#include <iostream>
 
int main() {
    double total = 7.5;
    int r = total % 2;
    std::cout << r << std::endl;
    return 0;
}

The compiler prints this error. This is the real output for the program saved as mistake2.cpp. The file name in your own error will match your file.

mistake2.cpp: In function ‘int main()’:
mistake2.cpp:5:19: error: invalid operands of types ‘double’ and ‘int’ to binary ‘operator%’
    5 |     int r = total % 2;
      |             ~~~~~ ^ ~
      |             |       |
      |             double  int

The error names the two types, double and int, which the % operator cannot combine. To get the remainder of a decimal number, use std::fmod from <cmath>. Its result for 7.5 and 2.0 is 1.5, as shown in the earlier program.

Mistake 3: checking for NaN with ==

A program that tries to find NaN with == never finds it:

#include <cmath>
#include <iostream>
#include <limits>
 
int main() {
    double x = std::numeric_limits<double>::quiet_NaN();
    if (x == std::numeric_limits<double>::quiet_NaN()) {
        std::cout << "x is NaN" << std::endl;
    } else {
        std::cout << "x is not NaN" << std::endl;
    }
    return 0;
}

The program prints x is not NaN, because NaN never equals anything. Replace the test with std::isnan(x).

Mistake 4: a tolerance that is too big or too small

A tolerance of 1e-9 works for values near 1. It accepts a gap of 0.000000001, so it treats 1e-12 and 2e-12 as equal. A tolerance that is too small rejects the rounding errors that a calculation normally produces, so two correct values can be reported as different. Choose the tolerance from the size of the numbers and the size of the errors you expect.

Practice

Exercise 1. Write a function bool almost_equal(double a, double b) that returns true when the two values differ by less than 1e-9. Test it with 0.1 + 0.2 and 0.3.

Show the answer

#include <cmath>
#include <iostream>
using namespace std;
 
bool almost_equal(double a, double b) {
    return std::abs(a - b) < 1e-9;
}
 
int main() {
    cout << boolalpha << almost_equal(0.1 + 0.2, 0.3) << endl;
    return 0;
}

The output is true. The difference between the two values is about 5.5e-17, which is smaller than 1e-9.

Exercise 2. The program below prints not equal. Explain why, without running it.

double price = 0.1 * 3;
if (price == 0.3) {
    cout << "equal" << endl;
} else {
    cout << "not equal" << endl;
}

Show the answer

The multiplication 0.1 * 3 gives the stored value 0.30000000000000004. The literal 0.3 is stored as 0.29999999999999999. The two stored values are different, so == returns false. Replace the if test with almost_equal(price, 0.3).

Exercise 3. The code below does not compile because % cannot take a double. Fix it so that r holds the remainder of 7.5 divided by 2. Its value should be 1.5.

double x = 7.5;
int r = x % 2;

Show the answer

Use std::fmod and store the result in a double:

#include <cmath>
#include <iostream>
using namespace std;
 
int main() {
    double x = 7.5;
    double r = std::fmod(x, 2.0);
    cout << r << endl;
    return 0;
}

The output is 1.5.

Exercise 4. A temperature sensor sends a NaN reading. The code below is supposed to sound an alarm when the temperature is above 100. Will the alarm sound? Why? Write a fix.

if (temperature > 100.0) {
    sound_alarm();
}

Show the answer

The alarm will not sound. Every ordering comparison with NaN returns false, so temperature > 100.0 is false. The fix is to test for NaN first and treat it as a fault:

if (std::isnan(temperature)) {
    report_sensor_fault();
} else if (temperature > 100.0) {
    sound_alarm();
}

Exercise 5. Add 0.1 to a total ten times, starting from 0.0. Print the total with 17 digits, and then print whether it is equal to 1.0 with == and with almost_equal.

Show the answer

#include <cmath>
#include <iomanip>
#include <iostream>
using namespace std;
 
bool almost_equal(double a, double b) {
    return std::abs(a - b) < 1e-9;
}
 
int main() {
    double total = 0.0;
    for (int i = 0; i < 10; ++i) total += 0.1;
    cout << boolalpha << setprecision(17);
    cout << "total = " << total << endl;
    cout << "total == 1.0 ? " << (total == 1.0) << endl;
    cout << "almost_equal(total, 1.0) ? " << almost_equal(total, 1.0) << endl;
    return 0;
}

The output is:

total = 0.99999999999999989
total == 1.0 ? false
almost_equal(total, 1.0) ? true

Summary

• A double stores a number as a sum of powers of two, so many decimal values such as 0.1 cannot be stored exactly.
• Two values that look the same on the screen can differ in their stored bits, so == can return false.
• Compare decimal values with a tolerance. An absolute gap suits numbers near 1, a relative gap suits very large numbers, and many programs use both.
• NaN is the result of a calculation with no number answer, such as 0.0 / 0.0. Every comparison with NaN is false except !=.
• Test for NaN with std::isnan, never with ==.
• The % operator takes only whole numbers. Use std::fmod for a decimal remainder.