Double.Epsilon for equality, greater than, less than, less than or equal to, greater than or equal to

Question

http://msdn.microsoft.com/en-us/library/system.double.epsilon.aspx

If you create a custom algorithm that determines whether two floating-point numbers can be considered equal, you must use a value that is greater than the Epsilon constant to establish the acceptable absolute margin of difference for the two values to be considered equal. (Typically, that margin of difference is many times greater than Epsilon.)

So is this not really an epsilon that could be used for comparisons? I don't really understand the MSDN wording.

Can it be used as the epsilon in the examples here? - What is the most effective way for float and double comparison?

And finally this seems really important so I'd like to make sure I have a solid implementation for equality, greater than, less than, less than or equal to, and greater than or equal to.

Catlee

Answer 1

I don't know what they were smoking when they wrote that. Double.Epsilon is the smallest representable non-denormal floating point value that isn't 0. All you know is that, if there's a truncation error, it will always be larger than this value. Much larger.

The System.Double type can represent values accurate to up to 15 digits. So a simple first order estimate if a double value x is equal to some constant is to use an epsilon of constant * 1E-15

public static bool AboutEqual(double x, double y) {
    double epsilon = Math.Max(Math.Abs(x), Math.Abs(y)) * 1E-15;
    return Math.Abs(x - y) <= epsilon;
}

You have to watch out though, truncation errors can accumulate. If both x and y are computed values then you have to increase the epsilon.

Gemmation

Microsoft says Represents the smallest positive Double value that is greater than zero. See msdn.microsoft.com/en-us/library/…. – Hypogeous

@Ian That's what Double.Epsilon should be, but david.pfx defines it as it is :-( – Warman

1.0 + double.Epsilon = 1.0 – Ladoga

1.0 + 1E-16 = 1.0, plenty more of those :) – Gemmation

After looking for many answers on the question, your answer look like the be the most accurate... to not say the only that seems to be right. Thanks! – Prewar

Saying double only supports 15 digits is completely absurd (maybe you're thinking of float?) -- that's the whole point of floating point values, and double is 64-bit! -- Anyway, Double.Epsilon is much smaller than 1E-15, like double.Epsilon * 1E20d is still in the 4.9E-304 range... – Rarely

Tsk, tsk, experienced programmers making statements like that does explain why we get so many questions about the topic. Base 2 != base 10. – Gemmation

Ok, so if I understand this correctly, will val > 0 always give the same result as val >= double.Epsilon for any val, within the set of numbers representable by double? – Minimal

Should it be Math.Abs(x-y) < epsilon? – Boorman

@Rarely - the missing word here is "significant" digits. System.Double is capable of storing the equivalent of 15 significant digits, because the mantissa is 51 bits. log10(2^51) is approximately 15. – Jakob

Answer 2

I'd like to make sure I have a solid implementation for equality, greater than, less than, less than or equal to, and greater than or equal to.

You are using binary floating point arithmetic.

Binary floating point arithmetic was designed to represent physical quantities like length, mass, charge, time, and so on.

Presumably then you are using binary floating point arithmetic as it was intended to be used: to do arithmetic on physical quantities.

Measurements of physical quantities always have a particular precision, depending on the precision of the device used to measure them.

Since you are the one providing the values for the quantities you are manipulating, you are the one who knows what the "error bars" are on that quantity. For example, if you are providing the quantity "the height of the building is 123.56 metres" then you know that this is accurate to the centimetre, but not to the micrometer.

Therefore, when comparing two quantities for equality, the desired semantics is to say "are these two quantities equal within the error bars specified by each measurement?"

So now we have an answer to your question. What you must do is keep track of what the error is on each quantity; for example, the height of the building is "within 0.01 of 123.56 meters" because you know that is how precise the measurement is. If you then get another measurement which is 123.5587 and want to know whether the two measurements are "equal" within error tolerances, then do the subtraction and see if it falls into the error tolerance. In this case it does. If the measurements were in fact precise to the micrometre, then they are not equal.

In short: you are the only person here who knows what sensible error tolerances are, because you are the only person who knows where the figures you are manipulating came from in the first place. Use whatever error tolerance makes sense for your measurements given the precision of the equipment you used to produce it.

Eppes

While certainly correct to point out that tolerance is a more practical and straightforward measure to define I was looking into the inaccuracies of the encoded representation as a general first pass rule and tolerance would be an optional second pass depending on the specific situation. – Catlee

Eric, if you create somekind of Data Structure (eg. for game) and want to allow other games to use it, everyone will have different tolerance as they all use different coordinate systems. So, deciding what your epsilon will be, is not just a matter of personal conditions, I think. – Zondra

You did not answer the question. It is not a question of taste or context, it's related the the definition of a double in memory (IEEE 754). – Prewar

Floating point values are used quite often for synthetic calculations that have no error bars based on a physical measurement technique. Understanding the precision limits of a data type is important for many problems. – Duma

Answer 3

If you have two double values that are close to 1.0, but they differ in only their least significant bits, then the difference between them will be many orders of magnitude greater than Double.Epsilon. In fact, the difference is 324 decimal orders of magnitude. This is because of the effect of the exponent portion. Double.Epsilon has a huge negative exponent on it, while 1.0 has an exponent of zero (after the biases are removed, of course).

If you want to compare two similar values for equality, then you will need to choose a custom epsilon value that is appropriate for the orders-of-magnitude size of the values to be compared.

If the double values that you are comparing are near 1.0. Then the value of the least siginificant bit would be near 0.0000000000000001. If the double values that you are comparing are in the quadrillions, then the value of the least significant bit could be as much as a thousand. No single value for epsilon could be used for equality comparisons in both of those circumstances.

Pleven

Answer 4

I just did this - using Kent Bogarts idea.

private bool IsApproximatelyEqual(double x, double y, double acceptableVariance)
{
     double variance = x > y ? x - y : y - x;
     return variance < acceptableVariance;

     //or
     //return Math.Abs(x - y) < acceptableVariance;
}

Pontone

Answer 5

It could be used for comparisons, assuming you want to ensure the two values are either exactly equal, or have the smallest representable difference for the double type. Generally speaking, you would want to use a number greater than double.Epsilon to check whether two doubles are approximately equal.

Why the .NET framework doesn't define something like

bool IsApproximatelyEqual(double value, double permittedVariance);

is beyond me.

Pictogram

Answer 6

I use the following

public static class MathUtil {
    /// <summary>
    /// smallest such that 1.0+EpsilonF != 1.0
    /// </summary>
    public const float EpsilonF = 1.192092896e-07F;

    /// <summary>
    /// smallest such that 1.0+EpsilonD != 1.0
    /// </summary>
    public const double EpsilonD = 2.2204460492503131e-016;

    [MethodImpl( MethodImplOptions.AggressiveInlining )]
    public static bool IsZero( this double value ) {
        return value < EpsilonD && value > -EpsilonD;
    }

    [MethodImpl( MethodImplOptions.AggressiveInlining )]
    public static int Sign( this double value ) {
        if ( value < -EpsilonD ) {
            return -1;
        }
        if ( value > EpsilonD )
            return 1;
        return 0;
    }

and if you want to check for equality of two doubles 'a' and 'b', you can use

(a-b).IsZero();

and if you want to get the comparison result, use

(a-b).Sign();

Aegina

Usually the epsilon is multiplied with the bigger of both absolute values of the subtraction, because the epsilon depends on the bigger operand. – Kimmel

Answer 7

The problem with comparing doubles is that when you do a comparison between two different math results that are equal but which, due to rounding errors, aren't evaluating to the same value, they will have some difference...which is larger than epsilon, except on edge cases. And using a reliable epsilon value is also difficult. Some people consider two doubles equal if the difference between them is less than some percentage value, since using a static minimum difference epsilon may mean your differences are too small or large when the double itself is high or low.

Outlander

Answer 8

Here's some code that included twice within the Silverlight Control Toolkit:

    public static bool AreClose(double value1, double value2)
    {
        //in case they are Infinities (then epsilon check does not work)
        if(value1 == value2) return true;
        // This computes (|value1-value2| / (|value1| + |value2| + 10.0)) < DBL_EPSILON
        double eps = (Math.Abs(value1) + Math.Abs(value2) + 10.0) * DBL_EPSILON;
        double delta = value1 - value2;
        return(-eps < delta) && (eps > delta);
    }

In one place they use 1e-6 for epsilon; in another they use 1.192093E-07. You will want to choose your own epsilon.

Carrera

This one looks promising. Looks like they take care of the problem with diminishing decimal precision, when comparing high values. Of course, you must consider whether you want this kind of precision scaling. It's just as applicable as the version with a static epsilon. – Klatt

Answer 9

There is no choice you have to calculate it yourself or define own constant.

double calculateMachineEpsilon() {
    double result = 1.0;
    double one = 1.0/256;

    while(one + result/2.0 != 1.0) {
        result/=2.0;
    }
    return result;
}

Celestinacelestine

Answer 10

Since Core 3.0 has been released in Sep 2019, we've got the real analogue of the machine epsilon in .NET.

These are the methods Math.BitIncrement() and Math.BitDecrement().

Machine epsilon is Math.BitIncrement(1.0) - 1.0. Alternatively you can calculate the error immediately for some x as Math.BitIncrement(x) - x.

Here's the example of Sqrt implementation with the Math.BitIncrement():

double static Sqrt(double x)
{
  if (x < 0)
    return Double.NaN;

  // will be Double.MinValue, if x == 0
  var error = Math.BitIncrement(x) - x;

  var a = x/2;
  while (Math.Abs(a * a - x) > error)
    a = (a + x/a)/2;

  return 0;
}

Buoyant