Matching Excel's floating point in Java

Question

I have an .xlsx spreadsheet with a single number in the top-left cell of sheet 1.

The Excel UI displays:

-130.98999999999

This is visible in the formula bar, i.e. not affected by the number of decimal places the containing cell is set to show. It's the most accurate number Excel will display for this cell.

In the underlying XML, we have:

<v>-130.98999999999069</v>

When trying to read the workbook with Apache POI, it feeds the number from the XML through Double.valueOf and comes up with:

-130.9899999999907

Unfortunately, this is not the same number the user can see in Excel. Can anyone point me to an algorithm to obtain the same number the user sees in Excel?

My research so far suggests that the Excel 2007 file format uses a slightly non-standard version of IEE754 floating point, where the value space is different. I believe in Excel's floating point, this number falls the other side of the boundary for rounding and hence comes out as though rounded down instead of up.

Hawthorne

Answer 1

I agree with jmcnamara's prior answer. This answer expands on it.

For each IEEE 754 64-bit binary floating point number, there is a range of decimal fractions that would round to it on input. Starting from -130.98999999999069, the closest representable value is -130.98999999999068677425384521484375. Under round to nearest with round half even rules, anything in the range [-130.9899999999907009851085604168474674224853515625, -130.9899999999906725633991300128400325775146484375] rounds to that value. (The range is closed because the binary representation of the central number is even. If it were odd, the range would be open). Both -130.98999999999069 and -130.9899999999907 are in range.

You do have the same floating point number as Excel. You do have the same floating point number as was input to Excel. Unfortunately, further experiments suggest that Excel 2007 is only converting the most significant 15 digits of your input. I pasted -130.98999999999069 into an Excel cell. Not only was it displayed as -130.98999999999, arithmetic using it was consistent with the closest double to that value, -130.989999999990004653227515518665313720703125, rather than the original input.

To get the same effect as Excel you may need to use e.g. BigDecimal to truncate to 15 decimal digits, then convert to double.

Java's default string conversion for floating point values basically picks the decimal fraction with the fewest decimal places that would convert back to the original value. -130.9899999999907 has fewer decimal places than -130.98999999999069. Apparently, Excel is displaying fewer digits, but Apache POI is getting one of the representations of the same number as you have in Java.

Here is the program I used to obtain the numbers in this answer. Note that I am using BigDecimal only to obtain exact printouts of doubles, and to calculate the mid point between two consecutive doubles.

import java.math.BigDecimal;

class Test {
  public static void main(String[] args) {
    double d = -130.98999999999069;
    BigDecimal dDec = new BigDecimal(d);
    System.out.println("Printed as double: "+d);
    BigDecimal down = new BigDecimal(Math.nextAfter(d, Double.NEGATIVE_INFINITY));
    System.out.println("Next down: " + down);
    System.out.println("Half down: " + down.add(dDec).divide(BigDecimal.valueOf(2)));
    System.out.println("Original: " + dDec);
    BigDecimal up = new BigDecimal(Math.nextAfter(d, Double.POSITIVE_INFINITY));
    System.out.println("Half up: " + up.add(dDec).divide(BigDecimal.valueOf(2)));
    System.out.println("Next up: " + up);
    System.out.println("Original in hex: "+Long.toHexString(Double.doubleToLongBits(d)));
  }
}

Here is its output:

Printed as double: -130.9899999999907
Next down: -130.989999999990715195963275618851184844970703125
Half down: -130.9899999999907009851085604168474674224853515625
Original: -130.98999999999068677425384521484375
Half up: -130.9899999999906725633991300128400325775146484375
Next up: -130.989999999990658352544414810836315155029296875
Original in hex: c0605fae147ae000

Barrage

Answer 2

Unfortunately, this is not the same number the user can see in Excel. Can anyone point me to an algorithm to obtain the same number the user sees in Excel?

I don't think that it is using an algorithm here. Excel uses IEEE754 double internally and I'd guess that it is just using a printf style format when displaying the number:

$ python -c 'print "%.14g" % -130.98999999999069' 
-130.98999999999

$ python -c 'print "%.14g" % -130.9899999999907' 
-130.98999999999

Forceps

The internal value is probably -130.98999999999068677425384521484375, the closest IEEE 754 64-bit binary floating point value to -130.9899999999907. – Barrage

Answer 3

You need to use BigDecimal for this (for not losing any precision).
E.g. read the value as a String, then construct a BigDecimal from it.

Here is an example where you don't lose any precision i.e. this
is the way to obtain exactly the same number which the user sees in Excel.

import java.math.BigDecimal;

public class Test020 {

    public static void main(String[] args) {
        BigDecimal d1 = new BigDecimal("-130.98999999999069");
        System.out.println(d1.toString());

        BigDecimal d2 = new BigDecimal("10.0");

        System.out.println(d1.add(d2).toString());
        System.out.println(d1.multiply(d2).toString());
    }

}

Subassembly

My problem is not accuracy, it's how to emulate Excel's interpretation of the number. Using BigDecimal preserves the number as reported in the file format, but it doesn't get me any closer to matching Excel's behaviour. – Hawthorne

@DavidNorth Well, once you have the number as a BigDecimal you can round it to any number of decimal places (which is what Excel visually presents to the user, it seems; 11 decimal positions in your particular example). In general, trying to "simulate Excel's behavior" is not the way to go, I think ;) – Subassembly

I'll have to add some more examples, but the difficulty we're having is that there is no single rounding algorithm we can find using BigDecimal which matches Excel in all cases. As for not trying to simulate Excel's behavour: the difficulty is that my users can see a number in Excel and want me to show them that same number when I consume the spreadsheet. Not unreasonable of them, even if Microsoft are doing their best to make it impossible... – Hawthorne

Probably not possible to use the string-constructor. POI will give you the value as a double, I think. – Azotobacter

Answer 4

I use this for calculating the same 15 digit display value.

private static final int EXCEL_MAX_DIGITS = 15;

/**
 * Fix floating-point rounding errors.
 *
 * https://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_Excel
 * https://support.microsoft.com/en-us/kb/214118
 * https://support.microsoft.com/en-us/kb/269370
 */
private static double fixFloatingPointPrecision(double value) {
    BigDecimal original = new BigDecimal(value);
    BigDecimal fixed = new BigDecimal(original.unscaledValue(), original.precision())
            .setScale(EXCEL_MAX_DIGITS, RoundingMode.HALF_UP);
    int newScale = original.scale() - original.precision() + EXCEL_MAX_DIGITS;
    return new BigDecimal(fixed.unscaledValue(), newScale).doubleValue();
}

Haviland

Answer 5

As suggested by peter.petrov I would use BigDecimal for this. As mentioned it let's you import the data without loss and be always setting the scale to 15 you have the same behaviour as in Excel

Bezoar

Answer 6

This function should produce the same thing you see in the formula bar:

    private static BigDecimal stringedDouble(Cell cell) {
            BigDecimal result =  new BigDecimal(String.valueOf(cell.getNumericCellValue())).stripTrailingZeros();
            result = result.scale() < 0 ? result.setScale(0) : result;
            return result;
    }

Demure