NSNumber double with many decimal places being rounded/truncated

Question

I have a double in an NSNumber.

double myDouble = 1363395572.6129999;

NSNumber *doubleNumber = @(myDouble); 
// using [NSNumber numberWithDouble:myDouble] leads to the same result

This is where it gets problematic.

doubleNumber.doubleValue seems to return the correct and full value (1363395572.6129999)

However, looking at doubleNumber in the debugger or doing doubleNumber.description gives me (1363395572.613).

I would understand if perhaps this was just some display formatting, but when I then stick this object into a JSON payload, the messed up rounded value gets inserted instead of the actual number.

The way I'm doing this is something like this:

NSData *jsonData = [NSJSONSerialization dataWithJSONObject:(Dictionary containing NSNumber)
                                                           options:0 error:nil];

NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];

Looking at the string at this point shows me the truncated number with 3 decimal places even though the NSNumber I inserted had 7.

My question is why is this happening and more importantly how can I stop it from happening?

EDIT with conclusion:

For anyone who stumbles onto this, the problem was not clear to me from the beginning but the actual issue is that NSNumber and double are both incapable of holding a number with the sort of precision I am looking for. As Martin's answer shows, my problem occurred as soon as I deserialized the initial number values from a JSON response.

I ended up working around my problem by reworking the whole system to stop depending on this level of precision(since these are timestamps, microseconds) of these numbers on the client, and instead use a different identifier to pass around with the API.

As Martin and Leo pointed out, in order to get around this problem one would need to use a custom JSON parser that allows parsing of a JSON number into an NSDecimalNumber rather than an NSNumber. A better solution to my problem in particular was what I outlined in the previous paragraph, so I did not pursue this route.

Grumous

1363395572.6129999 is 1363395572.613 – Kana

A double can only store about 13 significant digits. – Homeostasis

description is only to be used for debugging, and should never (except for a handful of object types) be counted on to accurately reflect the contents of the object. – Shoreless

@HotLicks yeah I'm not using it for anything other than debugging, I was just listing out everything I've tried. – Grumous

@Homeostasis - A double can hold nearly 16 significant decimal digits. – Shoreless

@Homeostasis I should add that the plain double shows the correct value at all times and never appears truncated. I can pass it around and use the full value in its primitive form without any issues. – Grumous

Hmmm. I would have sworn double was about 13 digits. Maybe that idea came about from seeing the same behavior seen here. – Homeostasis

@Homeostasis - en.wikipedia.org/wiki/IEEE_floating_point – Shoreless

Try using NSNumber *doubleNumber = [NSNumber numberWithDouble:myDouble]; and see if it makes a difference. I've seen the @() notation causing issues by selecting the incorrect internal NSNumber subclass. – Demars

Don't use the @ notation, create the NSNumber the "old fashioned" way. – Shoreless

@LeoNatan same result. Once again the NSNumber is actually storing the correct value. I see that it is a double in the debugger and when I print doubleValue it is indeed correct. – Grumous

@HotLicks no difference in the result unfortunately. going to edit my question to note that. – Grumous

One obvious thing to try is to make a [NSString stringWithFormat:@"%.7f", myDouble] and see what it looks like. If you can force what you want that way before putting it into JSON, that should solve things. {EDIT: posted as answer, tested and works] – Subirrigate

@Subirrigate - Except that's putting a string rather than a number into the JSON. – Shoreless

Who is generating your JSON, by the way? Do you do that, or is it some 3rd party? Where is the JSON coming from? – Shoreless

@HotLicks I do, no third party. – Grumous

How are you generating the floating-point number in the JSON, then? Does it originate as a double? If so, then it only has about 15.8 digits of precision, and you're expecting 17. If you want exact, don't use a float, but use a scaled long. (But of course if you're starting with an NSTimeInterval, double precision is all you get.) – Shoreless

I believe it is a Time object generated with Ruby. The server side precision was not the issue as it maintains microsecond precision. I did end up realizing that NSTimeInterval was not suitable for maintaining that precision however and went another route (as outlined in my edited question). – Grumous

Answer 1

As already said in above comments, the precision of double is about 16 decimal digits. 1363395572.612999 has 17 digits, and converting this decimal number to double gives exactly the same results as for 1363395572.613:

double myDouble = 1363395572.6129999;
double myDouble1 = 1363395572.613;

NSLog(@"%.20f", myDouble);  // 1363395572.61299991607666015625
NSLog(@"%.20f", myDouble1); // 1363395572.61299991607666015625
NSLog(@"%s", myDouble == myDouble1 ? "equal" : "different"); // equal

Therefore, within the precision of double, the output 1363395572.613 is correct.

If your goal is to send precisely the number "1363395572.6129999" then you cannot store it in a double first because that already looses the precision. A possible solution would be to use NSDecimalNumber (which has a precision of 38 decimal digits):

NSDecimalNumber *doubleNumber = [NSDecimalNumber decimalNumberWithString:@"1363395572.6129999"];
NSDictionary *dict = @{@"key": doubleNumber};
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:dict
                                                   options:0 error:nil];
NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];
// {"key":1363395572.6129999}

Example with long double and NSDecimalNumber:

long double ld1 = 1363395572.6129999L;
long double ld2 = 1363395572.613L;

NSDecimalNumber *num1 = [NSDecimalNumber decimalNumberWithString:[NSString stringWithFormat:@"%.7Lf", ld1]];
NSDecimalNumber *num2 = [NSDecimalNumber decimalNumberWithString:[NSString stringWithFormat:@"%.7Lf", ld2]];

NSDictionary *dict = @{@"key1": num1, @"key2": num2};
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:dict
                                                   options:0 error:nil];
NSString *jsonString = [[NSString alloc] initWithData:jsonData encoding:NSUTF8StringEncoding];
// {"key1":1363395572.6129999,"key2":1363395572.613}

Update: As it turned out in the discussion, the problem occurs already when the data is read from a JSON object sent by a server. The following example shows that NSJSONSerialization is not able to read floating point numbers with more than "double" precision from JSON data:

NSString *jsonString = @"{\"key1\":1363395572.6129999,\"key2\":1363395572.613}";
NSData *jsonData = [jsonString dataUsingEncoding:NSUTF8StringEncoding];
NSDictionary *dict2 = [NSJSONSerialization JSONObjectWithData:jsonData options:0 error:NULL];
NSNumber *n1 = dict2[@"key1"];
NSNumber *n2 = dict2[@"key2"];

BOOL b = [n1 isEqualTo:n2]; // YES

Fiance

The problem here is that doing double myDouble = 1363395572.613; generates the same result, which is not the behavior I'm looking for. These numbers are microsecond precision timestamps and those 2 numbers do not represent the same time when comparing at that level. – Grumous

@Dima: 1363395572.6129999 and 1363395572.613, when stored in a double, are exactly identical! - If you have to distinguish these values, you cannot use double. Use NSDecimalNumber or (perhaps) long double. – Fiance

@Dima: I have updated the answer with an example that show that your problem occurs already when reading the server JSON. – Fiance

Question: Are you sure the value that NSJSONSerialization produced is a double and not a LongDecimal? – Shoreless

@HotLicks: The class of n1/n2 is __NSCFNumber, and the objCType is "d". And both numbers compare as equal. (developer.apple.com/library/mac/documentation/Cocoa/Conceptual/… also mentions that Objective-C does not support "long double".) – Fiance

Answer 2

Use NSDecimalNumber:

NSDecimalNumber* dc = [NSDecimalNumber decimalNumberWithString:[NSString stringWithFormat:@"%.7f", myDouble]];

Use this decimal number inside your dictionary.

---

If you have a string value of the required number precisely, feed that directly to the NSDecimalNumber constructor to get a precise decimal number. Do not use an intermediate double stage, where you lose precision.

Tincher

And for double myDouble = 1363395572.613 this gives also 1363395572.6129999 ... – Fiance

@MartinR I guess precision has to be known? If more precision is necessary, what about long double? – Demars

My point is: There is no difference between double myDouble = 1363395572.6129999; and double myDouble = 1363395572.613. Therefore "1363395572.613" in the JSON data is fine and not an error. – Fiance

@MartinR So the error is in the use of double. Precision here matters, so matt's comment that "1363395572.6129999 is 1363395572.613" may be technically correct for "double", it is incorrect in the term of the asker's requirements. – Demars

Yes. If the goal is to send the value stored in double myDouble then the result "1363395572.613" is "correct". If the goal is to send the number "1363395572.6129999" then you cannot store it in double first. – Fiance

The problem here is that doing double myDouble = 1363395572.613; generates the same result, which is not the behavior I'm looking for. These numbers are microsecond precision timestamps and those 2 numbers do not represent the same time when comparing at that level. – Grumous

@Grumous How do you generate the double? – Demars

Actually you can make your method work by using "long double" consequently: long double ld1 = 1363395572.6129999l; NSDecimalNumber *num = [NSDecimalNumber decimalNumberWithString:[NSString stringWithFormat:@"%.7Lf", ld1]]; . – Fiance

@MartinR just tried that and it doesn't work. Both of my resulting decimal numbers are 1363395572.6129999 even though one of the long doubles is 1363395572.613. – Grumous

@Dima: I have updated my answer with an example that show how it works with long double. The important point is that you have to use "long double" consequently, such as 1363395572.6129999L with the L suffix for a "long double constant". – Fiance

@LeoNatan the number is generated by a server as a timestamp and sent over to the client as a JSON number. Then at some later point I am supposed to send it back to the API and the number should be the same. – Grumous

@Dima: In that case I assume that the problem occurs already when reading the JSON data from the server. "1363395572.6129999" in the JSON data is converted to a NSNumber object containing a double. "1363395572.613" would be converted to exactly the same number. – Fiance

@Grumous If you are receiving a string value of the number, feed that directly into a NSDecimalValue instead of going through a double step and losing precision. – Demars

@MartinR so I guess the question then is what the best way would be to extract that JSON number and maintain precision. – Grumous

@LeoNatan I don't actually need or have a string at any point, it is passed to me as a number. – Grumous

@Grumous "JSON number" has come to you as string, no? (In the form of a JSON data stream.) I understand the deserializer parses it to a NSNumber, but you can parse it yourself to get a string value. – Demars

@Dima: As far as I can see, NSJSONSerialization is not able to parse your number with the full precision, it uses NSNumber with doubles. Perhaps there are alternative JSON parsers with more precision (or you parse it yourself, as Leo suggested). – Fiance

JSON format has a very defined structure, so if necessary, it is not difficult to create a custom parser. You could find an open-source one and change it for your needs. – Demars

Yeah guys, it looks like this is beyond the scope of what the default serialization does. For what it's worth I have decided to use a different variable altogether for this particular API call which allows me to avoid having to send over these timestamps and having to deal with building my own serializer – Grumous

Answer 3

I just tried and it does seem that the precision is being lost in serialization, not in initialization of the number. Maybe JSON serialization is building a string with precision-losing number formatting, then building data from that.

One solution would be to save the integral and fractional values in their own 8 byte fields... maybe use two LP64 longs. The precision part can be multiplied by something big, then divided upon retrieval.

Grobe

The precision is already lost when you assign the value to a double. – Fiance

@MartinR - True. The original value is 17 digits, and double doesn't quite handle 16. – Shoreless

Answer 4

I tested the following code:

double myDouble = 1363395572.6129999;
NSString *s = [NSString stringWithFormat:@"%.7f", myDouble];
NSLog(@"%@", s);

Output is:

1363395572.6129999

So just format the string yourself before putting into JSON object and you should be fine.

EDIT: If you want more precision, not just more control over what goes into the JSON, a long double will store it. Thus:

long double myDouble = 1363395572.6129999;
NSString *s = [NSString stringWithFormat:@"%.7L", myDouble];
NSLog(@"%@", s);

Subirrigate

And for double myDouble = 1363395572.613 you get the same output 1363395572.6129999. So this does not "increase" the precision. – Fiance

Yup, good point, but at least this point you aren't subject to the mysteries of what NSJSONSerialization does. I did just verify that even with type long double (at least on a 64-bit architecture) that 1363395572.613 == 1363395572.6129999 evaluates to true. So this is all just a 'what format do you want it in' game because the underlying value is the same. – Subirrigate

...which I just upvoted! I still think it's not a bad idea to format the string in one's one code if you have specific ideas in mind about the JSON you want. But if the issue is underlying precision then yes, changing type of storage for the value is required! – Subirrigate

Note that that's an NSString, not an NSNumber. There is a significant difference. – Shoreless

Sure there's a difference, but depending where the JSON is going it's not necessarily bad. – Subirrigate

The problem here is that doing double myDouble = 1363395572.613; generates the same result, which is not the behavior I'm looking for. These numbers are microsecond precision timestamps and those 2 numbers do not represent the same time when comparing at that level. – Grumous

then you need higher precision storage, and Martin's answer is the way to go! – Subirrigate

Actually you can make your method work by using "long double" consequently: long double ld1 = 1363395572.6129999l; NSString *s = [NSString stringWithFormat:@"%.7Lf", ld1]; . – Fiance

@Grumous - One should question whether the server should be transmitting time values as decimal fractions vs a scaled decimal integer (which would represent as long or "BigDecimal"). Any time you get a numeric value in JSON that's larger than the common 64-bit double and long representations you're apt to get into trouble. – Shoreless