How to determine the true data type of an NSNumber?

Question

Consider this code:

NSNumber* interchangeId = dict[@"interchangeMarkerLogId"];
long long llValue = [interchangeId longLongValue];
double dValue = [interchangeId doubleValue];
NSNumber* doubleId = [NSNumber numberWithDouble:dValue];
long long llDouble = [doubleId longLongValue];
if (llValue > 1000000) {
    NSLog(@"Have Marker iD = %@,  interchangeId = %@, long long value = %lld, doubleNumber = %@, doubleAsLL = %lld, CType = %s, longlong = %s", self.iD, interchangeId, llValue, doubleId, llDouble, [interchangeId objCType], @encode(long long));
}

The results:

Have Marker iD = (null), interchangeId = 635168520811866143, long long value = 635168520811866143, doubleNumber = 6.351685208118661e+17, doubleAsLL = 635168520811866112, CType = d, longlong = q

dict is coming from NSJSONSerialization, and the original JSON source data is "interchangeId":635168520811866143. It appears that all 18 digits of the value have been captured in the NSNumber, so it could not possibly have been accumulated by NSJSONSerialization as a double (which is limited to 16 decimal digits). Yet, objCType is reporting that it's a double.

We find this in the documentation for NSNumber: "The returned type does not necessarily match the method the receiver was created with." So apparently this is a "feechure" (i.e., documented bug).

So how can I determine that this value originated as an integer and not a floating point value, so I can extract it correctly, with all the available precision? (Keep in mind that I have some other values that are legitimately floating-point, and I need to extract those accurately as well.)

I've come up with two solutions so far:

The first, which does not make use of knowledge of NSDecimalNumber --

NSString* numberString = [obj stringValue];
BOOL fixed = YES;
for (int i = 0; i < numberString.length; i++) {
    unichar theChar = [numberString characterAtIndex:i];
    if (theChar != '-' && (theChar < '0' || theChar > '9')) {
        fixed = NO;
        break;
    }
}

The second, which assumes that we only need worry about NSDecimalNumber objects, and can trust the CType results from regular NSNumbers --

if ([obj isKindOfClass:[NSDecimalNumber class]]) {
    // Need to determine if integer or floating-point.  NSDecimalNumber is a subclass of NSNumber, but it always reports it's type as double.
    NSDecimal decimalStruct = [obj decimalValue];
    // The decimal value is usually "compact", so may have a positive exponent even if integer (due to trailing zeros).  "Length" is expressed in terms of 4-digit halfwords.
    if (decimalStruct._exponent >= 0 && decimalStruct._exponent + 4 * decimalStruct._length < 20) {
        sqlite3_bind_int64(pStmt, idx, [obj longLongValue]);            
    }
    else {
        sqlite3_bind_double(pStmt, idx, [obj doubleValue]);           
    }
}
else ... handle regular NSNumber by testing CType.

The second should be more efficient, especially since it does not need to create a new object, but is slightly worrisome in that it depends on "undocumented behavior/interface" of NSDecimal -- the meanings of the fields are not documented anywhere (that I can find) and are said to be "private".

Both appear to work.

Though on thinking about it a bit -- The second approach has some "boundary" problems, since one can't readily adjust the limits to assure that the maximum possible 64-bit binary int will "pass" without risking loss of a slightly larger number.

Rather unbelievably, this scheme fails in some cases:

BOOL fixed = NO;
long long llValue = [obj longLongValue];
NSNumber* testNumber = [[NSNumber alloc] initWithLongLong:llValue];
if ([testNumber isEqualToNumber:obj]) {
    fixed = YES;
}

I didn't save the value, but there is one for which the NSNumber will essentially be unequal to itself -- the values both display the same but do not register as equal (and it is certain that the value originated as an integer).

This appears to work, so far:

BOOL fixed = NO;
if ([obj isKindOfClass:[NSNumber class]]) {
     long long llValue = [obj longLongValue];
     NSNumber* testNumber = [[[obj class] alloc] initWithLongLong:llValue];
     if ([testNumber isEqualToNumber:obj]) {
         fixed = YES;
     }
 }

Apparently isEqualToNumber does not work reliably between an NSNumber and an NSDecimalNumber.

(But the bounty is still open, for the best suggestion or improvement.)

Alfilaria

I don't think you can. You need to know what to ask for some other way. Perhaps by adding a type property to your source json so you know what type something is supposed to be. – Atone

I have tested your code with the JSON input {"interchangeId":635168520811866143}, and the output was CType = q, i.e. a long long. (iOS 7 Simulator, 32- and 64-bit). – Cesya

Looks like you won't be able to do it with NSJSONSerialization, take a look at SBJson framework, the current master version has separate callbacks when parsing integer and real values which you should be able to utilize. – Petr

It appears that the problem is due to the value returned by NSJSONSerialization actually being an NSDecimalNumber, and CType of an NSDecimalNumber is apparently always reported as double. (Note that this is on the simulator. It's entirely possible that the hardware version behaves differently.) – Alfilaria

@HotLicks: I do not get a NSDecimalNumber when I try it with your input. I get a NSNumber with underlying type "long long". – Cesya

@MartinR - What are you testing on? I'm running Xcode 4.6.3 on the iPad 6.0 simulator. – Alfilaria

@HotLicks: I tested Xcode 4.6.3 with iPhone 6.1 Simulator, and Xcode 5.0.2 with iPad 7.0 Simulator (32 bit and 64 bit). – Cesya

@MartinR - My deployment target is iOS 4.3. – Alfilaria

@HotLicks: Same result with iOS 4.3: NSNumber/longlong. The strange thing is that I claimed myself in other answers that NSJSONSerialization uses NSDecimalNumber - but I cannot reproduce that anymore. – Cesya

So this is what I did: NSString *jsonString = @"{\"interchangeId\":635168520811866143}"; NSData *jsonData = [jsonString dataUsingEncoding:NSUTF8StringEncoding]; NSDictionary *dict = [NSJSONSerialization JSONObjectWithData:jsonData options:0 error:NULL]; NSNumber *interchangeId = dict[@"interchangeId"]; NSLog(@"%s - %d", [interchangeId objCType], [interchangeId isKindOfClass:[NSDecimalNumber class]]); - Output: q - 0. – Cesya

I'm having the deserialization done as a part of a LARGE payload, but it's going through NSJSONSerialization and should, in theory, be identical regardless of the size of the surrounding data. Otherwise, the code is exactly as shown above -- copy/paste, with only some proprietary names removed. – Alfilaria

@HotLicks Are you sure that it isn't a coincidence that the double was able to hold your long long value? I believe it is. Do you experience the same behavior with every large value? I'm sure you can indeed produce large integers that don't fit into a double and observe the rounding error. – Quatrefoil

In my tests, NSDecimalNumber is used if the JSON number is 1000000000000000000 (10^18) or greater. – Cesya

@H2CO3 - In my tests NSDecimalNumber was apparently being used for all numbers. And I only ran into the problem because the double couldn't hold the large number -- was losing 2 digits off the bottom of 18-digit numbers (as shown above). – Alfilaria

I don't suppose the difference is somehow weirdly associated with using NSJSONReadingMutableContainers, which we specify. – Alfilaria

(In my most recent tests smaller numbers (below 10000 or so, that I've observed) come through as a regular NSNumber, not NSDecimalNumber.) – Alfilaria

yes, use SBJSON instead. – Borchert

@Borchert - Not an option. And it's not clear that SBJSON would solve the problem, since we need the numbers to be presented as NSNumbers. – Alfilaria

I was basing this on the above comment that SBJSON gives you different callbacks for integer and real values. Additionally you can combine that with a category on NSNumber that would allow you to tag each NSNumber with a type (implemented with objc_[set|get]AssociatedObject() – Borchert

huh--so it turns out I solved the slightly wrong problem, although I believe my solution may be a workaround... I guess it's still a mystery why a) NSDecimalNumber is being returned so much and b) what NSDecimalNumber always returns double for it's type. – Borchert

@HotLicks: Could you perhaps share a JSON file demonstrating the problem (as I still cannot reproduce the issue)? - Are you not satisfied with your last approach? – Cesya

Yes, oddly NSDecimalNumber is often returned for small numbers of 2-4 digits. And NSNumber is used for quite large values. It seems pretty random. – Alfilaria

@MartinR - Aside from the fact that the data's proprietary, the JSON files are generally large (300kb is a small one). (But I just discovered that this particular chunk of JSON is being processed through SBJSON, not the NSJSONSerialization that is used most everywhere else. The app has a long and somewhat inglorious history.) – Alfilaria

@HotLicks: My idea was that the problem could perhaps be reduced to a smaller JSON file. - But you are using SBJson, which explains that your and my results are different. - Btw. I downloaded the latest SBJson release, and it is mentioned in SBJson4Parser.h that they don't use NSDecimalNumber anymore. – Cesya

Here are some typical numbers: 635181972857880412, 635186295223202884, 635163617138212067, 1379959474327, 634885209476900044, 1379959474240, 635167855910121903, 635167855894340350. Probably both long and short are based on timestamps, only derived differently. And I really only have trouble with the longer ones. – Alfilaria

Yeah, I'll probably change this code to use NSJSONSerialization, but the issue is still an open one, given the poor facilities iOS gives you. – Alfilaria

The curious thing (well, not the only one) is that this version of SBJson only creates NSDecimalNumbers. But a lot are flowing through as pure NSNumbers, testing false for isKindOfClass:[NSDecimalNumber class]. So there must be something strange inside the NSDecimalNumber constructor (which is decimalNumberWithString). – Alfilaria

(I apologize for not capturing a list of "problem" numbers, but you know how things are in the heat of battle.) – Alfilaria

@HotLicks: The test that you have titled with "Rather unbelievably, ..." recognizes all of the above numbers as "fixed" (both as NSNumber and NSDecimalNumber). - Your last test "This appears to work, ..." crashes if obj is a __NSCFNumber object: *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** initialization method -initWithLongLong: cannot be sent to an abstract object of class __NSCFNumber: Create a concrete instance!' – Cesya

Yeah, apparently the NSNumber comes through as a cluster name and the alloc of that fails. But while testing that I did discover that a 19-digit number (6351819728578804120LL) will cause the NSNumber <> NSDecimalNumber failure. – Alfilaria

@HotLicks: [NSNumber numberWithLongLong:6351819728578804120LL] passes the test when I try it in the iOS 6 Simulator (I am on OS X 10.9 now, which means that "older" releases cannot be simulated anymore). – Cesya

I'm running iOS 6 simulator on Xcode 4.6.3. My OS X claims to be 10.8.5. Note: To get the NSDecimalNumber I used [NSDecimalNumber decimalNumberWithString:[theNSNumber stringValue]]. – Alfilaria

(BTW, I'll be away from work for the next week, and away from the Mac where all this is. I'll still check occasionally, but won't be able to consult the app.) – Alfilaria

Just as I was leaving work it came to me that there are two different paths that the JSON takes, depending on the phase of operation of the app. I strongly suspect that one path goes through NSJSONSerialization, while the other (as noted a few lines up) goes through SBJson. This is probably why I was getting both NSNumbers and NSDecimalNumbers. But the code supporting this is a "pod" that needs to work with any setup, so I can't simply standardize on one JSON package or the other. – Alfilaria

Answer 1

As documented in NSDecimalNumber.h, NSDecimalNumber always returns "d" for it's return type. This is expected behavior.

- (const char *)objCType NS_RETURNS_INNER_POINTER;
    // return 'd' for double

And also in the Developer Docs:

Returns a C string containing the Objective-C type of the data contained in the
receiver, which for an NSDecimalNumber object is always “d” (for double).

CFNumberGetValue is documented to return false if the conversion was lossy. In the event of a lossy conversion, or when you encounter an NSDecimalNumber, you will want to fall back to using the stringValue and then use sqlite3_bind_text to bind it (and use sqlite's column affinity).

Something like this:

NSNumber *number = ...
BOOL ok = NO;

if (![number isKindOfClass:[NSDecimalNumber class]]) {
    CFNumberType numberType = CFNumberGetType(number);

    if (numberType == kCFNumberFloat32Type ||
        numberType == kCFNumberFloat64Type ||
        numberType == kCFNumberCGFloatType)
    {
        double value;
        ok = CFNumberGetValue(number, kCFNumberFloat64Type, &value);

        if (ok) {
            ok = (sqlite3_bind_double(pStmt, idx, value) == SQLITE_OK);
        }

    } else {
        SInt64 value;
        ok = CFNumberGetValue(number, kCFNumberSInt64Type, &value);

        if (ok) {
            ok = (sqlite3_bind_int64(pStmt, idx, value) == SQLITE_OK);
        }
    }
}

// We had an NSDecimalNumber, or the conversion via CFNumberGetValue() was lossy.
if (!ok) {
    NSString *stringValue = [number stringValue];
    ok = (sqlite3_bind_text(pStmt, idx, [stringValue UTF8String], -1, SQLITE_TRANSIENT) == SQLITE_OK);
}

Arquit

Answer 2

Simple answer: You can't.

In order to do what you're asking, you'll need to keep track of the exact type on your own. NSNumber is more of a "dumb" wrapper in that it helps you use standard numbers in a more objective way (as Obj-C objects). Using solely NSNumber, -objCType is your only way. If you want another way, you'd have to do it on your own.

Here are some other discussions that may be of help:

get type of NSNumber

What's the largest value an NSNumber can store?

Why is longLongValue returning the incorrect value

NSJSONSerialization unboxes NSNumber?

Subdominant

I think CFNumberGetType() is better since it is simple syntax known to all (no @encode). Both CFNumberGetType() and objCType have the same caveat of not being a guarantee of the type. – Broadcast

If you are creating the NSNumber objects yourself you can add type information to each number instance using objc_setAssociatedObject()/objc_getAssociatedObject() – Borchert

Please read my full answer. I noted that it cannot be done using solely NSNumber, and that he/she would have to implement it on their own. In this case, doing something like iccir suggested is definitely feasible. – Subdominant

Answer 3

NSJSONSerializer returns:

an integer NSNumber for integers up to 18 digits

an NSDecimalNumber for integers with 19 or more digits

a double NSNumber for numbers with decimals or exponent

a BOOL NSNumber for true and false.

Compare directly with the global variables kCFBooleanFalse and kCFBooleanTrue (spelling might be wrong) to find booleans. Check isKindOfClass:[NSDecimalNumber class] for decimal numbers; these are actually integers. Test

strcmp (number.objCType, @encode (double)) == 0

for double NSNumbers. This will unfortunately match NSDecimalNumber as well, so test that first.

Mews

Answer 4

Ok--It's not 100% ideal, but you add a little bit of code to SBJSON to achieve what you want.

1. First, add NSNumber+SBJson to the SBJSON project:

NSNumber+SBJson.h

@interface NSNumber (SBJson)
@property ( nonatomic ) BOOL isDouble ;
@end

NSNumber+SBJson.m

#import "NSNumber+SBJSON.h"
#import <objc/runtime.h>

@implementation NSNumber (SBJson)

static const char * kIsDoubleKey = "kIsDoubleKey" ;

-(void)setIsDouble:(BOOL)b
{
    objc_setAssociatedObject( self, kIsDoubleKey, [ NSNumber numberWithBool:b ], OBJC_ASSOCIATION_RETAIN_NONATOMIC ) ;
}

-(BOOL)isDouble
{
    return [ objc_getAssociatedObject( self, kIsDoubleKey ) boolValue ] ;
}

@end

2. Now, find the line in SBJson4StreamParser.m where sbjson4_token_real is handled. Change the code as follows:

case sbjson4_token_real: {
    NSNumber * number = @(strtod(token, NULL)) ;
    number.isDouble = YES ;
    [_delegate parserFoundNumber:number ];
    [_state parser:self shouldTransitionTo:tok];
    break;
}

note the bold line... this will mark a number created from a JSON real as a double.

3. Finally, you can check the isDouble property on your number objects decoded via SBJSON

HTH

edit:

(Of course you could generalize this and replace the added isDouble with a generic type indicator if you like)

Borchert

Answer 5

if ([data isKindOfClass: [NSNumber class]]) {
           NSNumber *num = (NSNumber *)data;
           if (strcmp([data objCType], @encode(float)) == 0) {
               return [NSString stringWithFormat:@"%0.1f} ",num.floatValue];
           } else if (strcmp([data objCType], @encode(double)) == 0) {
               return [NSString stringWithFormat:@"%0.1f} ",num.doubleValue];
           } else if (strcmp([data objCType], @encode(int)) == 0) {
               return [NSString stringWithFormat:@"%d} ",num.intValue];
           } else if (strcmp([data objCType], @encode(BOOL)) == 0) {
               return  num.boolValue ? @"Yes} " : @"No} ";
           } else if (strcmp([data objCType], @encode(long)) == 0) {
               return [NSString stringWithFormat:@"%ld} ",num.longValue];
           }
       } 

Piperine