XCTAssertEqual error: ("3") is not equal to ("3")

Question

NSMutableArray *arr = [NSMutableArray array];
[arr addObject:@"1"];
[arr addObject:@"2"];
[arr addObject:@"3"];

// This statement is fine.
XCTAssertTrue(arr.count == 3, @"Wrong array size.");

// This assertion fails with an error: ((arr.count) equal to (3)) failed: ("3") is not equal to ("3")
XCTAssertEqual(arr.count, 3, @"Wrong array size.");

What don't I understand about XCTAssertEqual? Why does the last assertion fail?

Morn

Some other great matcher libraries are: OCHamcrest and Expecta. . there's also Kiwi and Cedar - fully fledged test frameworks with nice inbuilt matcher libraries. . (Just in case you haven't tried these yet). – Pinzler

Answer 1

I have also had quite a bit of trouble with Xcode 5's tests. It still seems quite buggy with some odd behaviour - however I have found the definitive reason your particular XCTAssertEqual isn't working.

If we have a look at the test code we see it actually does the following (taken directly from XCTestsAssertionsImpl.h - it may be easier to view there):

#define _XCTPrimitiveAssertEqual(a1, a2, format...) \
({ \
    @try { \
        __typeof__(a1) a1value = (a1); \
        __typeof__(a2) a2value = (a2); \
        NSValue *a1encoded = [NSValue value:&a1value withObjCType:@encode(__typeof__(a1))]; \
        NSValue *a2encoded = [NSValue value:&a2value withObjCType:@encode(__typeof__(a2))]; \
        float aNaN = NAN; \
        NSValue *aNaNencoded = [NSValue value:&aNaN withObjCType:@encode(__typeof__(aNaN))]; \
        if ([a1encoded isEqualToValue:aNaNencoded] || [a2encoded isEqualToValue:aNaNencoded] || ![a1encoded isEqualToValue:a2encoded]) { \
                _XCTRegisterFailure(_XCTFailureDescription(_XCTAssertion_Equal, 0, @#a1, @#a2, _XCTDescriptionForValue(a1encoded), _XCTDescriptionForValue(a2encoded)),format); \
        } \
    } \
    @catch (id exception) { \
        _XCTRegisterFailure(_XCTFailureDescription(_XCTAssertion_Equal, 1, @#a1, @#a2, [exception reason]),format); \
    }\
})

Here's the problem:

What the test is actually doing is encoding the values into an NSValue and then comparing them. "Okay," you say, "but what's the problem with that?" I didn't think there was one either until I made my own test case for it. The problem is that NSValue's -isEqualToValue must also compare the NSValue's encoding type as well as its actual value. Both must be equal for the method to return YES.

In your case, arr.count is an NSUInteger which is a typedef of unsigned int. The compile-time constant 3 presumably degenerates into a signed int at runtime. Thus when the two are put into an NSValue object, their encoding types are not equal and thus the two CANNOT be equal according to -[NSValue isEqualToValue].

You can prove this with a custom example. The following code explicitly does exactly what XCTAssertEqual does:

// Note explicit types
unsigned int a1 = 3;
signed int a2 = 3;

__typeof__(a1) a1value = (a1);
__typeof__(a2) a2value = (a2);

NSValue *a1encoded = [NSValue value:&a1value withObjCType:@encode(__typeof__(a1))];
NSValue *a2encoded = [NSValue value:&a2value withObjCType:@encode(__typeof__(a2))];

if (![a1encoded isEqualToValue:a2encoded]) {
    NSLog(@"3 != 3 :(");
}

"3 != 3 :(" will appear in the log every time.

I hasten to add here that this is, in fact, expected behaviour. NSValue is supposed to check its type encoding when doing comparisons. Unfortunately it's just not what we were expecting when testing two ('equal') integers.

XCTAssertTrue, incidentally, has much more straightforward logic, and behaves generally as expected (again, see the actual source for how it determines whether the assertion fails).

Marek

It's worth noting that the correct fix for this is to simply include the type information. XCTAssertEqual(arr.count, (NSUInteger)3, @"Wrong array size."); – Unconquerable

Thanks, an even easier way is: XCTAssertEqual(arr.count, 3U, @"Wrong array size."); – Stertor

Better to use (NSUInteger)3 rather than 3U because NSUInteger is typedef'ed differently for 64bit and 32bit compilation. For 64bit, NSUInteger is an unsigned long versus unsigned int for 32bit. – Ruthenian

Or use XCTAssertEqualWithAccuracy(arr.count, 3, 0.000000001); which I learned from an answer given by @WayneHartman in link – Villa

Answer 2

I have had this problem, too. As @ephemera and @napier indicated, this is a type issue.

It can be solved by supplying a value of the correct type, using the c-literal modifiers.

XCTAssertEqual(arr.count, 3ul, @"Wrong array size.");

You can find the correct type by looking up the return type of the function used on the left hand side - ALT-click on arr.count:

- (NSUInteger)count;

Now ALT-click on NSUInteger to find its type:

typedef unsigned long NSUInteger;

Now find the c literal numeric format for unsigned long - google is a good friend but this page works:

http://www.tutorialspoint.com/cprogramming/c_constants.htm

As a quick hint here, you may need to use U (unsigned) L (long) or F (float), and make sure you write 1.0 instead of 1 to get a double. Lowercase also works, as in my example above.

Dissipate

I don't think this works if you want your tests to work on both 32 and 64 bit. Using 3ul will cause a failure with 32 bit. – Kopans

Answer 3

In case someone else is looking for the issue lead by double comparing like me (solution above won't work for float & double), try:

XCTAssertEqualWithAccuracy(number.doubleValue, 12.34, 0.01);

Generates a failure when (difference between (\a expression1) and (\a expression2) is > (\a accuracy))).

Dealate

Answer 4

One alternative is to just use casting:

XCTAssertEqual(arr.count, (NSUInteger)3, @"Wrong array size.");

It might be the best solution with the current state of the tools, especially if you have code where you're using XCTAssertEqual a lot and don't want to switch to XCTAssertTrue.

(I noticed @RobNapier made this suggestion in a comment.)

Kopans

Answer 5

I got snagged by this issue as well, very thankful for the workarounds provided here. Quick FYI, it seems this was fixed in Xcode 5.1 release.

https://developer.apple.com/library/mac/releasenotes/DeveloperTools/RN-Xcode/xc5_release_notes/xc5_release_notes.html

The XCTAssertEqual macro (formerly STAssertEquals using OCUnit) correctly compares scalar values of different types without casting, for example, int and NSInteger. It can no longer accept nonscalar types, such as structs, for comparison. (14435933)

I have not yet upgraded from Xcode 5.0.2 but my colleague has, and the same XC tests that were previously failing due to this problem are now passing without the casting workaround.

Rating