Unit testing and TDD, OCUnit vs Google Tool Box

Question

I'm trying to choose between OCUnit and Google Tool Box, do you have any preferences, would recommend one or the other, why ? I would be very interested to hear about your experiences with any of the 2.

The main problem i have with both of them is the managment of crashes in tested methods (ex: BAD ACCESS) None of them was able to tell me in what class the crash occured !!!

With Google Tool Box i can see which test suite was being run but not the test case (how are you supposed to do when your test suite has 50 test cases ?)

With OCUnit i can at least see what test case in what test suite caused the crash.

Here is the kind of message i have with GTB :

Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds

Test Suite 'LogicTests' started at 2009-12-14 18:03:15 +0100

 /Users/admin/Documents/Tests/GTBTest/RunIPhoneUnitTest.sh: line 122:   688    Segmentation fault      "$TARGET_BUILD_DIR/$EXECUTABLE_PATH" -RegisterForSystemEvents

 Command /bin/sh failed with exit code 139

I can see that it's it the test suite 'LogicTests' that originated the crash but that's all.

With OCunit here is the message for the same error :

Test Suite 'LogicTests' started at 2009-12-14 17:51:26 +0100
Test Case '-[LogicTests testFail]' started.
/Developer/Tools/RunPlatformUnitTests.include: line 415:   536 Segmentation fault      "${THIN_TEST_RIG}" "${OTHER_TEST_FLAGS}" "${TEST_BUNDLE_PATH}"

At least with OCUnit i can track what test case was being run and eventually debug it (but that could take a very long time without any class and line number info...)

How do you deal with these problems ?

Thanks in advance.

PS: here is how to reproduce the problem, it's very simple :

Just create a class with a method that crashes when it's called (which happens all the time when you're doing TDD) :

- (void) crashMethod {
 NSMutableArray *crashArray;
 [crashArray addObject:[NSObject new]];
}

And then create a test case that calls this methods :

- (void) testFail {
    ClassToTest *test = [[ClassToTest alloc] init];
 [test crashMethod];
 [test release];
 }

Thanks in advance, Vincent

Rockfish

Answer 1

I think i'll go with GTB anyway..

With xCode 3.2 OCUnit errors and warnings are not showing up inside the code. Seems it's a know issue : lhttp://osdir.com/ml/xcode-users/2009-10/msg00216.html

With GTB it works fine. I cant believe it but it seems GTB is better integrated with newer versions of xCode than OCUnit....

Debugging of unit tests doesn't require anything, it works great from the start. (with xCode you need a bunch of settings : http://chanson.livejournal.com/119578.html

With GTB you can run your tests on the device and you have tools for UI testing (seems you can create a fake UIView hierarchy and then compare it with what you have at runtime). I'm skeptical about UI automatic testing (expensive and hard to maintain) but that's a nice feature !

http://code.google.com/p/google-toolbox-for-mac/wiki/CodeVerificationAndUnitTesting

Rockfish

@user142764: how do you debug unit tests with GTB then? Surely you still need a custom executable? – Gilgamesh

No that's what is great with GTB: since you use a normal target to run your tests, you can debug it ! Just write a test, put a breakpoint inside, then just do Run->Debug from the menu and there it is : you are debugging your test step by step. – Rockfish

Answer 2

BTW the Google Toolbox now prints Test case started messages in case anyone was wondering ;-)

Kenishakenison