What are the advantages of XCTest over SenTestingKit?

Question

I want to do unit testing in iOS. By default test classes are created using XCTest framework classes in Xcode 5. I have used SenTestingKit in earlier versions of iOS. What are the differences between these two frameworks? What are the advantages of XCTest framework. I googled for related documentation but I did not find any. Only thing I found is the interface classes to use inside the kit. Can anyone point me to the related resources.

Flocculent

Answer 1

Apple's documentation is notably lacking with regards to testing. It's a shame, because spending five minutes with XCTest made me fall for it. So, here are some docs that you may find useful.

First, stop and watch the WWDC video about testing: https://developer.apple.com/wwdc/videos/?include=409#409 (must be a registered developer)

Then, read about Bots. They're the real magic in XCTest. I'm currently setting up an environment of a Mac Mini running Mavericks Server connected to a small array of iPod Touches, iPhones and iPads that will simultaneously run my unit tests and report back. It's damn near magic.

https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/000-About_Continuous_Integration/about_continuous_integration.html#//apple_ref/doc/uid/TP40013292

All that said, I don't know SenTest well enough to compare the two. But I am learning XCTest, so here are a few thoughts on what I'm really digging about it so far.

• It's already there. There's nothing to download, and you can make your test class objects at the same time as you make that which you wish to test. Rad.
• Because it's integrated so deeply with Xcode, you can see test coverage in the gutter. You can also use Companion View to see the testers from source, or the source from tester methods. Also, it will always be up to date - no waiting to fix integration with Xcode because Apple "broke" something.
• Continuous Integration. The bots are just magical, and they're so easy. You also get Git hosting, which is a small fringe perk, but more so if you prefer your Git internally hosted but don't want to pay for GitHub Enterprise.

And, a few cons for measure:

• Lack of Documentation. Asking me to read the headers for a list of assertions is a ballsy move. I have no doubt Apple will fix this soon, but not soon enough.
• You are tied to Xcode a little more deeply. Not really a con for development, but scripting XCTest isn't as elegant as other options may be.
• Continuous Integration requires OS X Server. It can be on the same machine if you really want, but it's $20 to buy, even if you owned Mountain Lion Server. I am not at all bitter about this one (ha).

My two cents. Hope it helps. I'd really recommend watching that video I linked to - they do live test creation and make a really compelling case for unit testing in general, regardless of if you use XCTest.

Altitude

Are these benefits provided by XCTest specifically or are they just part of the generally improved testing features of Xcode 5? Many (all?) of the things you mentioned also work with SenTestingKit with Xcode 5. – Retinol

As I mentioned, I'm not familiar with SenTestingKit, so I can't speak to the differences. The CI stuff is new to Xcode 5, but the rest of it is an extension of OCUnit, which was provided by Xcode prior to 5. I didn't do much with OCUnit, either, so I'm afraid I'm at a loss again as to differences. – Altitude

I do know that I could not breakpoint my test code with SenTestingKit. Using XCTest I could breakpoint – Arabian

409 video is not about testing, it's about debugging. – Ammonate

Answer 2

It seems the advantages are presently few, but that will hopefully change over time, so starting a community wiki to document some:

• The XCTAssert… family of macros can handle you leaving out the format parameter (used for commenting why a test should pass) entirely. This is handy for keeping simple tests (where it's obvious from the test itself what it's doing) clean in the editor. STAssert… required us to make the format parameter nil or some such instead.
• There's a generic XCTAssert() macro, which feels neater than bodging some types of check into STAssertTrue.

Wsan

Answer 3

XCTest are included by default with ios7 and integrate better with the new continuous integration functionality.
https://developer.apple.com/technologies/tools/

Guanabara

Yeah.. Info there is not sufficient. – Flocculent