Conditional unit testing based on iOS version

Question

My unit test suite contains a few test cases that are only applicable for the latest iOS version. Expectedly, these tests fail when running the suite on an older iOS version.

Is there a standard way to indicate that these test cases should run on a specific iOS version only?

One option I thought about:

- (void) testSomethingThatOnlyWorksOniOS7
{
    const BOOL iOS7OrHigher = floor(NSFoundationVersionNumber) > NSFoundationVersionNumber_iOS_6_1;
    if (!iOS7OrHigher) return;

    // Actual test goes here
}

Alternatively, could I tell SenTestCase to skip certain tests dynamically?

Unbeliever

Answer 1

Your approach seems fine to me.

Generally speaking, I do not think that there is something like a "standard way" to selectively execute unit tests based on iOS version.

On the one hand, your approach is the same that can be used to implement one feature selectively or in different ways according to iOS version. Possibly you already know about this discussion on CocoaWithLove: Tips and Tricks from conditional.... It's pretty old, but the approaches described here to hold still.

You do not specify the way your unit tests are executed, but the real cool way to handle this, IMO, would be to be able to specify which tests to execute outside of the implementation, so that you state which ones are for iOS7 only and do not pollute your test implementation.

This could done, e.g., through a version number associated to each test; before calling the unit test implementation you check the version number in the function calling, e.g., testSomethingThatOnlyWorksOniOS7.

EDIT:

Actually, things could be easier that what I thought in the first place. Now I am going a bit hacky...

You could modify the place in OCUnit where the actual test method call is done (no idea about this, sorry, but I do not think it should be hard to find out...) so that it checks on the selector name: you convert the selector name into a string (NSStringFromSelector) and if it matches some regular expression, you take some specific branch (which would simply be in your case, ignoring that test).

If you are worried by modifying OCUnit, which might not be sensible, what I said above could be done through method swizzling: only if the selector name does not match your expression, you call the original implementation, otherwise do nothing.

Laroche

"so that you state which ones are for iOS7 only and do not pollute your test implementation." Yes, I was hoping to find some obscure OCUnit feature that enabled me to do this. :) – Unbeliever

I am not aware of any... but I have just added some ideas to my answer... does not look that difficult to do... – Laroche

Maybe overriding defaultTestSuite could work. Worth checking out. – Unbeliever

Answer 2

The best way is to run respondsToSelector: on a method you know is only available on iOS 7. E.g.

[view respondsToSelector:@selector(snapshotViewAfterScreenUpdates:)]

or

[UIView instancesRespondToSelector:@selector(snapshotViewAfterScreenUpdates:)]

Claustral

Using respondsToSelector: is the best solution, cause it's future proof. Apple can remove a method anytime in the future and the version number check fails in this situation. – Augustinaaugustine

Not in this case. I want to explicitly mark that the test should not run on iOS 7. This is not app code. This is test code. – Unbeliever

You can still run that code in the test and return at the start of the test method. – Claustral

Answer 3

There is not a standard way of doing this, and generally speaking testing is not something the iOS community talks too much about. The way I would approach this is by putting specific iOS7 tests on a test target on its own and then basically decide to run that suite accordingly

Bonds

"and generally speaking testing is not something the iOS community talks too much about": that's why it's worth asking these questions on SO. ;) – Unbeliever

Sorry, I didn't mean to complain. Was to emphasise the fact that there is not an "industry standard", or at least one that I am aware of. – Bonds

No need to say sorry. I don't know who downvoted you but I just upvoted to compensate. :) Also, I think the target approach is a viable solution, albeit a little cumbersome. – Unbeliever

+1 I think targets is the way I'd do this too: have one target for each necessary iOS version and use an aggregate target to allow tests to be run with a single action. With Xcode 5 bots, you could have the iOS 6 test target run on an iOS 6 device, etc. – Afrikaans

Answer 4

I tried playing around with OCUnit as @sergio suggested but I couldn't get far away. So far I'm annotating version-specific tests with a macro. It looks like this:

#define SKIP_IF_VERSION(v) if (floor(NSFoundationVersionNumber) <= v) return;

- (void)testSomethingThatOnlyWorksOniOS7
{ SKIP_IF_VERSION(NSFoundationVersionNumber_iOS_6_1)
// Test goes here
}

What I don't like about this approach is that the test runs anyway (and doesn't do anything). I would prefer the test to be skipped entirely.

Unbeliever