SenTestingKit in Xcode 4: Asynchronous testing?

Question

I have been searching for a way to use SenTestingKit to do some integration testing between my client-side code and our server. I haven't had any luck. It seems that once the code is run in a method, the object gets destroyed. This means that any asynchronous responses never call the selectors.

Questions:

1. Is there a way to keep the object instantiated until such time as I see fit to destroy it - ie. after the tests have completed?
2. If not, how could I create a class that blocks (ie. acts synchronously) until the tests are completed?

FYI, I'm running a test server where I know the expected results.

I've done a fair bit of Googling but haven't seen proof one way or another about this. I'm sure others would be interested as well.

Eogene

The accepted answer can be viewed as "out of date". There are several good, new, answers. Might want to reconsider? – Ardrey

Answer 1

Two options:

• switch to GHUnit, which actually contains support for waiting for asynchronous events
• narrow the design of your tests so that you can test things as they stand. E.g. test that your controller code causes a selector to be detached and run, and (separately) test that this selector does what it ought. If both of those things work, then you can be confident that your controller detaches the correct work.

Regulable

Unfortunately, what I'm testing is the connect-encrypt-authenticate-getsomedata process, which can't be broken down without having something persistent to handle callbacks. SenTestingKit doesn't seem to give me the persistence that I need. But I will look at GHUnit. – Eogene

GHUnit seems like the best bet here. The examples page at gabriel.github.com/gh-unit/_examples.html shows exactly the situation I need to test for... time to learn a new tool! – Eogene

Answer 2

You can use a semaphore to wait until the asynchronous method finishes.

- (void)testBlockMethod {
    dispatch_semaphore_t semaphore = dispatch_semaphore_create(0);

    // Your block method eg. AFNetworking
    NSURL *url = [NSURL URLWithString:@"http://httpbin.org/ip"];
    NSURLRequest *request = [NSURLRequest requestWithURL:url];
    AFJSONRequestOperation *operation = [AFJSONRequestOperation JSONRequestOperationWithRequest:request success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON) {
        NSLog(@"IP Address: %@", [JSON valueForKeyPath:@"origin"]);
        STAssertNotNil(JSON, @"JSON not loaded");
        // Signal that block has completed
        dispatch_semaphore_signal(semaphore);
    } failure:nil];
    [operation start];

    // Run loop
    while (dispatch_semaphore_wait(semaphore, DISPATCH_TIME_NOW))
        [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
                                 beforeDate:[NSDate dateWithTimeIntervalSinceNow:10]];
    dispatch_release(semaphore);

}

http://samwize.com/2012/10/03/sentestingkit-does-not-support-wait-for-blocks/

Proteolysis

I did exactly as your code but it's never printing the log inside the success block. – Best

Brilliant till you consider the code you're calling probably doesn't dispatch a semaphore signal on completion. – Spelunker

Perfect for me as well. I saw on another SO thread a use of NSCondionalLocking which doesn't work well since it locks the main thread and prevent AFNetworking from dispatching the success/error callbacks. – Adorn

If your loop here and your callback are running on the same (main) thread then a flag like __block BOOL finished = NO and then set finished to true in the callback will work. The callback and the test are serialised on the main thread so there's no /need/ for a semaphore, like this: mattconnolly.wordpress.com/2012/05/29/… – Balakirev

Not that brilliant, actually. There are couple of issues, here's one: the run loop only returns when there was an event or when the timeout fires. That means in the worst case, that - even the delegate signals the semaphore, but there was no event which has been enqueued on that run loop and mode - the while loop will still wait up to 10 seconds until it checks the semaphore again. Another is, that you cannot have a timeout. Not that good for testing... – Scavenger

This is a really good situation for using a dispatch group instead of a semaphore. Also, if you know that the callback should be prompt, then the while loop should exit after the timeout and the test marked as a failure if the operation hasn't yet finished [successfully]. – Sine

Answer 3

Kiwi supports asynchronous testing. Kiwi is a Behavior Driven Development (BDD) library for iOS that extends SentTestingKit (OCUnit), so it's easy to set up & use.

Also, check out:

• iOS Tests/Specs TDD/BDD and Integration & Acceptance Testing.
• Testing asynchronous code on iOS with OCunit

Endotoxin

Kiwi looks interesting. Does it handle asynchronous callbacks? – Eogene

Answer 4

This project https://github.com/hfossli/AGAsyncTestHelper has a very convenient macro

WAIT_WHILE(<expression_to_evaluate>, <max_duration>);

Which ables you to write the test like so

- (void)testDoSomething {

    __block BOOL somethingIsDone = NO;

    [MyObject doSomethingAsyncThenRunCompletionBlockOnMainQueue:^{
        somethingIsDone = YES;
    }];

    WAIT_WHILE(!somethingIsDone, 1.0); 
    NSLog(@"This won't be reached until async job is done");
}

Ardrey

Answer 5

Checkout the SenTestingKitAsync project - https://github.com/nxtbgthng/SenTestingKitAsync. The related blog is here - http://www.objc.io/issue-2/async-testing.html

Mancilla