Why does a false assertion in async test in GHUnit crash the app instead of just failing the test?

Question

This question has very few views and no answers yet. If you have a suggestion what to change about this question to get more eyeballs, I'd be happy to hear them. Cheers!

I'm using GHAsyncTestCase to test a custom NSOperation of mine. I'm setting the test case as a delegate on the operation object and I'm calling didFinishAsyncOperation on the main thread when it's done.

When an assertion fails it throws an exception, which ought to be caught by the test case to render the test as "failed". But instead of this expected behavior, my app get's aborted by Xcode as soon as the assertion fails.

*** Terminating app due to uncaught exception 'GHTestFailureException', reason: ''NO' should be TRUE. This should trigger a failed test, but crashes my app instead.'

I'm obviously doing something wrong. Who can tell me?

@interface TestServiceAPI : GHAsyncTestCase
@end

@implementation TestServiceAPI

    - (BOOL)shouldRunOnMainThread
    {
        return YES;
    }

    - (void)testAsyncOperation
    {
        [self prepare];

        MyOperation *op = [[[MyOperation alloc] init] autorelease];

        op.delegate = self; // delegate method is called on the main thread.

        [self.operationQueue addOperation:op];

        [self waitForStatus:kGHUnitWaitStatusSuccess timeout:1.0];
    }

    - (void)didFinishAsyncOperation
    {
        GHAssertTrue(NO, @"This should trigger a failed test, but crashes my app instead.");

        [self notify:kGHUnitWaitStatusSuccess forSelector:@selector(testAsyncOperation)];
    }

@end

Severson

I'm guessing the exception's going off in a context (different thread, eg) where there's no exception handler present to catch it. – Taxable

Yes, that's what I'm guessing too. But this must be a standard issue scenario for async tests, right? What should I do to fix it? – Severson

I'm also getting the same exception. But I use GHTestCase. When a test case failed, should it crash the app too? – Ockham

No it shouldn't, the reason your app crashes is that the test suite's exception handlers don't catch your exception. Refactor your tests to not break out of the test methods. I would advise you to start using XCTest by now, or even Kiwi. – Severson

Answer 1

I've been digging for a week to find a solution to this when I finally caught a break. It's been a bit weird having next to no views on a bounty question and no one cared to attempt an answer. I was thinking the question might be stupid, but there were no downvotes and no one cared to correct it either. Has StackOverflow become that saturated?

A solution.

The trick is to not assert anything from the callback method, but put the assertions back in the original test. The wait method is actually blocking the thread, which I didn't think of before. If your async callback receives any values, just store them in an ivar or property and then make assertions based on them in your original test method.

This takes care of the assertions not causing any crashes.

- (void)testAsyncOperation
{
    [self prepare];

    MyOperation *op = [[[MyOperation alloc] init] autorelease];

    op.delegate = self; // delegate method is called on the main thread.

    [self.operationQueue addOperation:op];

    // The `waitfForStatus:timeout` method will block this thread.
    [self waitForStatus:kGHUnitWaitStatusSuccess timeout:1.0];

    // And after the callback finishes, it continues here.
    GHAssertTrue(NO, @"This triggers a failed test without anything crashing.");
}

- (void)didFinishAsyncOperation
{
    [self notify:kGHUnitWaitStatusSuccess forSelector:@selector(testAsyncOperation)];
}

Severson

Awesome! Thanks for this (even though it was asked awhile ago) – Panhellenic

Glad it was of help! One's gotta love stack overflow for archiving like this :) – Severson

You are savior my friend, how I couldn't think of it, was struggling two days with that, but I think GHUnit should handle it more appropriately. I couldn't assert a boolean value inside a block code. This fixed my logic. Though I don't like so much using __block variables, it works. – Enaenable

Best comment ever :D. Might I ask, why are you 'still' using GHUnit, instead of XCTest? – Severson

Answer 2

Look up your Xcode Breakpoints navigator,delete all exception breakpoints,That's all !!!

Catamnesis

Answer 3

Looking at the header files for GHUnit, it looks like that may be what is supposed to happen with your code. A subclass of GHUnit could override this method:

// Override any exceptions; By default exceptions are raised, causing a test failure
- (void)failWithException:(NSException *)exception { }

To not throw exceptions, but an easier solution is to use the GHAssertTrueNoThrow instead of GHAssertTrue macro.

Crystalloid

In my actual tests, I'm using OCHamcrest to write the assertions, because the GHAssert* syntax does not match the way I like to read & write assertions that well. Thanks for the failWithException: suggestion, but I think I'm happy with my own solution for now. – Severson

Answer 4

I think this question should be rather, "how to test methods with blocks in GHUnit"?

And the answer can be found here: http://samwize.com/2012/11/25/create-async-test-with-ghunit/

California