How do I unit test for EXC_BAD_ACCESS?

Question

I know how to resolve EXC_BAD_ACCESS issues, but I'm not sure how to unit test for it. Is there a way to capture EXC_BAD_ACCESS in code instead of simply crashing?

Here's why I ask: I have written a library that heavily uses blocks, like this:

- (void)doSomething:(void (^)())myBlock;

In my implementation of doSomething: I'm going to eventually run the block, like this:

myBlock();

If a caller passes nil for the block, then it will crash with EXC_BAD_ACCESS, so the solution is to check that the block exists, like this:

if (myBlock) {
    myBlock();
}

This nil check is pretty easy to forget, so I'd like a way to write a unit test that fails when the crash occurs. I suppose a crash could be considered a test failure, but I think it would be nicer for others trying to run the tests to see a nice failure message rather than a crash. Any ideas?

Limbic

Answer 1

I think you'll need to run the test in a subprocess; then you can let the subprocess crash, check for that crash, and fail the test neatly if it occurs.

Working from Peter Hosey's singleton test code.

- (void) runTestInSubprocess:(SEL)testCmd {
        pid_t pid = fork();
        // The return value of fork is 0 in the child process, and it is
        // the id of the child process in the parent process.
        if (pid == 0) {
            // Child process: run test
            // isInSubprocess is an ivar of your test case class
            isInSubprocess = YES;
            [self performSelector:testCmd];
            exit(0);
        } else {
            // Parent process: wait for child process to end, check 
            // its status
            int status;
            waitpid(pid, &status, /*options*/ 0);
            // This was a crash; fail the test
            STAssertFalse(WIFSIGNALED(status), @"Test %@ crashed due to signal %d", NSStringFromSelector(testCmd), WTERMSIG(status));
        }
}

Each test will then run itself in a subprocess like so:

- (void) testSomething {
    if (!isInSubprocess) {
            // Hand off this test's selector to be run in a subprocess
            [self runTestInSubprocess:_cmd];
            return;
    }

    // Put actual test code here
    STAssertEquals(1, 1, @"Something wrong with the universe.");

}

You may need to tweak this; I haven't tested it.

Floats

This is very interesting. At first attempt, it ends up failing the test no matter what I do, but the project is targeted as a Cocoa Touch static library, and running the tests launches the iPhone Simulator. I don't think fork() is going to work there, so I'm going to try this out as a Mac / Cocoa target as well and see what happens. Thanks! – Limbic

Ooh, yeah, I don't know if fork() is even available on iOS, sorry. I'm having trouble getting this to work completely right, too; I'll update if I figure anything out. – Floats

@Limbic Did you get a chance to work this out successfully on iOS? I'm also looking for a way to make a test fail when a crash occurs, especially in Swift. – Tootsy

Answer 2

I would suggest using one of the assertion macros found in the Assertions and Logging Programming Guide

So you could do something like:

NSAssert(myBlock != nil, @"myBlock must not be nil")

This enforces the preconditions that must be met before the method continues executing. It also allows the app to crash and will give you a reason why other than EXEC_BAD_ACCESS.

Cog

This doesn't really help with the unit testing; greenisus could equally well include the if(myBlock){ myBlock() }; check that he mentioned in the question. The problem, unless I've completely misunderstood, is how to write a unit test that catches forgetting to check the block in the code under test. – Floats

Josh is right. I could very well use that assertion, but what I'm really trying to test is that my methods don't crash when I pass them a nil block, so that it doesn't matter whether or not myBlock is nil. – Limbic

By using the assertion, you are guaranteeing that myBlock is not nil. If you were to test that method with a unit test, the test would fail due to the assertion failing. – Cog

That's not what I'm asking. I want nil to be an acceptable value for myBlock, so I'm looking for a test that makes sure passing nil doesn't cause a crash. – Limbic

Gotcha. I didn't realize that. – Cog