What if XCTestExpectation is unexpected

Question

I'm writing an XCTest unit test in Swift. The idea is that a callback mustn't be called in a certain case.

So what I do, is

func testThatCallbackIsNotFired() {

    let expectation = expectationWithDescription("A callback is fired")
    // configure an async operation

    asyncOperation.run() { (_) -> () in
        expectation.fulfill()    // if this happens, the test must fail
    }

    waitForExpectationsWithTimeout(1) { (error: NSError?) -> Void in

        // here I expect error to be not nil,
        // which would signalize that expectation is not fulfilled,
        // which is what I expect, because callback mustn't be called
        XCTAssert(error != nil, "A callback mustn't be fired")
    }
}

When the callback is called, everything works fine: it fails with a message "A callback mustn't be fired" which is exactly what I need.

But if expectation hasn't been fulfilled, it fails and says

Asynchronous wait failed: Exceeded timeout of 1 seconds, with unfulfilled expectations: "Callback is fired".

Since a not fulfilled expectation is what I need, I don't want to have a failed test.

Do you have any suggestions what can I do to avoid this? Or, maybe, I can reach my goal in a different way? Thanks.

Literalism

It sounds like you might need to rethink the code you are testing. If the code runs asynchronously, I probably want a callback when it's done, regardless of whether or not there was an error. – Pedicab

With that said, if your goal is to only call the callback when there is an error and your test only cares whether or not error is nil, and you're writing Swift code, then why is the error an optional at all? You can assure it's never nil at compile time by simply making it non-optional. – Pedicab

Thanks for the comment. Maybe, you're right. Let me explain the purpose. I'm working on an image cache. The logic is: if an image is already in a memory, it returns it immediately (callback is not needed). Otherwise - you get the image (or an error) from the callback. Does it make sense? – Literalism

Even if it's in memory and isn't actually asynchronous, as a user, I still probably prefer getting it in the callback. It makes using it more simple. I will always get the image in the same place: in the callback. The only difference is how long it takes to get that image. – Pedicab

@Pedicab you're probably right. A single way for a response delivery is much better. I just need to investigate how would it influence the performance of table and collection views. But the question remains, I'd still like to know how to check that a handler is NOT called. – Literalism

I understand your question. I'm not suggesting you close this question or anything. It's just that I really don't think you should be writing code for which you actually care about the answer to what you're asking. Effectively, for every method you write that accepts a closure, you should be verifying that the closure IS called. – Pedicab

I've found a better example. It's AFNetworking. Every single request contains a 'success' and a 'failure' block. So you probably want to test that only one of them is called. – Literalism

Having separate success and failure blocks isn't what I'd consider best practice. Even the people who wrote AFNetworking probably think this way, as mattt is one of the primary contributors to Alamofire, the Swift successor to AFNetworking, which opts for a single completion block rather than separate success & failure blocks. – Pedicab

With that said, I'm sure that AFNetworking is a well-tested library and you could probably check their unit testing to determine what, if anything, they're doing to verify that only one of the two blocks is called. – Pedicab

They don't check this. – Literalism

Answer 1

I had this same problem, and I am annoyed that you can't use a handler to override the timeout fail of waitForExpectationsWithTimeout. Here is how I solved it (Swift 2 syntax):

func testThatCallbackIsNotFired() {
    expectationForPredicate(NSPredicate{(_, _) in
        struct Holder {static let startTime = CACurrentMediaTime()}

        if checkSomehowThatCallbackFired() {
            XCTFail("Callback fired when it shouldn't have.")
            return true
        }

        return Holder.startTime.distanceTo(CACurrentMediaTime()) > 1.0 // or however long you want to wait
        }, evaluatedWithObject: self, handler: nil)
    waitForExpectationsWithTimeout(2.0 /*longer than wait time above*/, handler: nil)
}

Revest

Answer 2

Use isInverted like in this post https://www.swiftbysundell.com/posts/unit-testing-asynchronous-swift-code

class DebouncerTests: XCTestCase {
    func testPreviousClosureCancelled() {
        let debouncer = Debouncer(delay: 0.25)

        // Expectation for the closure we'e expecting to be cancelled
        let cancelExpectation = expectation(description: "Cancel")
        cancelExpectation.isInverted = true

        // Expectation for the closure we're expecting to be completed
        let completedExpectation = expectation(description: "Completed")

        debouncer.schedule {
            cancelExpectation.fulfill()
        }

        // When we schedule a new closure, the previous one should be cancelled
        debouncer.schedule {
            completedExpectation.fulfill()
        }

        // We add an extra 0.05 seconds to reduce the risk for flakiness
        waitForExpectations(timeout: 0.3, handler: nil)
    }
}

Laurence