How can I expect multiple failures in google test?

Question

How can I expect multiple failures in google test? I use this when testing that asserts happen in my code under test. Because these asserts are not fatal, multiple can happen.

The following testcase reproduces this:

void failTwice()
{
   EXPECT_TRUE(false) << "fail first time";
   EXPECT_TRUE(false) << "fail second time";
}

TEST_F(FailureTest, testMultipleFails)
{
   EXPECT_NONFATAL_FAILURE(failTwice(), "time");
}

This produces the following output:

gtest/src/gtest.cc:657: Failure
Expected: 1 non-fatal failure
  Actual: 2 failures
FailureTest.h:20: Non-fatal failure:
Value of: false
  Actual: false
Expected: true
fail first time

FailureTest.h:20: Non-fatal failure:
Value of: false
  Actual: false
Expected: true
fail second time

The problem is this: Expected: 1 non-fatal failure

How can I tell google test to expect multiple failures?

Lat

What is the problem, exactly?! You get two failures from two EXPECTs. – Able

The problem is that the failures are intended, the test case verifies that they actually do fail (in case of asserts detect the failure). As you see in the code there are 3 EXPECT_ calls. The first says that I expect another call to fail. – Lat

Do you mean you want to specify how many failures EXPECT_NONFATAL_FAILURE() should get in order to pass? – Able

Yes, even though I don't really care about the amount. The problem is that my testcase fails because it is not precisely 1 failure. But I don't know how tell it to allow for 2 failures (or n failures). – Lat

Oh, I see. I should've read your question more thoroughly. – Able

Answer 1

I had the same problem and the one approach that works is:

EXPECT_NONFATAL_FAILURE({
    EXPECT_NONFATAL_FAILURE(failTwice(), "");
},"Actual: 2");

With the "Actual: 2" I set the expectation that 2 non-fatal failures will occur. One drawback is that it is not easy to tell which error messages you are expecting.

Caerleon

I created a macro based on this answer for my purposes: #define EXPECT_NONFATAL_FAILURES(statement_, n_) EXPECT_NONFATAL_FAILURE({EXPECT_NONFATAL_FAILURE(statement_, "");}, "Actual: " #n_) – Vertigo

Answer 2

This is the solution we came up with, which is fairly general but covered most of our cases:

//adapted from EXPECT_FATAL_FAILURE
do {
  //capture all expect failures in test result array
  ::testing::TestPartResultArray gtest_failures;
  //run method in its own scope
  ::testing::ScopedFakeTestPartResultReporter gtest_reporter(
    ::testing::ScopedFakeTestPartResultReporter::
    INTERCEPT_ONLY_CURRENT_THREAD, &gtest_failures);
  //run your method
  failTwice();
  //only check on number, this will include fatal and nonfatal, but if you just care about number then this is sufficient
  ASSERT_EQ(gtest_failures.size(), 2) << "Comparison did not fail FATAL/NONFATAL twice";
} while (::testing::internal::AlwaysFalse());

Arjun