Test a specific exception type is thrown AND the exception has the right properties

Question

I want to test that MyException is thrown in a certain case. EXPECT_THROW is good here. But I also want to check the exception has a specific state e.g e.msg() == "Cucumber overflow".

How is this best implemented in GTest?

Hydrosol

Answer 1

I mostly second Lilshieste's answer but would add that you also should verify that the wrong exception type is not thrown:

#include <stdexcept>
#include "gtest/gtest.h"

struct foo
{
    int bar(int i) {
        if (i > 100) {
            throw std::out_of_range("Out of range");
        }
        return i;
    }
};

TEST(foo_test,out_of_range)
{
    foo f;
    try {
        f.bar(111);
        FAIL() << "Expected std::out_of_range";
    }
    catch(std::out_of_range const & err) {
        EXPECT_EQ(err.what(),std::string("Out of range"));
    }
    catch(...) {
        FAIL() << "Expected std::out_of_range";
    }
}

int main(int argc, char **argv) {
  ::testing::InitGoogleTest(&argc, argv);
  return RUN_ALL_TESTS();
}

Leafstalk

Functionally correct, but onerous/long. For one test, it's not alot of code. But, it is alot of code if I have to write this many times ... as one often does in test code. Defining a macro is a better approach. OP asks for 'best' which may be hard to achieve, but I think there are better ways. – Alberic

I don't know if this changed on some gtest version, but the catch(...) is not necessary to make the test fail. Ending a test with an exception is a test failure. – Southard

Answer 2

A colleague came up with the solution by just re-throwing the exception.

The knack: no need of extra FAIL() statements, just the two EXPECT... calls that test the bits you actually want: the exception as such and its value.

TEST(Exception, HasCertainMessage )
{
    // this tests _that_ the expected exception is thrown
    EXPECT_THROW({
        try
        {
            thisShallThrow();
        }
        catch( const MyException& e )
        {
            // and this tests that it has the correct message
            EXPECT_STREQ( "Cucumber overflow", e.what() );
            throw;
        }
    }, MyException );
}

Tonitonia

This should be the selected answer since it cleanly tests both the correct type and message using two simple EXPECT macros. – Cayuse

I think this is the best answer. Readable and doesn't introduce more macros. – Vicinage

The failure message becomes quite ugly, as it prints the whole statement. – Talishatalisman

Functionally correct, but onerous/long. For one test, it's not alot of code. But, it is alot of code if I have to write this many times ... as one often does in test code. Defining a macro is a better approach. – Alberic

Answer 3

I had previously offered a macro to solve this in an older answer. However time has passed and a new feature was added to GTest, which allows for this without macros.

The feature is a set of matchers, e.g., Throws that can be used in combination with EXPECT_THAT(). However the documentation does not seem to have been updated, so the only information is hidden in this GitHub issue.

---

The feature is used like this:

EXPECT_THAT([]() { throw std::runtime_error("message"); },
    Throws<std::runtime_error>());

EXPECT_THAT([]() { throw std::runtime_error("message"); },
    ThrowsMessage<std::runtime_error>(HasSubstr("message")));

EXPECT_THAT([]() { throw std::runtime_error("message"); },
    ThrowsMessageHasSubstr<std::runtime_error>("message"));

EXPECT_THAT([]() { throw std::runtime_error("message"); },
    Throws<std::runtime_error>(Property(&std::runtime_error::what,
         HasSubstr("message"))));

Note that due to how EXPECT_THAT() works you need to put the throwing statement into something invokable without arguments. Hence the lambdas in the examples above.

---

Edit: This feature is included beginning with version 1.11.

Also note that this feature is not included in version 1.10, but it has been merged into master. Because GTest follows abseil's live at head policy there is are no new versions planned at the moment. Also they don't seem to follow abseil's policy to release specific versions for those of use who can't/won't live at head.

Beluga

This is better than all other answers IMHO since it is concise and checks both the exception and its message. Just note that this requires using Gmock (gmock/gmock.h), and not just pure GTest(gtest/gtest.h). – Mandrill

I absolutely agree that this is the cleaner way than my solution from 2016 (77 likes, yay), since it uses direct commands and does not need any workaround. Thanks for sharing. A simple to use macro that reduces the syntax overhead in GTEST would be nice, though. I guess the new philosophy is to stay flexible, with the downside of needing "Throws", "Property", "HasSubstr" etc. – Tonitonia

Unfortunately this open bug makes usage very limited github.com/google/googletest/issues/4073 – Arruda

Answer 4

Jeff Langr describes a good approach in his book, Modern C++ Programming with Test-Driven Development:

If your [testing] framework does not support a single-line declarative assert that ensures an exception is thrown, you can use the following structure in your test:

    TEST(ATweet, RequiresUserNameToStartWithAnAtSign) {
        string invalidUser("notStartingWith@");
        try {
            Tweet tweet("msg", invalidUser);
            FAIL();
        }
        catch(const InvalidUserException& expected) {}
    }

[...] You might also need to use the try-catch structure if you must verify any postconditions after the exception is thrown. For example, you may want to verify the text associated with the thrown exception object.

    TEST(ATweet, RequiresUserNameToStartWithAtSign) {
        string invalidUser("notStartingWith@");
        try {
            Tweet tweet("msg", invalidUser);
            FAIL();
        }
        catch(const InvalidUserException& expected) {
            ASSERT_STREQ("notStartingWith@", expected.what());
        }
    }

(p.95)

This is the approach I've used, and have seen in practice elsewhere.

Edit: As has been pointed out by @MikeKinghan, this doesn't quite match the functionality provided by EXPECT_THROW; the test doesn't fail if the wrong exception is thrown. An additional catch clause could be added to address this:

catch(...) {
    FAIL();
}

Spiller

It doesn't quite test that the exception thrown is of the correct type though - that's where GTest's EXPECT_THROW is nice. I guess extending to add a catch(...){ FAIL(); } is an option? – Hydrosol

Thanks, that's a really good point. I'll update my answer to include this detail. – Spiller

Answer 5

A new feature was added to GTest master on 2020-08-24 (post v1.10) which I explained in a separate answer. However I'll leave this answer because it still helps if the version you're using doesn't support the new feature. Also this and similar solution is a bit easier to use.

---

Also see Bryant's solution which improves on this one. (no longer an exception_ptr polluting your scope)

---

As I need to do several of such tests I wrote a macro that basically includes Mike Kinghan's answer but "removes" all the boilerplate code:

#define ASSERT_THROW_KEEP_AS_E(statement, expected_exception) \
    std::exception_ptr _exceptionPtr; \
    try \
    { \
        (statement);\
        FAIL() << "Expected: " #statement " throws an exception of type " \
          #expected_exception ".\n  Actual: it throws nothing."; \
    } \
    catch (expected_exception const &) \
    { \
        _exceptionPtr = std::current_exception(); \
    } \
    catch (...) \
    { \
        FAIL() << "Expected: " #statement " throws an exception of type " \
          #expected_exception ".\n  Actual: it throws a different type."; \
    } \
    try \
    { \
        std::rethrow_exception(_exceptionPtr); \
    } \
    catch (expected_exception const & e)

Usage:

ASSERT_THROW_KEEP_AS_E(foo(), MyException)
{
    ASSERT_STREQ("Cucumber overflow", e.msg());
}

Caveats:

• As the macro defines a variable in the current scope, so it can only be used once.
• C++11 is needed for std::exception_ptr

Beluga

Now we're cookin with gas. This covers all the bases. ... and other sayings of the like. It's a macro so I don't have to re-invent the wheel in each test. It verifies the exception type (a must). It allows for full control of testing the exception state -- not just that the message matches a string exactly. – Alberic

@Alberic Note that a new feature was added to GTest master. Check my new answer for more information. – Beluga

I still find this a more concise and all around better user experience than the new gtest features. However, I dealt with the caveats (and added some other quality of life things) and added a new answer. I didn't want to add another answer, but there's no way to add all of it to just a comment: https://mcmap.net/q/258156/-test-a-specific-exception-type-is-thrown-and-the-exception-has-the-right-properties – Deter

Answer 6

I recommend defining a new macro based on Mike Kinghan's approach.

#define ASSERT_EXCEPTION( TRY_BLOCK, EXCEPTION_TYPE, MESSAGE )        \
try                                                                   \
{                                                                     \
    TRY_BLOCK                                                         \
    FAIL() << "exception '" << MESSAGE << "' not thrown at all!";     \
}                                                                     \
catch( const EXCEPTION_TYPE& e )                                      \
{                                                                     \
    EXPECT_EQ( MESSAGE, e.what() )                                    \
        << " exception message is incorrect. Expected the following " \
           "message:\n\n"                                             \
        << MESSAGE << "\n";                                           \
}                                                                     \
catch( ... )                                                          \
{                                                                     \
    FAIL() << "exception '" << MESSAGE                                \
           << "' not thrown with expected type '" << #EXCEPTION_TYPE  \
           << "'!";                                                   \
}

Mike's TEST(foo_test,out_of_range) example would then be

TEST(foo_test,out_of_range)
{
    foo f;
    ASSERT_EXCEPTION( { f.bar(111); }, std::out_of_range, "Out of range" );
}

which I think ends up being much more readable.

Vicinage

you should put the braces right into the macro, then the macro call is much more like the original calls - at least when you investigate one command. – Tonitonia

Defining a macro is the way to go (instead of coding this logic in each test ... DRY). But, this requires the caller to test the exact message via EXPECT_EQ. I rarely want to do that. What I want is to write the assertion based on the exception object. OP asks for 'best' (which is hard to achieve), but I think there are better answers. – Alberic

Answer 7

My version; it produces the same output as EXPECT_THROW and just adds the string test:

#define EXPECT_THROW_MSG(statement, expected_exception, expected_what)                    \
  try                                                                                     \
  {                                                                                       \
    statement;                                                                            \
    FAIL() << "Expected: " #statement " throws an exception of type " #expected_exception \
              ".\n"                                                                       \
              "  Actual: it throws nothing.";                                             \
  }                                                                                       \
  catch (const expected_exception& e)                                                     \
  {                                                                                       \
    EXPECT_EQ(expected_what, std::string{e.what()});                                      \
  }                                                                                       \
  catch (...)                                                                             \
  {                                                                                       \
    FAIL() << "Expected: " #statement " throws an exception of type " #expected_exception \
              ".\n"                                                                       \
              "  Actual: it throws a different type.";                                    \
  }

Cheapjack

Answer 8

I found the original answer given by Brandlingo (the macro solution) to create a better experience than the EXPECT_THAT alternative, which is still limiting if you need to examine the content of the exception in a non-trivial way.

There were some caveats to Brandlingo's answer though that I've resolved here:

1. Removed the unscoped exception_ptr that makes it annoying to use multiple times, by nesting try blocks instead and throwing to the outer block only if the expected exception occurs.
2. Assume any unexpected exception derives from std::exception, and print the value of "what()". Useful for providing hints as to what happened. If the exception doesn't derive from std::exception, the test will still fail, you just won't get a line number. You could handle it by adding another catch, but that really shouldn't be happening.
3. Pass in the name of the variable to use for the exception (for readability).

#define ASSERT_THROW_AND_USE(statement, expected_exception, exception_varname)                   \
    try {                                                                                         \
        try {                                                                                     \
            (statement);                                                                          \
            FAIL() << "Expected: " #statement " throws an exception of type " #expected_exception \
                      ".\n  Actual: it throws nothing.";                                          \
        } catch (const expected_exception&) {                                                     \
            throw;                                                                                \
        } catch (const std::exception& e) {                                                       \
            FAIL() << "Expected: " #statement " throws an exception of type " #expected_exception \
                      ".\n  Actual: it throws a different type, with message: "                   \
                   << std::quoted(e.what());                                                      \
        }                                                                                         \
    } catch (const expected_exception& exception_varname)

Then use it like this:

ASSERT_THROW_AND_USE(foo(), MyException, e)
{
    ASSERT_STREQ("fail message", e.MyMethod());
}

If the wrong exception is thrown, you'll get something like this:

Failed
Expected: foo() throws an exception of type MyException.
  Actual: it throws a different type, with message: "some other thing"

Deter

Very nice solution to avoid the exception_ptr indeed. – Beluga

Answer 9

I like most of the answers. However, since it seems that GoogleTest provides EXPECT_PRED_FORMAT that helps facilitating this, I'd like to add this option to the list of answers:

MyExceptionCreatingClass testObject; // implements TriggerMyException()

EXPECT_PRED_FORMAT2(ExceptionChecker, testObject, "My_Expected_Exception_Text");

where ExceptionChecker is defined as:

testing::AssertionResult ExceptionChecker(const char* aExpr1,
                                          const char* aExpr2,
                                          MyExceptionCreatingClass& aExceptionCreatingObject,
                                          const char* aExceptionText)
{
  try
  {
    aExceptionCreatingObject.TriggerMyException();
    // we should not get here since we expect an exception
    return testing::AssertionFailure() << "Exception '" << aExceptionText << "' is not thrown.";
  }
  catch (const MyExpectedExceptionType& e)
  {
    // expected this, but verify the exception contains the correct text
    if (strstr(e.what(), aExceptionText) == static_cast<const char*>(NULL))
    {
      return testing::AssertionFailure()
          << "Exception message is incorrect. Expected it to contain '"
          << aExceptionText << "', whereas the text is '" << e.what() << "'.\n";
    }
  }
  catch ( ... )
  {
    // we got an exception alright, but the wrong one...
    return testing::AssertionFailure() << "Exception '" << aExceptionText
    << "' not thrown with expected type 'MyExpectedExceptionType'.";
  }
  return testing::AssertionSuccess();
}

Chemesh

Answer 10

I use Matthäus Brandl's macro with the following minor modification:

Put the line

std::exception_ptr _exceptionPtr;

outside (f.e. before) the macro definition as

static std::exception_ptr _exceptionPtr;

to avoid multiple definition of the symbol _exceptionPtr.

Luciferase

You could also pass it to the macro as an argument, then you can choose the name freely. – Beluga

Perhaps a better solution would have been to just add an extra set of brackets to introduce a new scope as part of the macro. – Wyandotte

@Wyandotte I actually thought about that, but then you cannot add the testing scope right after the EXPECT_THROW_KEEP_AS_E. – Beluga

See my answer for a workaround that just works: https://mcmap.net/q/258156/-test-a-specific-exception-type-is-thrown-and-the-exception-has-the-right-properties – Deter

Answer 11

Expanding on previous answers, a macro that verifies that an exception of a given type was thrown and the message of which starts with the provided string.

The test fails if either no exception is thrown, the exception type is wrong or if the message doesn't start with the provided string.

#define ASSERT_THROWS_STARTS_WITH(expr, exc, msg) \
    try\
    {\
            (expr);\
            FAIL() << "Exception not thrown";\
    }\
    catch (const exc& ex)\
    {\
            EXPECT_THAT(ex.what(), StartsWith(std::string(msg)));\
    }\
    catch(...)\
    {\
            FAIL() << "Unexpected exception";\
    } 

Usage example:

ASSERT_THROWS_STARTS_WITH(foo(-2), std::invalid_argument, "Bad argument: -2");

Spongy

Answer 12

I would suggest to use

EXPECT_THROW(function_that_will_throw(), exception);

If your function is returning something void it:

EXPECT_THROW((void)function_that_will_throw(), exception);

Bolling

Answer 13

The above answers were helpful, but I wanted to share a solution I am using that keeps each test short while still testing both the exception type and the 'what' value. It works for any function that throws an exception that is derived from std::exception, but could be modified (or templated) to catch other types if needed.

I originally tried using a perfect forwarding type wrapper function, but ended up just using a lambda as a wrapper. Using a lambda is really flexible for re-use, allows all the expected implicit type conversions that happen when calling functions directly, avoids passing pointer-to-member functions, etc..

The important point is to have the wrapper template function throw a custom exception for a 'what-mismatch' that has a type that would not be thrown from the function being tested. This causes EXPECT_THROW to print a nice error about the mismatch. I derived from std::runtime_error since that class has a constructor that accepts a std::string.

class WhatMismatch : public std::runtime_error {
public:
  WhatMismatch(const std::string& expectedWhat, const std::exception& e)
    : std::runtime_error(std::string("expected: '") + expectedWhat +
                         "', actual: '" + e.what() + '\'') {}
};

template<typename F> auto call(const F& f, const std::string& expectedWhat) {
  try {
    return f();
  } catch (const std::exception& e) {
    if (expectedWhat != e.what()) throw WhatMismatch(expectedWhat, e);
    throw;
  }
}

A test for a function foo that should throw a std::domain_error would look like:

EXPECT_THROW(call([] { foo(); }, "some error message"), std::domain_error);

If the function takes parameters, just capture them in the lambda like this:

EXPECT_THROW(call([p1] { foo(p1); }, "some error message"), std::domain_error);

Raspings

Answer 14

You can try Boost lightweight test:

#include <boost/detail/lightweight_test.hpp>
#include <stdexcept>

void function_that_would_throw(int x)
{
  if (x > 0) {
    throw std::runtime_error("throw!");
  }
}

int main() {
 BOOST_TEST_THROWS(function_that_would_throw(10), std::runtime_error);
 boost::report_errors();
}

Selma

What this has to do with the question? – Exacting