How do I assert my exception message with JUnit Test annotation?

Question

I have written a few JUnit tests with @Test annotation. If my test method throws a checked exception and if I want to assert the message along with the exception, is there a way to do so with JUnit @Test annotation? AFAIK, JUnit 4.7 doesn't provide this feature but does any future versions provide it? I know in .NET you can assert the message and the exception class. Looking for similar feature in the Java world.

This is what I want:

@Test (expected = RuntimeException.class, message = "Employee ID is null")
public void shouldThrowRuntimeExceptionWhenEmployeeIDisNull() {}

Wheatear

Now that I think about it a little more... Are you sure it is a good idea to assert the message? Your question made me dig into the junit source code a bit and it seems they could have easily added this feature. The fact that they did not, makes me think it might not be considered a good practice. Why is it important in your project to assert the message? – Geo

good question.Say that a method containing 15 lines of code throws the same exception from 2 different places. My test cases need to assert not just the exception class but also the message in it. In an ideal world, any abnormal behavior should have its own exception.If that had been the case, my question would never arise but production applications donot have their unique custom exception for each abnormal behavior. – Wheatear

As a side note - there is @expectedExceptionMessage annotation in PHPUnit. – Kliber

Answer 1

You could use the @Rule annotation with ExpectedException, like this:

@Rule
public ExpectedException expectedEx = ExpectedException.none();

@Test
public void shouldThrowRuntimeExceptionWhenEmployeeIDisNull() throws Exception {
    expectedEx.expect(RuntimeException.class);
    expectedEx.expectMessage("Employee ID is null");

    // do something that should throw the exception...
    System.out.println("=======Starting Exception process=======");
    throw new NullPointerException("Employee ID is null");
}

Note that the example in the ExpectedException docs is (currently) wrong - there's no public constructor, so you have to use ExpectedException.none().

Adactylous

Note: For me when the expectMessage was specified as an empty string, the comparison for the message was not performed – Effloresce

Useful to me. Thanks. The testmethod should have throws RuntimeException after adding code that throws an exception. Do not catch it... – Heterosis

I personally wouldn't want to use this since creating fields for the purpose of a small subset of methods is bad practice. Not a criticism of the response, but of JUnit's design. The OP's hypothetical solution would be so much better if it existed. – Alleris

@redDevil: The expectedMessage checks if the error message "contains" the string specified in this function (like a substring of the error message) – Outride

Excellent solution, for readability. I call mine e.g. failure.expectMessage(...) – Sasnett

expectMessage with string parameter does a String.contains checking, for exact match of exception message use hamcrest matcher failure.expectMessage(CoreMatchers.equalTo(...)) – Preventive

ExpectedException.none() is deprecated since Junit 4.13 – Peeress

@Peeress should have provided the replacement ;) – Iaea

ExpectedException.none() Deprecated. Since 4.13 Assert.assertThrows can be used to verify that your code throws a specific exception. – Iaea

Answer 2

In JUnit 4.13 you can do:

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertThrows;

...

@Test
void exceptionTesting() {
  IllegalArgumentException exception = assertThrows(
    IllegalArgumentException.class, 
    () -> { throw new IllegalArgumentException("a message"); }
  );

  assertEquals("a message", exception.getMessage());
}

This also works in JUnit 5 but with different imports:

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

...

Chessa

Like this solution. Should move to JUnit 5. – Heterochromatic

Gaaaaaaaaa. 4.13 still in beta as of today (Fall, 2019)? mvnrepository.com/artifact/junit/junit – Mcintyre

v4.13 is not in beta state anymore (release in January 2020) – Charlettecharley

Since assertThrows is available in JUnit 4.13, this should be the accepted answer – Brittaniebrittany

I was already using 4.13 assertThrows, but did not yet know that it returns the exception for subsequent inspection. +1, exactly what I needed :-D – Tramline

Answer 3

I like the @Rule answer. However, if for some reason you don't want to use rules. There is a third option.

@Test (expected = RuntimeException.class)
public void myTestMethod()
{
   try
   {
      //Run exception throwing operation here
   }
   catch(RuntimeException re)
   {
      String message = "Employee ID is null";
      assertEquals(message, re.getMessage());
      throw re;
    }
    fail("Employee Id Null exception did not throw!");
  }

Manchester

(expected = RuntimeException.class) and throw re; are not neccessary;. There should be just method which throws exception within try and in catch assertions. – Jara

@janusz j: I personally prefer to leave the (expected... and throw re; lines, but remove the fail(... line. Can you or anyone else please tell me why my preference is/isn't a good practice? – Hoekstra

within try catch, you catch exception wherever you want. When you have for ex: same Exception types throwing in different places you won't get to know where it has been thrown. – Jara

@janusz j: Thank you and I see. In other words, if my test method has 0 line of code outside the try catch block, it would be fine? – Hoekstra

Answer 4

Do you have to use @Test(expected=SomeException.class)? When we have to assert the actual message of the exception, this is what we do.

@Test
public void myTestMethod()
{
  try
  {
    final Integer employeeId = null;
    new Employee(employeeId);
    fail("Should have thrown SomeException but did not!");
  }
  catch( final SomeException e )
  {
    final String msg = "Employee ID is null";
    assertEquals(msg, e.getMessage());
  }
}

Geo

I m aware of writing a catch block and using assert within that but for better code readability i want to do with annotations. – Wheatear

Also you won't get such nice message as when doing it the "right" way. – Singlet

The problem with the try/catch version, now that JUnit provides @Test(expected=...) and ExpectedException, is that I have seen on numerous occasions someone forgetting to put the call to fail() at the end of the try block. If not caught by code review, your test may be false-positive and always pass. – Representational

This is why I don't like all this declarative stuff. It makes it difficult to access what you want. – Alleris

Answer 5

Actually, the best usage is with try/catch. Why? Because you can control the place where you expect the exception.

Consider this example:

@Test (expected = RuntimeException.class)
public void someTest() {
   // test preparation
   // actual test
}

What if one day the code is modified and test preparation will throw a RuntimeException? In that case actual test is not even tested and even if it doesn't throw any exception the test will pass.

That is why it is much better to use try/catch than to rely on the annotation.

Alow

Sadly, this is my answer too. – Alleris

The concerns regarding code changes are alleviated by having small, permutation-specific test cases. Sometimes it is inevitable and we have to rely in the catch/try method, but if that happens frequently, then chances are that we need to revise the way we write our test case functions. – Hindsight

That's a problem with your test and/or code. You do NOT expect a general RuntimeException, you expect a specific exception, or at the very least a specific message. – Disvalue

I used RuntimeException as an example, replace this exception with any other exception. – Alow

Answer 6

I never liked the way of asserting exceptions with Junit. If I use the "expected" in the annotation, seems from my point of view we're violating the "given, when, then" pattern because the "then" is placed at the top of the test definition.

Also, if we use "@Rule", we have to deal with so much boilerplate code. So, if you can install new libraries for your tests, I'd suggest to have a look to the AssertJ (that library now comes with SpringBoot)

Then a test which is not violating the "given/when/then" principles, and it is done using AssertJ to verify:

1 - The exception is what we're expecting. 2 - It has also an expected message

Will look like this:

 @Test
void should_throwIllegalUse_when_idNotGiven() {

    //when
    final Throwable raisedException = catchThrowable(() -> getUserDAO.byId(null));

    //then
    assertThat(raisedException).isInstanceOf(IllegalArgumentException.class)
            .hasMessageContaining("Id to fetch is mandatory");
}

Vertumnus

Answer 7

Raystorm had a good answer. I'm not a big fan of Rules either. I do something similar, except that I create the following utility class to help readability and usability, which is one of the big plus'es of annotations in the first place.

Add this utility class:

import org.junit.Assert;

public abstract class ExpectedRuntimeExceptionAsserter {

    private String expectedExceptionMessage;

    public ExpectedRuntimeExceptionAsserter(String expectedExceptionMessage) {
        this.expectedExceptionMessage = expectedExceptionMessage;
    }

    public final void run(){
        try{
            expectException();
            Assert.fail(String.format("Expected a RuntimeException '%s'", expectedExceptionMessage));
        } catch (RuntimeException e){
            Assert.assertEquals("RuntimeException caught, but unexpected message", expectedExceptionMessage, e.getMessage());
        }
    }

    protected abstract void expectException();

}

Then for my unit test, all I need is this code:

@Test
public void verifyAnonymousUserCantAccessPrivilegedResourceTest(){
    new ExpectedRuntimeExceptionAsserter("anonymous user can't access privileged resource"){
        @Override
        protected void expectException() {
            throw new RuntimeException("anonymous user can't access privileged resource");
        }
    }.run(); //passes test; expected exception is caught, and this @Test returns normally as "Passed"
}

Mcneil

Answer 8

Let's say, we have a method doStuff in the class FooService. This method can throw a NullPointerException when a null flag is passed in the argument.

public class FooService {

  public void doStuff(Boolean flag) {
    try{
      if(flag){
        // do stuff
      }
    }catch (Exception e){
      throw new RuntimeException("Unexpected error occurred", e);
    }
  }
}

Using Junit 5

Junit 5 assertThrows returns exception, which can be used to assert exception message as per below example:-

import static org.junit.jupiter.api.Assertions.*;

@Test
public void doStuff_testThrownException(){   
    // null is passed, expected NullPointerException
    Throwable exception = assertThrows(RuntimeException.class, () -> fooService.doStuff(null)); 

    assertEquals("Unexpected error occurred", exception.getMessage());
}

Using AssertJ

AssertJ has various assertions to test exception, one of them is assertThatRuntimeException which can be chained with withMessage to test exception message

import static org.assertj.core.api.Assertions.*;

@Test
public void doStuff_testThrownException(){
  // null is passed, expected NullPointerException
  assertThatRuntimeException().isThrownBy(() -> fooService.doStuff(null))
          .withMessage("Unexpected error occurred");
}

Source: CodingNConcepts

Hagood

Answer 9

I would prefer AssertJ for this.

        assertThatExceptionOfType(ExpectedException.class)
        .isThrownBy(() -> {
            // method call
        }).withMessage("My message");

Fid

Answer 10

If using @Rule, the exception set is applied to all the test methods in the Test class.

Sabec

Using Jesse Merriman response, the exception is only checked in test methods that call to expectedEx.expect() and expectedEx.expectMessage(). The other methods will use the definition expectedEx = ExpectedException.none(), that is, no exception expected. – Lareine

Answer 11

I like user64141's answer but found that it could be more generalized. Here's my take:

public abstract class ExpectedThrowableAsserter implements Runnable {

    private final Class<? extends Throwable> throwableClass;
    private final String expectedExceptionMessage;

    protected ExpectedThrowableAsserter(Class<? extends Throwable> throwableClass, String expectedExceptionMessage) {
        this.throwableClass = throwableClass;
        this.expectedExceptionMessage = expectedExceptionMessage;
    }

    public final void run() {
        try {
            expectException();
        } catch (Throwable e) {
            assertTrue(String.format("Caught unexpected %s", e.getClass().getSimpleName()), throwableClass.isInstance(e));
            assertEquals(String.format("%s caught, but unexpected message", throwableClass.getSimpleName()), expectedExceptionMessage, e.getMessage());
            return;
        }
        fail(String.format("Expected %s, but no exception was thrown.", throwableClass.getSimpleName()));
    }

    protected abstract void expectException();

}

Note that leaving the "fail" statement within the try block causes the related assertion exception to be caught; using return within the catch statement prevents this.

Afroasian

Answer 12

Import the catch-exception library, and use that. It's much cleaner than the ExpectedException rule or a try-catch.

Example form their docs:

import static com.googlecode.catchexception.CatchException.*;
import static com.googlecode.catchexception.apis.CatchExceptionHamcrestMatchers.*;

// given: an empty list
List myList = new ArrayList();

// when: we try to get the first element of the list
catchException(myList).get(1);

// then: we expect an IndexOutOfBoundsException with message "Index: 1, Size: 0"
assertThat(caughtException(),
  allOf(
    instanceOf(IndexOutOfBoundsException.class),
    hasMessage("Index: 1, Size: 0"),
    hasNoCause()
  )
);

Tades

Answer 13

@Test (expectedExceptions = ValidationException.class, expectedExceptionsMessageRegExp = "This is not allowed")
public void testInvalidValidation() throws Exception{
     //test code
}

Leavetaking

Can someone help me understand why this answer is a -1 – Leavetaking

The question is asking for Junit but ur answer is giving TestNG – Protostele

@Leavetaking Your answer indeed helped me. Thanks. – Curculio