mockito better expected exception test using spy

Question

How can I make the 3rd test to check for the existence of cause1 in the message of the exception? I also listed in the first two tests that have drawbacks. First is not checking for the message second needs a lot of boilerplate code.

public class CheckExceptionsWithMockitoTest {

    @Test(expected = RuntimeException.class)
    public void testExpectedException1() {
        A a = new A();
        a.doSomethingThatThrows();
    }

    @Test
    public void testExpectedException2() {
        A a = new A();
        try {
            a.doSomethingThatThrows();
            fail("no exception thrown");
        } catch (RuntimeException e) {
            assertThat(e.getMessage(), org.hamcrest.Matchers.containsString("cause1"));
        }
    }

    @Test
    public void testExpectedException3() {
        A a = new A();
        A spyA = org.mockito.Mockito.spy(a);
        // valid but doesnt work
        // doThrow(new IllegalArgumentException()).when(spyA).doSomethingThatThrows();
        // invalid but in the spirit of what i want 
        //chekThrow(RuntimeException.class,containsString("cause1")).when(spyA).doSomethingThatThrows();
    }

}

I couldn't find in Mockito something that works but there is something that looks like could be possible (at the level of syntax) and capabilities.

---

Using catchexception I created the test like this

import static com.googlecode.catchexception.CatchException.*;
import static com.googlecode.catchexception.apis.CatchExceptionHamcrestMatchers.*;
import static org.hamcrest.Matchers.*;
import static org.junit.Assert.*;

import org.junit.*;
public class CheckExceptionsWithMockitoTest{  
    //...
    @Test
    public void testExpectedException3() {
        A a = new A();
        verifyException(a,IllegalArgumentException.class)
            .doSomethingThatThrows();
        //if more details to be analized are needed
        assertThat(
            (IllegalStateException) caughtException(),
            allOf(
                is(IllegalStateException.class),
                hasMessageThat(
                        containsString("is not allowed to add counterparties")), 
                hasNoCause()));
        //more asserts could come
        assertNotNull(a);
    }
}

Lumpy

Answer 1

Use catch-exception library, or I guess that the solution you are looking for is your second implementation.

@expected doesn't provide any way to assert on the thrown exception except for its class, so you can't avoit try/catching (not that much boiler plate code !)

Mockito doesn't provide something likes a verifyThrows method.

So you can trade try/catching for an additional library : using catch-exception, you'll be able to catch exception in a single line and have it ready for further assertion(s).

Sample source code

A a = new A();

when(a).doSomethingThatThrows();

then(caughtException())
        .isInstanceOf(IllegalStateException.class)
        .hasMessageContaining("is not allowed to add counterparties")
        .hasNoCause();

Dependencies

'com.googlecode.catch-exception:catch-exception:1.2.0'

Bilharziasis

Answer 2

If A is your system under test, it doesn't make any sense to mock it, and it rarely makes sense to spy on it. Your implementation in testExpectedException2 is the right one; the boilerplate code is necessary because without a try block Java will not let any code run after the method is intercepted (as I described in this previous SO answer).

Though Mockito won't be any help, JUnit will. The @Test(expected=foo) parameter actually has a more-flexible alternative, the built-in ExpectedException JUnit rule:

public class CheckExceptionsWithMockitoTest {

  @Rule public ExpectedException thrown = ExpectedException.none();

  @Test
  public void testExpectedException1() {
    A a = new A();
    thrown.expect(RuntimeException.class);
    thrown.expectMessage(containsString("cause1"));
    a.doSomethingThatThrows();
  }
}

Mockito would come in handy in a separate test checking whether your method wraps an arbitrary exception while preserving its message, which would look roughly like this:

@Test
public void doSomethingShouldWrapExceptionWithPassedMessage() {
  Dependency dependency = Mockito.mock(Dependency.class);
  when(dependency.call()).thenThrow(new IllegalArgumentException("quux"));
  A a = new A(dependency);
  thrown.expect(RuntimeException.class);
  thrown.expectMessage(containsString("quux"));
  a.doSomethingThatThrows();
}

Be careful to avoid the temptation to make this a common pattern in your tests. If you are catching an exception thrown from your system under test, you're effectively ceding control back to the SUT's consumer. There should be little left to test in the method afterwards, except the properties of the exception and MAYBE the state of your system, both of which should be rare enough that try/catch boilerplate is forgivable.

Hoppe

Where do "thrown.expect" and "thrown.expectMessage" come from ?? – Bilharziasis

@Bilharziasis - They are methods of the ExpectedException rule declared at the top of the test case. – Prejudice

Thank you for your answer. Is understandable that junit needs that boilerplate code. I was thinking that it is possible to have a proxy to the real class, that will catch any exception thrown verify if this is expected and return normally from the call. In this way I could continue the test with additional investigation of the state (including the type of the exception, the content). If it makes sense and such thing doesn't exist in mockito I could try to contribute. – Lumpy

@Lumpy My personal opinion is that your case is rare enough—and the workaround small and straightforward enough—that it doesn't make sense to add that functionality in Mockito, especially if it encourages a known anti-pattern (mocking the system under test). If you do want to contribute it, please make sure you at least ask the Mockito development mailing list so you don't waste time if the project owners don't see the value in adding it. – Hoppe

@Lumpy You can certainly do exactly what you are describing, using the mechanism that's already in JUnit. Just add more stuff to your test. I wouldn't recommend it though - I'm a member of the "one assertion, one test case" brigade. Moreover, I can't imagine the Mockito team wanting to add something to handle a case that JUnit already caters adequately for. – Cattleman

It seems that catch-exception is using the mechanisms similar to what i was describing. I don't know if is feasible and/or desired to integrate that in mockito. @DavidWallace Adding an Rule is a little strange to me since is based on a convention that is not simple and clear as I would like. Besides, regarding the "one assertion, one test case" topic: we have the assert keyword and fully programmatic access to check conditions in case no exceptions are involved. We need the same in case exceptions are thrown. – Lumpy

Answer 3

If you have the opportunity to use scala, scalaTest's fun suite has concise way of testing exceptions using intercept (http://www.scalatest.org/getting_started_with_fun_suite).

It's as simple as

  test(a list get method catches exceptions){
    intercept[IndexOutBoundsException]{
      spyListObject.get(-1)
    }
  }

You could potentially write your tests to your java project in scala if you are looking for easy to write / clear test. But this may present other challenges.

Shadow

Answer 4

Updated answer for 06/19/2015 (if you're using java 8)

Using assertj-core-3.0.0 + Java 8 Lambdas

@Test
public void shouldThrowIllegalArgumentExceptionWhenPassingBadArg() {
      assertThatThrownBy(() -> myService.sumTingWong("badArg"))
                                  .isInstanceOf(IllegalArgumentException.class);
}

Reference: http://blog.codeleak.pl/2015/04/junit-testing-exceptions-with-java-8.html

Edmead

Answer 5

Using catchexception I created the test like this

import static com.googlecode.catchexception.CatchException.*;
import static com.googlecode.catchexception.apis.CatchExceptionHamcrestMatchers.*;
import static org.hamcrest.Matchers.*;
import static org.junit.Assert.*;

import org.junit.*;
public class CheckExceptionsWithMockitoTest{  
    //...
    @Test
    public void testExpectedException3() {
        A a = new A();
        verifyException(a,IllegalArgumentException.class)
            .doSomethingThatThrows();
        //if more details to be analized are needed
        assertThat(
            (IllegalStateException) caughtException(),
            allOf(
                is(IllegalStateException.class),
                hasMessageThat(
                        containsString("is not allowed to add counterparties")), 
                hasNoCause()));
        //more asserts could come
        assertNotNull(a);
    }
}

Lumpy

Answer 6

If you have a look in Mockito.class on spy method it creates mock with spiedInstance:

 public static <T> T spy(T object) {
    return MOCKITO_CORE.mock((Class<T>) object.getClass(), withSettings()
            .spiedInstance(object)
            .defaultAnswer(CALLS_REAL_METHODS));
}

In MockSettings it is possible to register Invocation listeners: https://static.javadoc.io/org.mockito/mockito-core/3.0.0/org/mockito/listeners/InvocationListener.html

I created simple listener which stores all reported invocations:

import java.util.ArrayList;
import java.util.Collections;
import java.util.List;

import org.mockito.listeners.InvocationListener;
import org.mockito.listeners.MethodInvocationReport;

public class StoringMethodInvocationListener implements InvocationListener {

private List<MethodInvocationReport> methodInvocationReports = new ArrayList<>();

@Override
public void reportInvocation(MethodInvocationReport methodInvocationReport) {
    this.methodInvocationReports.add(methodInvocationReport);

}

public List<MethodInvocationReport> getMethodInvocationReports() {
    return Collections.unmodifiableList(methodInvocationReports);
}

}

After the invocation you can go through reports and find the one needed and verify that stored throwable is the one expected.

Example:

    StoringMethodInvocationListener listener = new StoringMethodInvocationListener();
    Consumer mock2 = mock(Consumer.class, withSettings()
            .spiedInstance(consumerInstance)
            .defaultAnswer(CALLS_REAL_METHODS)
            .invocationListeners(listener));

    try {
        mock2.listen(new ConsumerRecord<String, String>(RECEIVER_TOPIC, 0, 0,  null, "{}"));
    } catch (Exception e){
        //nothing
    }
    Assert.notEmpty(listener.getMethodInvocationReports(), "MethodInvocationReports list must not be empty");
    Assert.isInstanceOf(BindException.class, listener.getMethodInvocationReports().get(1).getThrowable());

Cerussite