Strict @MockBean in a Spring Boot Test

Question

I am developing a Spring Boot application. For my regular service class unit tests, I am able to extend my test class with MockitoExtension, and the mocks are strict, which is what I want.

interface MyDependency {
  Integer execute(String param);
}

class MyService {
  @Autowired MyDependency myDependency;

  Integer execute(String param) {
    return myDependency.execute(param);
  }
} 

@ExtendWith(MockitoExtension.class)
class MyServiceTest {
  @Mock
  MyDependency myDependency;

  @InjectMocks
  MyService myService;

  @Test
  void execute() {
    given(myDependency.execute("arg0")).willReturn(4);
    
    myService.execute("arg1"); //will throw exception
  }
}

In this case, the an exception gets thrown with the following message (redacted):

org.mockito.exceptions.misusing.PotentialStubbingProblem: 
Strict stubbing argument mismatch. Please check:
 - this invocation of 'execute' method:
    myDependency.execute(arg1);
 - has following stubbing(s) with different arguments:
    1. myDependency.execute(arg0);

In addition, if the stubbing was never used there would be the following (redacted):

org.mockito.exceptions.misusing.UnnecessaryStubbingException: 
Unnecessary stubbings detected.
Clean & maintainable test code requires zero unnecessary code.
Following stubbings are unnecessary (click to navigate to relevant line of code):
  1. -> at MyServiceTest.execute()

However, when I use @MockBean in an integration test, then none of the strict behavior is present. Instead, the stubbed method returns null because the stubbing "fails" silently. This is behavior that I do not want. It is much better to fail immediately when unexpected arguments are used.

@SpringBootTest
class MyServiceTest {
  @MockBean
  MyDependency myDependency;

  @Autowired
  MyService myService;

  @Test
  void execute() {
    given(myDependency.execute("arg0")).willReturn(4);
    
    myService.execute("arg1"); //will return null
  }
}

Is there any workaround for this?

Zachar

Use constructor autowiring for your service, and then you can pass in mockito mocks directly. Instead of using mockbean. – Dorcus

Offer a way for MockitoTestExecutionListener to enable strict stubbing #19383 – Libertine

That's just another reason not to use @MockBean. The important reason is that MockBean introduces global mocking, and lets you keep any closed design you may have which does not allow injecting dependencies properly. Without MockBean you would need to refactor your design, and MockBean lets bypass the necessary refactoring. – Turf

Everyone: I WANT the failure, and I need to use MockBean in a SpringBootTest, otherwise the application context does not load. I am looking for a way to make the MockBean mocks strict. – Zachar

Answer 1

As mentioned in this comment, this GitHub issue in the spring-boot project addresses this same problem and has remained open since 2019, so it's unlikely that an option for "strict stubs" will be available in @SpringBootTest classes anytime soon.

One way that Mockito recommends to enable "strict stubs" is to start a MockitoSession with Strictness.STRICT_STUBS before each test, and close the MockitoSession after each test. Mockito mocks for @MockBean properties in @SpringBootTest classes are generated by Spring Boot's MockitoPostProcessor, so a workaround would need to create the MockitoSession before the MockitoPostProcessor runs. A custom TestExecutionListener can be implemented to handle this, but only its beforeTestClass method would run before the MockitoPostProcessor. The following is such an implementation:

public class MyMockitoTestExecutionListener implements TestExecutionListener, Ordered {
    // Only one MockitoSession can be active per thread, so ensure that multiple instances of this listener on the
    // same thread use the same instance
    private static ThreadLocal<MockitoSession> mockitoSession = ThreadLocal.withInitial(() -> null);

    // Count the "depth" of processing test classes. A parent class is not done processing until all @Nested inner
    // classes are done processing, so all @Nested inner classes must share the same MockitoSession as the parent class
    private static ThreadLocal<Integer> depth = ThreadLocal.withInitial( () -> 0 );

    @Override
    public void beforeTestClass(TestContext testContext) {
        depth.set(depth.get() + 1);
        if (depth.get() > 1)
            return; // @Nested classes share the MockitoSession of the parent class

        mockitoSession.set(
                Mockito.mockitoSession()
                        .strictness(Strictness.STRICT_STUBS)
                        .startMocking()
        );
    }

    @Override
    public void afterTestClass(TestContext testContext) {
        depth.set(depth.get() - 1);
        if (depth.get() > 0)
            return; // @Nested classes should let the parent class end the MockitoSession

        MockitoSession session = mockitoSession.get();
        if (session != null)
            session.finishMocking();
        mockitoSession.remove();
    }

    @Override
    public int getOrder() {
        return Ordered.LOWEST_PRECEDENCE;
    }
}

Then, MyMockitoTestExecutionListener can be added as a listener in test classes:

@SpringBootTest
@TestExecutionListeners(
    listeners = {MyMockitoTestExecutionListener.class},
    mergeMode = MergeMode.MERGE_WITH_DEFAULTS
)
public class MySpringBootTests {
    @MockBean
    Foo mockFoo;

    // Tests using mockFoo...
}

Alternatively, it can be enabled globally by putting the following in src/test/resources/META-INF/spring.factories:

org.springframework.test.context.TestExecutionListener=\
  com.my.MyMockitoTestExecutionListener

Edina

Answer 2

Yes there are some workarounds but it is quite involved. It may be better to just wait for Mockito 4 where the default will be strict mocks.

The first option:

1. Replace @MockBean with @Autowired with a test configuration with @Primary ( this should give the same effect as @MockBean, inserting it into the application as well as into the test )

2. Create a default answer that throws an exception for any unstubbed function

Then override that answer with some stubbing - but you have to use doReturn instead of when thenReturn

// this is the component to mock
@Component
class ExtService {
    int f1(String a) {
        return 777;
    }
}
// this is the test class
@SpringBootTest
@RunWith(SpringRunner.class)
public class ApplicationTests {

    static class RuntimeExceptionAnswer implements Answer<Object> {
        @Override
        public Object answer(InvocationOnMock invocation) throws Throwable {
            throw new RuntimeException(
                    invocation.getMethod().getName() + " was not stubbed with the received arguments");
        }
    }

    @TestConfiguration
    public static class TestConfig {

        @Bean
        @Primary
        public ExtService mockExtService() {
            ExtService std = Mockito.mock(ExtService.class, new RuntimeExceptionAnswer());
            return std;
        }

    }

    // @MockBean ExtService extService;
    @Autowired
    ExtService extService; // replace mockBean

    @Test
    public void contextLoads() {

        Mockito.doReturn(1).when(extService).f1("abc"); // stubbing has to be in this format
        System.out.println(extService.f1("abc")); // returns 1
        System.out.println(extService.f1("abcd")); // throws exception
    }

}

Another possible but far from ideal option: instead of using a default answer is to stub all your function calls first with an any() matcher, then later with the values you actually expect.

This will work because the stubbing order matters, and the last match wins.

But again: you will have to use the doXXX() family of stubbing calls, and worse you will have to stub every possible function to come close to a strict mock.

// this is the service we want to test
@Component
class ExtService {
    int f1(String a) {
        return 777;
    }
}
// this is the test class
@SpringBootTest
@RunWith(SpringRunner.class)
public class ApplicationTests {


    @MockBean ExtService extService;

    @Test
    public void contextLoads() {

        Mockito.doThrow(new RuntimeException("unstubbed call")).when(extService).f1(Mockito.any()); // stubbing has to be in this format
        Mockito.doReturn(1).when(extService).f1("abc"); // stubbing has to be in this format
        System.out.println(extService.f1("abc")); // returns 1
        System.out.println(extService.f1("abcd")); // throws exception
    }

}

Yet another option is to wait until after the test finishes using the mock, and then use

verifyNoMoreInteractions();

Turf

"just wait for Mockito 4 where the default will be strict mocks" Mockito 5 is pulled as a default dependency of a latest spring boot, yet strict behaviour is not default. – Cicatrize

Answer 3

By asking Mockito to print the invocation details for a mock in the @AfterEach-method, you can find unused stubbings. I just tried it on one of my test classes and it did work, it did indeed find a case of unused stubbings. Not a perfect solution since one has to remember to do it for each mock and relying on string searching is brittle, but you can make it work.

@AfterEach
void afterEach() {
    String invocations = Mockito.mockingDetails(authorizationService).printInvocations();
    assertThat(invocations).doesNotContainIgnoringCase("unused");
}

Bloc

Interesting. I guess that means I could at least write a JUnit Extension to do this manual work. – Zachar