How to reuse Testcontainers between multiple SpringBootTests?

Question

I'm using TestContainers with Spring Boot to run unit tests for repositories like this:

@Testcontainers
@ExtendWith(SpringExtension.class)
@ActiveProfiles("itest")
@SpringBootTest(classes = RouteTestingCheapRouteDetector.class)
@ContextConfiguration(initializers = AlwaysFailingRouteRepositoryShould.Initializer.class)
@TestExecutionListeners(listeners = DependencyInjectionTestExecutionListener.class)
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Tag("docker")
@Tag("database")
class AlwaysFailingRouteRepositoryShould {

  @SuppressWarnings("rawtypes")
  @Container
  private static final PostgreSQLContainer database =
      new PostgreSQLContainer("postgres:9.6")
          .withDatabaseName("database")
          .withUsername("postgres")
          .withPassword("postgres");

But now I have 14 of these tests and every time a test is run a new instance of Postgres is spun up. Is it possible to reuse the same instance across all tests? The Singleton pattern doesn't help since every test starts a new application.

I've also tried testcontainers.reuse.enable=true in .testcontainers.properties and .withReuse(true), but that didn't help.

Kaon

Did you try the withReuse(true) option? – Luisaluise

@P3trur0: Yes. And testcontainers.reuse.enable=true - doesn't help. – Serin

Try to place container instance into separate @TestConfiguration as a @Bean and then import this configuration in all your relevant tests. – Phoenicia

@NikolaiShevchenko: Tried that, but it's needed in the @ContextConfiguration(initializers and I can't get it to work there. – Serin

Answer 1

You can't use the JUnit Jupiter annotation @Container if you want to have reusable containers. This annotation ensures to stop the container after each test.

What you need is the singleton container approach, and use e.g. @BeforeAll to start your containers. Even though you then have .start() in multiple tests, Testcontainers won't start a new container if you opted-in for reusability using both .withReuse(true) on your container definition AND the following .testcontainers.properties file in your home directory:

testcontainers.reuse.enable=true

A simple example might look like the following:

@SpringBootTest
public class SomeIT {

  public static GenericContainer postgreSQLContainer = new PostgreSQLContainer().
    withReuse(true);

  @BeforeAll
  public static void beforeAll() {
    postgreSQLContainer.start();
  }

  @Test
  public void test() {

  }

}

and another integration test:

@SpringBootTest
public class SecondIT {

  public static GenericContainer postgreSQLContainer = new PostgreSQLContainer().
    withReuse(true);

  @BeforeAll
  public static void beforeAll() {
    postgreSQLContainer.start();
  }

  @Test
  public void secondTest() {

  }

}

There is currently a PR that adds documentation about this

I've put together a blog post explaining how to reuse containers with Testcontainers in detail.

Tragedian

Many thanks. Then I also need to increase my max_connections with .withCommand("postgres -c max_connections=200"). How can I control the lifetime of reusable containers? – Serin

I can't answer this question. Maybe it's worth having a look at the PR which introduced the feature: github.com/testcontainers/testcontainers-java/pull/1781 – Tragedian

Where is testcontainers.reuse.enable=true documented? – Abcoulomb

Is there a way to configure testcontainers.reuse.enable=true elsewhere? I don't want reusable containers to be dependant on the developer's laptop configuration – Klausenburg

Yes, there are multiple ways to configure this property. Take a look at the Configuration locations section of the Testcontainers documentation. – Tragedian

You can't set testcontainers.reuse.enable via classpath. See this comment. – Captive

@Tragedian how can we ensure that it is reusing the same container as I applied these settings and I do not see any time difference in my integration test run time. it seems to same with these conditions enabled and disabled. thanks – Genitive

Take a look at docker ps while your tests are running. You should only see a single container being used. If you are on Linux/macOS watch -n 1 docker ps may be handy – Tragedian

@Klausenburg You can do this via TestcontainersConfiguration.getInstance().updateUserConfig("testcontainers.reuse.enable", "true"); before starting the container – Naked

There are 2 calls of new PostgreSQLContainer() But only one container will be used. Is it correct ? – Maniacal

Yes, only one container will be created for the tests – Tragedian

After the last test, Ryuk will still take care of shutting down the container, won't it ? – Damage

For some reasons my containers are not reused: #77144907 – Maniacal

Answer 2

Accepted answer is great but the problem is you still have to repeat the configurations(creating, starting and etc.) for each integration tests. It would be better to have simpler configuration with fewer lines of code. I think cleaner version would be using JUnit 5 extensions.

This is how I solved the problem. Below sample uses MariaDB container but the concept is applicable to all.

1. Create the container config holding class:

public class AppMariaDBContainer extends MariaDBContainer<AppMariaDBContainer> {

    private static final String IMAGE_VERSION = "mariadb:10.5";
    private static final String DATABASE_NAME = "my-db";
    private static final String USERNAME = "user";
    private static final String PASSWORD = "strong-password";

    public static AppMariaDBContainer container = new AppMariaDBContainer()
            .withDatabaseName(DATABASE_NAME)
            .withUsername(USERNAME)
            .withPassword(PASSWORD);

    public AppMariaDBContainer() {
        super(IMAGE_VERSION);
    }

}

2. Create an extension class that starts the container and sets the DataSource properties. And run migrations if needed:

public class DatabaseSetupExtension implements BeforeAllCallback {

    @Override
    public void beforeAll(ExtensionContext context) {
        AppMariaDBContainer.container.start();
        updateDataSourceProps(AppMariaDBContainer.container);
        //migration logic here (if needed)
    }

    private void updateDataSourceProps(AppMariaDBContainer container) {
        System.setProperty("spring.datasource.url", container.getJdbcUrl());
        System.setProperty("spring.datasource.username", container.getUsername());
        System.setProperty("spring.datasource.password", container.getPassword());
    }

}

3. Add @ExtendWith to your test class

@SpringBootTest
@ExtendWith(DatabaseSetupExtension.class)
class ApplicationIntegrationTests {

    @Test
    void someTest() {
    }

}

Another test

@SpringBootTest
@ExtendWith(DatabaseSetupExtension.class)
class AnotherIntegrationTests {

    @Test
    void anotherTest() {
    }

}

Phytosociology

Should be @ExtendWith(DatabaseSetupExtension.class) – Phosphoroscope

For some reasons my containers are not reused: #77144907 – Maniacal

I have similar setup, but I noticed ryuk is no longer launched along with the extension. I'm certain it was at some stage but is no longer true. Do you have the same issue? If so, do you know the workaround to keep the extension? – Snowshed

Answer 3

If you decide go forward with the singleton pattern, mind the warning in "Database containers launched via JDBC URL scheme". I took hours till I note that, even though I was using the singleton pattern, an additional container was always being created mapped on a different port.

In summary, do not use the test containers JDBC (host-less) URIs, such as jdbc:tc:postgresql:<image-tag>:///<databasename>, if you need use the singleton pattern.

Alphanumeric

Answer 4

Using either singleton containers or reusable containers are possible solutions but because they don't scope the life-cycle of the container to that of the application context both are less then ideal.

It is however possible to scope the container to the application contexts lifecycle by using a ContextCustomizerFactory and I've written about this in more detail in a blog post.

In a test use:

@Slf4j
@SpringBootTest
@EnabledPostgresTestContainer
class DemoApplicationTest {

    @Test
    void contextLoads() {
        log.info("Hello world");
    }

}

Then enable the annotation in META-INF/spring.factories:

org.springframework.test.context.ContextCustomizerFactory=\
  com.logarithmicwhale.demo.EnablePostgresTestContainerContextCustomizerFactory

Which can be implemented as:

public class EnablePostgresTestContainerContextCustomizerFactory implements ContextCustomizerFactory {

    @Target(ElementType.TYPE)
    @Retention(RetentionPolicy.RUNTIME)
    @Documented
    @Inherited
    public @interface EnabledPostgresTestContainer {
    }

    @Override
    public ContextCustomizer createContextCustomizer(Class<?> testClass,
            List<ContextConfigurationAttributes> configAttributes) {
        if (!(TestContextAnnotationUtils.hasAnnotation(testClass, EnabledPostgresTestContainer.class))) {
            return null;
        }
        return new PostgresTestContainerContextCustomizer();
    }

    @EqualsAndHashCode // See ContextCustomizer java doc
    private static class PostgresTestContainerContextCustomizer implements ContextCustomizer {

        private static final DockerImageName image = DockerImageName
                .parse("postgres")
                .withTag("14.1");

        @Override
        public void customizeContext(ConfigurableApplicationContext context, MergedContextConfiguration mergedConfig) {
            var postgresContainer = new PostgreSQLContainer<>(image);
            postgresContainer.start();
            var properties = Map.<String, Object>of(
                    "spring.datasource.url", postgresContainer.getJdbcUrl(),
                    "spring.datasource.username", postgresContainer.getUsername(),
                    "spring.datasource.password", postgresContainer.getPassword(),
                    // Prevent any in memory db from replacing the data source
                    // See @AutoConfigureTestDatabase
                    "spring.test.database.replace", "NONE"
            );
            var propertySource = new MapPropertySource("PostgresContainer Test Properties", properties);
            context.getEnvironment().getPropertySources().addFirst(propertySource);
        }

    }

}

Phalange

I'm working on a reusable testcontainers solution myself and really liked your approach a lot, but I'm testing it and it seems that return new PostgresTestContainerContextCustomizer(); will create a new instance every time it'll get hit, and that will dirty-up the context, forcing spring boot to reload it. Do I understand it wrong? Maybe it'll be best to cache the ContextCustomizer as a static field/singleton so it won't be created for every test class? – Yetac

I'm guessing you missed this in the example @EqualsAndHashCode // See ContextCustomizer java doc . The Javadoc explains why you need equals and hashcode and how that relates to the behaviour you see right now. – Phalange

See it in action here! github.com/jhipster/generator-jhipster/blob/main/generators/… – Circumvolution

@Circumvolution very cool! – Phalange

#77144907 – Maniacal

Great solution! Changing AnnotatedElementUtils.hasAnnotation to TestContextAnnotationUtils.hasAnnotation allows @Nested test classes to inherit the context customisation triggered by the annotation on the containing class. – Boll

Answer 5

The accepted answer has the caveat that the container will survive the execution of the tests, which may be not desirable as it can make your tests run not reproducible.

If the goal is to have only one PostgreSQL container instance across multiple test classes and have it discarded at the end of the test phase, it can be easily achieved without relying on reusable testcontainers.

First create an abstract class that will be in charge of the setup of the needed testcontainers

public abstract class BaseIntegrationTest {

    @ServiceConnection
    protected static final PostgreSQLContainer<?> dbContainer = new PostgreSQLContainer<>(
            DockerImageName.parse("postgres")
                    .withTag("15.4"))
            .withDatabaseName("testcontainer")
            .withUsername("user")
            .withPassword("pass");
 
    static {
        dbContainer.start();
    }
}

Implement each test class as an extension of this abstract class

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ExtendWith(SpringExtension.class)
public class BootstrapApplicationTest extends BaseIntegrationTest {

    @Test
    @DisplayName("My test case")
    void myTest() throws Exception {

        ...
    }

}

Notice that the @TestContainers annotation is not used here as it tells Spring to create a container per test class.

Selfrespect

Answer 6

Simply follow the guide from the testcontainers documentation. about the Singleton pattern. As they say - it is an option for JUnit5. Do not use the @Testcontainers annotation and @Container annotation. Those are related to the JUnit4. Also add to your class path testcontainers.properties file with:

testcontainers.reuse.enable=true

That did the trick for me.

Epa

its do not work. its do not see file in classpath and do not override default file – Melisma

Try also to remove the Testcontainers annotation. For jUnit5 that do the trick for me. Also use base test class with static block in which to start the containers. – Epa

@Melisma It does not see file in classpath because of conflict with Ryuk container. To fix it, go to IntelliJ Run Configuration and set environment variable TESTCONTAINERS_RYUK_DISABLED to equal true. – Brigandine

Answer 7

I'm not sure how @Testcontainers works, but I suspect it might work per class.

Just make your singleton static as described in Singleton pattern and get it in every test from your signleton holder, don't define it in every test class.

Pantheas

"The Singleton pattern doesn't help since every test starts a new application." – Serin