Cannot find Docker environment for Test Container (Mysql) in Spring Boot

Question

I have a problem to run docker-compose up -d to run Spring Boot on Docker. I have a MySQL database connection issue when the app tries to run integration test with test container.

As I cannot connect to MySQL provided by test container, I got this error shown below

Here are logs shown

#0 131.6 2023-09-17T15:12:16.594Z  INFO 103 --- [           main] com.zaxxer.hikari.HikariDataSource       : mysql - Starting...
#0 132.8 2023-09-17T15:12:17.783Z ERROR 103 --- [           main] com.zaxxer.hikari.pool.HikariPool        : mysql - Exception during pool initialization.
#0 132.8
#0 132.8 com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
 The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure


#0 139.8 [ERROR]   AuthControllerTest>AbstractTestContainerConfiguration.beforeAll:17 » IllegalState Previous attempts to find a Docker environment failed. Will not retry. Please see l
ogs and check configuration
#0 139.8 [ERROR]   BookControllerTest>AbstractTestContainerConfiguration.beforeAll:17 » IllegalState Previous attempts to find a Docker environment failed. Will not retry. Please see l
ogs and check configuration
#0 139.8 [ERROR]   OrderControllerTest>AbstractTestContainerConfiguration.beforeAll:17 » IllegalState Could not find a valid Docker environment. Please see logs and check configuration
#0 139.8 [ERROR]   StatisticsControllerTest>AbstractTestContainerConfiguration.beforeAll:17 » IllegalState Previous attempts to find a Docker environment failed. Will not retry. Please
 see logs and check configuration

Here is the Dockerfile shown below

# Stage 1: Build stage
FROM maven:3.8.4-openjdk-17-slim AS build

# Copy Maven files for dependency resolution
COPY pom.xml ./
COPY .mvn .mvn

# Copy application source code
COPY src src

# Build the project and create the executable JAR
RUN mvn package

# Stage 2: Run stage
FROM openjdk:17-jdk-slim

# Set working directory
WORKDIR bookdelivery

# Copy the JAR file from the build stage
COPY --from=build target/*.jar bookdelivery.jar

# Expose port 1221
EXPOSE 1221

# Set the entrypoint command for running the application
ENTRYPOINT ["java", "-jar", "bookdelivery.jar"]

Here is the docker-compose.yml shown below

version: "3.9"

services:
  database:
    container_name: database
    image: mysql:8.0.33
    restart: always
    env_file:
      - .env  # Use the .env file for environment variables
    environment:
      MYSQL_DATABASE: bookdelivery
      MYSQL_PASSWORD: ${DATABASE_PASSWORD}
      MYSQL_ROOT_PASSWORD: ${DATABASE_PASSWORD}
      MYSQL_ROOT_HOST: '%'
      MYSQL_PORT: 3307
    volumes:
      - ./db:/var/lib/mysql
    ports:
      - "3307:3306"
    networks:
      - bookDeliveryNetwork

  bookdelivery:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: bookdelivery
    restart: on-failure
    env_file:
      - .env  # Use the .env file for environment variables
    ports:
      - "1221:1221"
    environment:
      - server.port=1221
      - spring.datasource.username=${DATABASE_USERNAME}
      - spring.datasource.password=${DATABASE_PASSWORD}
      - BOOK_DELIVERY_DB_IP=database
      - BOOK_DELIVERY_DB_PORT=3307
      - spring.datasource.url=jdbc:mysql://database:3307/bookdelivery
    depends_on:
      - database
    networks:
      - bookDeliveryNetwork


networks:
  bookDeliveryNetwork:

Here is the testContainer shown below

@Testcontainers
public abstract class AbstractTestContainerConfiguration {

    static MySQLContainer<?> MYSQL_CONTAINER = new MySQLContainer<>("mysql:8.0.33");

    @BeforeAll
    static void beforeAll() {
        MYSQL_CONTAINER.withReuse(true);
        MYSQL_CONTAINER.start();
    }

    @DynamicPropertySource
    private static void overrideProps(DynamicPropertyRegistry dynamicPropertyRegistry) {
        dynamicPropertyRegistry.add("spring.datasource.username", MYSQL_CONTAINER::getUsername);
        dynamicPropertyRegistry.add("spring.datasource.password", MYSQL_CONTAINER::getPassword);
        dynamicPropertyRegistry.add("spring.datasource.url", MYSQL_CONTAINER::getJdbcUrl);
    }

}

Here is the application.yml of test shown below

spring:
  config:
    import: optional:file:.env[.properties]
  sql:
    init:
      mode: always
  jpa:
    properties:
      hibernate:
        dialect: org.hibernate.dialect.MySQLDialect
        format_sql: true
    hibernate:
      naming:
        physical-strategy: org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl

  datasource:
    name: mysql
    username: ${DATABASE_USERNAME:root}
    password: ${DATABASE_PASSWORD:password}
    url: jdbc:mysql://${BOOK_DELIVERY_DB_IP:localhost}:${BOOK_DELIVERY_DB_PORT:3306}/bookdelivery
    maximum-pool-size: 5
    connection-timeout: 180000
    maximum-lifetime: 170000


jwt:
  secret: 404E635266556A586E3272357538782F413F4428472B4B6250645367566B5970
  expireMs: 60000
  refrEshexpireMs: 120000

Here is the application.yml of main shown below

server:
  port: 1221

# MYSQL
spring:
  config:
    import: optional:file:.env[.properties]
  datasource:
    url: jdbc:mysql://${BOOK_DELIVERY_DB_IP:localhost}:${BOOK_DELIVERY_DB_PORT:3306}/bookdelivery
    username: ${DATABASE_USERNAME:root}
    password: ${DATABASE_PASSWORD:password}
  jpa:
    hibernate:
      ddl-auto: update
    properties:
      hibernate:
        dialect: org.hibernate.dialect.MySQLDialect
      show-sql: true

# JWT
jwt:
  secret: 404E635266556A586E3272357538782F413F4428472B4B6250645367566B5970
  expireMs: 600000 # 10 Minutes
  refrEshexpireMs: 120000

# SWAGGER
springdoc:
  api-docs:
    enabled: true
  show-actuator: true

How can I fix it?

Here is the repo : Link

Crosswind

show docker service ls. – Gav

I'm a little confused how Testcontainers comes into this setup: you're already starting a standalone MySQL instance in a separate Compose container, and you should be able to just configure spring.datasource.url to point at it, without needing to start a new container (and without the attendant security concerns). – Burr

@RichardRublev As I cannot run the app through docker-compose up --build , I cannot see the docker services by docker service ls. – Crosswind

@DavidMaze I already configured spring.datasource.url – Crosswind

Answer 1

Few things are wrong and overly complicated here:

First, inside your ‘docker-compose.yml’ file for the database service, “MYSQL_USER” is not set.

Secondly i beleive your ‘.env’ file or your enviorment file is containing two environment variable named “DATABASE_USERNAME” and “DATABASE_PASSWORD”.

Thirdly, it’s not mandatory, but things getting complicated in management perspective to set the “MYSQL_PORT” to 3306. Then exposing this port to the host in port 3306. Keep the port 3306 in the container end and expose that port (if you want to access it directly from the host) otherwise no need this exposure. Simplifying the “spring.datasource.url” values port part to “3306”

Fourth, follows the third, we are setting “spring.datasource.url” directly, then we don’t need to set the “BOOK_DELIVERY_DB_IP” and “BOOK_DELIVERY_DB_PORT” setting. because they got overidden by the setting of “spring.datasource.url”

Now the most important thing,

like every database services out there, mysql need time to get ready for incoming connection. You can call it readiness of database service. You already specified that the your application service is dependent on the database service, but thats not enough, because, docker have no way to know, that your database service is ready for incoming connection. In that case, you case introduce HEALTHCHECK for your database service, which will tell docker that your database service is healthy, and then docker tries to spin up your application container.

Simplest healthcheck can be formulated as following

healthcheck:
      test: ["CMD-SHELL", "/usr/bin/mysql --user=webrest-user --password=secret --execute \"SHOW DATABASES;\""]
      interval: 30s
      timeout: 10s
      retries: 10

Complete working implementation can be found here https://github.com/ratulSharker/webrest-starter

Happy coding.

Edited

1 ) You don't need to use MYSQL_USER 2 ) DATABASE_USERNAME and DATABASE_PASSWORD come from .env file

There is a problem to get variables from AbstractTestContainerConfiguration

I tried to run docker compose config and here is the result shown below

name: bookdelivery
services:
  bookdelivery:
    build:
      context: C:\Users\Username\IdeaProjects\bookdelivery
      dockerfile: Dockerfile
    container_name: bookdelivery
    depends_on:
      database:
        condition: service_started
    environment:
      BOOK_DELIVERY_DB_IP: database
      BOOK_DELIVERY_DB_PORT: "3307"
      DATABASE_PASSWORD: value
      DATABASE_USERNAME: value
      server.port: "1221"
      spring.datasource.password: value
      spring.datasource.url: jdbc:mysql://database:3307/bookdelivery
      spring.datasource.username: value
    networks:
      bookDeliveryNetwork: null
    ports:
    - mode: ingress
      target: 1221
      published: "1221"
      protocol: tcp
    restart: on-failure
  database:
    container_name: database
    environment:
      DATABASE_PASSWORD: value
      DATABASE_USERNAME: root
      MYSQL_DATABASE: value
      MYSQL_PASSWORD: value
      published: "3307"
      protocol: tcp
    restart: always
    volumes:
    - type: bind
      source: C:\Users\Username\IdeaProjects\bookdelivery\db
      target: /var/lib/mysql
      bind:
        create_host_path: true
networks:
  bookDeliveryNetwork:
    name: bookdelivery_bookDeliveryNetwork

I can get the BOOK_DELIVERY_DB_IP, BOOK_DELIVERY_DB_PORT, DATABASE_USERNAME and DATABASE_PASSWORD.

Can you revise the answer?

Ingrowth

Can you revise the answer? – Crosswind

Are you connecting to your containerised database using the root user ? if it the case, then you don’t need to set your MYSQL_USER variable set. – Ingrowth

I can run all junit service test but I cannot run integration with testcontainer as MySQL cannot be connect through test container. – Crosswind

Can you connect to your mysql database using any database client after starting the database container ? – Ingrowth

Another thing, from your docker compose config, your MYSQL_DATABASE value is set to “value” where it should be the last part of your “spring.datasource.url” (bookdelivery). – Ingrowth

I can get the right value. I want to show them as value. – Crosswind

If you can connect to the database from host using any database client, then it’s the healthcheck issue. Please place them appropriately, you will be fine. If you are unable to connect to the e database using client, please configure the values correctly, then try setting the healthcheck. – Ingrowth

I want to share some screenshots. ibb.co/QfHZSdg ibb.co/tp9Q9Pk . As you can see, ControllerTest cannot work because it needs to connect mysql database through test container. – Crosswind

The reason is your database container is not ready for connection yet. – Ingrowth

How can I solve database container issue? – Crosswind

Mate please follow this comment #77122954 – Ingrowth

I added healthcheck with my configuration (as username and password) then I run docker-compose up -d. Next, I get the same error. – Crosswind

change the depends on check from “database: condition: service_started” to “service_healthy” – Ingrowth

I added condition: service_healthy under database of depend_on but nothing changed. I still get the same issue. Is it possible to look through my docker-compose or AbstractTestContainerConfiguration in my repo ? – Crosswind

can you access your db from host machine via a db client ? – Ingrowth

I shared two img link. As there is a problem, I cannot see services through docker ps -a. I can run it through mvn clean install and I get build success message. – Crosswind

just spin up the database service “docker compose up database -d” then try connecting to that database using any db client from host. – Ingrowth

I run mvn clean install through terminal. I forgot to say that. – Crosswind

I still couldn't fix it. – Crosswind

Have you tried it ? #77122954 – Ingrowth

I already tried it but nothing changed. Is it possible to test if you don't mind? – Crosswind

start the database container, and then try connecting with the database with a database client like dbeaver. Don’t start your application container to start testing. can you connect to your database using a UI RDBMS client ? – Ingrowth

Which command do I use? – Crosswind

Is it possible to test it for me if you don't mind? – Crosswind

I tried to revise the dockerfile. Here is the result. CI/CD works but there is still MySQL connection. ibb.co/Fqrc1WZ ibb.co/KbpxyHc – Crosswind

Answer 2

I fixed the issue

Change mvn package to mvn clean install -DskipTests in Dockerfile as Testcontainer tries to find Docker itself but it cannot find it. That's why I have to skip tests.

Crosswind