Run dotnet tests in docker which use testcontainers
Asked Answered
K

2

8

I have an integration tests project that executes as expected in VS. The integration tests use a MsSql testcontainer (from https://dotnet.testcontainers.org/).

My goal is to run these tests in an Azure DevOps pipeline within a docker image, as I do successfully for other projects which do not use testcontainers. For now I am just trying to run the tests within a docker image in my local machine. Unfortunately I am facing connection issues.

My environment:

  • .NET 6
  • OS: Windows
  • Docker Desktop with linux containers

My code:

Authentication.Api/MyProject.Authentication.Api/Dockerfile:

##########################################################
# build

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["Authentication.Api/MyProject.Authentication.Api/MyProject.Authentication.Api.csproj", "Authentication.Api/MyProject.Authentication.Api/"]
COPY ["Authentication.Api/MyProject.Authentication.Api.IntegrationTests/MyProject.Authentication.Api.IntegrationTests.csproj", "Authentication.Api/MyProject.Authentication.Api.IntegrationTests/"]
RUN dotnet restore "Authentication.Api/MyProject.Authentication.Api/MyProject.Authentication.Api.csproj"
RUN dotnet restore "Authentication.Api/MyProject.Authentication.Api.IntegrationTests/MyProject.Authentication.Api.IntegrationTests.csproj"
COPY . .

WORKDIR "/src/Authentication.Api/MyProject.Authentication.Api"
RUN dotnet build "MyProject.Authentication.Api.csproj" -c Release -o /app/build

WORKDIR "/src/Authentication.Api/MyProject.Authentication.Api.IntegrationTests"
RUN dotnet build -c Release

##########################################################
# run test projects

FROM build AS tests
WORKDIR /src
VOLUME /var/run/docker.sock:/var/run/docker.sock
RUN dotnet test --no-build -c Release --results-directory /testresults --logger "trx;LogFileName=testresults_authentication_api_it.trx" /p:CollectCoverage=true /p:CoverletOutputFormat=json%2cCobertura /p:CoverletOutput=/testresults/coverage/ -p:MergeWith=/testresults/coverage/coverage.json  Authentication.Api/MyProject.Authentication.Api.IntegrationTests/MyProject.Authentication.Api.IntegrationTests.csproj

##########################################################
# create image

FROM build AS publish
WORKDIR "/src/Authentication.Api/MyProject.Authentication.Api"
RUN dotnet publish "MyProject.Authentication.Api.csproj" -c Release -o /app/publish /p:UseAppHost=false

FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS final
WORKDIR /app
EXPOSE 80
EXPOSE 443
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "MyProject.Authentication.Api.dll"]

Authentication.Api/MyProject.Authentication.Api.IntegrationTests/Factory/CustomWebApplicationFactory.cs:

public class CustomWebApplicationFactory<TProgram, TDbContext> : WebApplicationFactory<TProgram>, IAsyncLifetime, ICustomWebApplicationFactory
    where TProgram : class
    where TDbContext : DbContext
{
    private readonly MsSqlDatabaseProvider _applicationMsSqlDatabaseProvider;

    public CustomWebApplicationFactory()
    {
        _applicationMsSqlDatabaseProvider = new MsSqlDatabaseProvider();
    }

    protected override void ConfigureWebHost(IWebHostBuilder builder)
        => builder.ConfigureServices(services =>
        {
            services.Remove(services.SingleOrDefault(d => d.ServiceType == typeof(DbContextOptions<ApplicationDbContext>)) ?? throw new InvalidOperationException());
            services.AddDbContext<ApplicationDbContext>(options => { options.UseSqlServer(_applicationMsSqlDatabaseProvider.Database.ConnectionString); });

            ServiceProvider? sp = services.BuildServiceProvider();
            using IServiceScope scope = sp.CreateScope();
            IServiceProvider scopedServices = scope.ServiceProvider;
            ILogger<CustomWebApplicationFactory<TProgram, TDbContext>> logger = scopedServices.GetRequiredService<ILogger<CustomWebApplicationFactory<TProgram, TDbContext>>>();

            ApplicationDbContext applicationDbContext = scopedServices.GetRequiredService<ApplicationDbContext>();
            applicationDbContext.Database.EnsureCreated();
            logger.LogInformation("Ensured that the ApplicationDbContext DB is created.");
        });

    public async Task InitializeAsync() =>
        await _applicationMsSqlDatabaseProvider.Database.StartAsync();

    public new async Task DisposeAsync() =>
        await _applicationMsSqlDatabaseProvider.Database.DisposeAsync().AsTask();
}

{shared library path}/MsSqlDatabaseProvider.cs:

public class MsSqlDatabaseProvider
{
    private const string DbPassword = "my_dummy_password#123";
    private const string DbImage = "mcr.microsoft.com/mssql/server:2019-latest";

    public readonly TestcontainerDatabase Database;

    public MsSqlDatabaseProvider() =>
        Database = new TestcontainersBuilder<MsSqlTestcontainer>()
            .WithDatabase(new MsSqlTestcontainerConfiguration
            {
                Password = DbPassword,
            })
            .WithImage(DbImage)
            .WithCleanUp(true)
            .Build();
}

On command line I run docker build --progress=plain -f Authentication.Api\MyProject.Authentication.Api\Dockerfile --target tests --tag myproject-tests ..

And I am getting the following error:

Cannot detect the Docker endpoint. Use either the environment variables or the ~/.testcontainers.properties file to customize your configuration: https://dotnet.testcontainers.org/custom_configuration/ (Parameter 'DockerEndpointAuthConfig')

I tried adding the environment variable in docker, changing dockerfile to

RUN export DOCKER_HOST="tcp://192.168.99.100:2376" && dotnet test --no-build -c Release --results-directory /testresults --logger "trx;LogFileName=testresults_authentication_api_it.trx" /p:CollectCoverage=true /p:CoverletOutputFormat=json%2cCobertura /p:CoverletOutput=/testresults/coverage/ -p:MergeWith=/testresults/coverage/coverage.json  Authentication.Api/MyProject.Authentication.Api.IntegrationTests/MyProject.Authentication.Api.IntegrationTests.csproj

and adding .WithDockerEndpoint("tcp://192.168.99.100:2376") in MsSqlDatabaseProvider, but I ended up with another error:

System.Net.Http.HttpRequestException : Connection failed

System.Net.Sockets.SocketException : Connection refused

I do not know what value(s) I should use for docker host / docker endpoint. Or is the solution something else?

Thank you in advance!

Karyolymph answered 11/1, 2023 at 12:9 Comment(5)
Should not it be something like RUN export DOCKER_HOST=tcp://host.docker.internal:2375 see more?Coy
@GuruStron I get the same: System.Net.Http.HttpRequestException : Connection failed | System.Net.Sockets.SocketException : Connection refusedKaryolymph
Your Docker image build does not contain or expose a Docker endpoint Testcontainers can connect to. You are trying to set up something that is called Docker-in-Docker. This works for docker run, but I am not aware of a configuration that works during the build.Shirl
Thanks, @AndreHofmeister. currently I am busy with other tasks. I will come back to this later this week or next week.Karyolymph
With the big help of @AndreHofmeister in a related GitHub discussion (github.com/testcontainers/testcontainers-dotnet/discussions/733), I managed to make it work using docker compose, running tests on run (and not on build) and with some more configurations. I will share my solution soon.Karyolymph
K
6

I could manage to do it, with two major differences:

  1. The tests do not run on the docker image, but rather on the docker container.
  2. I am using docker compose now.

docker-compose-tests.yml:

version: '3.4'

services:
  myproject.authentication.api.tests: # docker compose -f docker-compose-tests.yml up myproject.authentication.api.tests
    build:
      context: .
      dockerfile: Authentication.Api/MyProject.Authentication.Api/Dockerfile
      target: build
    command: >
        sh -cx "
                dotnet test /src/Authentication.Api/MyProject.Authentication.Api.IntegrationTests/MyProject.Authentication.Api.IntegrationTests.csproj -c Release --results-directory /testresults --logger \"trx;LogFileName=testresults_authentication_api_it.trx\" /p:CollectCoverage=true /p:CoverletOutputFormat=json%2cCobertura /p:CoverletOutput=/testresults/coverage/ -p:MergeWith=/testresults/coverage/coverage.json"
    environment:
      - TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal # Needed in Docker Desktop (Windows), needs to be removed on linux hosts. Can be done with a override compose file.
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - coverage:/testresults/coverage
    container_name: myproject.authentication.api.tests

("sh" command is useful if more test projects are expected to run.)

Authentication.Api/MyProject.Authentication.Api/Dockerfile:

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["Authentication.Api/MyProject.Authentication.Api/MyProject.Authentication.Api.csproj", "Authentication.Api/MyProject.Authentication.Api/"]
COPY ["Authentication.Api/MyProject.Authentication.Api.IntegrationTests/MyProject.Authentication.Api.IntegrationTests.csproj", "Authentication.Api/MyProject.Authentication.Api.IntegrationTests/"]
RUN dotnet restore "Authentication.Api/MyProject.Authentication.Api/MyProject.Authentication.Api.csproj"
RUN dotnet restore "Authentication.Api/MyProject.Authentication.Api.IntegrationTests/MyProject.Authentication.Api.IntegrationTests.csproj"
COPY . .

WORKDIR "/src/Authentication.Api/MyProject.Authentication.Api"
RUN dotnet build "MyProject.Authentication.Api.csproj" -c Release -o /app/build

WORKDIR "/src/Authentication.Api/MyProject.Authentication.Api.IntegrationTests"
RUN dotnet build -c Release

Authentication.Api/MyProject.Authentication.Api.IntegrationTests/Factory/CustomWebApplicationFactory.cs: same as in the question.

{shared library path}/MsSqlDatabaseProvider.cs:

public class MsSqlDatabaseProvider
{
    private const string DbImage = "mcr.microsoft.com/mssql/server:2019-latest";
    private const string DbUsername = "sa";
    private const string DbPassword = "my_dummy_password#123";
    private const ushort MssqlContainerPort = 1433;


    public readonly TestcontainerDatabase Database;

    public MsSqlDatabaseProvider() =>
        Database = new TestcontainersBuilder<MsSqlTestcontainer>()
            .WithDatabase(new MsSqlTestcontainerConfiguration
            {
                Password = DbPassword,
            })
            .WithImage(DbImage)
            .WithCleanUp(true)
            .WithPortBinding(MssqlContainerPort, true)
            .WithEnvironment("ACCEPT_EULA", "Y")
            .WithEnvironment("MSSQL_SA_PASSWORD", DbPassword)
            .WithWaitStrategy(Wait.ForUnixContainer().UntilCommandIsCompleted("/opt/mssql-tools/bin/sqlcmd", "-S", $"localhost,{MssqlContainerPort}", "-U", DbUsername, "-P", DbPassword))
            .Build();
}

And I can run the tests in docker with docker compose -f docker-compose-tests.yml up myproject.authentication.api.tests.

Karyolymph answered 1/2, 2023 at 7:8 Comment(3)
This works! But since I am running this in azure devops pipeline, I need to run docker compose up --abort-on-container-exit so that docker automatically exits after the all the tests have been run.Amarette
when you run the dotnet test command on the csproj (in the docker compose file) doesn't that make the building of the tests project in the dockerfile redundant? How could one use the docker image instead of essentially building it twice?Hand
follow-up: seems I misunderstood where the command runs. So it seems to run on the image. I just did in the command dotnet test --no-restore --no-build instead.Hand
S
5

Disclaimer:

I am a maintainer of Testcontainers for .NET and work as an engineer at AtomicJar, the company behind Testcontainers and Testcontainers Cloud.


Running Testcontainers in a Docker image build is very difficult. It is challenging (but possible) to provide access to a Docker endpoint running outside and independent of the docker build process. Usually, it requires a complex setup. As a much simpler solution, leveraging Testcontainers Cloud as part of the Docker image build works surprisingly well. The following Dockerfile configuration runs the Testcontainers Cloud agent inside the Docker image build (L:5):

FROM mcr.microsoft.com/dotnet/sdk:7.0
ARG TC_CLOUD_TOKEN
WORKDIR /tests
COPY . .
RUN TC_CLOUD_TOKEN=$TC_CLOUD_TOKEN curl -fsSL https://app.testcontainers.cloud/bash | bash && dotnet test

Running docker build --build-arg TC_CLOUD_TOKEN=${TC_CLOUD_TOKEN} . spins up the test dependencies in Testcontainers Cloud. I ran a simple test against a mssql/server:2022-latest container:


namespace DockerImageBuild.Test;

using System.Data.SqlClient;
using DotNet.Testcontainers.Builders;
using DotNet.Testcontainers.Configurations;
using DotNet.Testcontainers.Containers;
using Microsoft.Extensions.Logging;
using Xunit;

public class UnitTest1 : IAsyncLifetime
{
    private const string Database = "master";

    private const string Username = "sa";

    private const string Password = "yourStrong(!)Password";

    private const ushort MssqlContainerPort = 1433;

    private readonly TestcontainersContainer _dbContainer = new TestcontainersBuilder<TestcontainersContainer>()
        .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
        .WithPortBinding(MssqlContainerPort, true)
        .WithEnvironment("ACCEPT_EULA", "Y")
        .WithEnvironment("MSSQL_SA_PASSWORD", Password)
        .WithWaitStrategy(Wait.ForUnixContainer().UntilCommandIsCompleted("/opt/mssql-tools/bin/sqlcmd", "-S", $"localhost,{MssqlContainerPort}", "-U", Username, "-P", Password))
        .Build();

    [Fact]
    public Task Test1()
    {
        var connectionString = $"Server={_dbContainer.Hostname},{_dbContainer.GetMappedPublicPort(MssqlContainerPort)};Database={Database};User Id={Username};Password={Password};";

        using (var sqlConnection = new SqlConnection(connectionString))
        {
            try
            {
                sqlConnection.Open();
            }
            catch
            {
                Assert.Fail("Could not establish database connection.");
            }
            finally
            {
                TestcontainersSettings.Logger.LogInformation(connectionString);
            }
        }

        return Task.CompletedTask;
    }

    public Task InitializeAsync()
    {
        return _dbContainer.StartAsync();
    }

    public Task DisposeAsync()
    {
        return _dbContainer.DisposeAsync().AsTask();
    }
}

Make sure you are using a multi-stage build and not expose your token in a layer.

Shirl answered 18/1, 2023 at 11:52 Comment(1)
This solution sounds promising, but at the moment we have some privacy and pricing concerns about it. I will give it some more tries without it, but I might come back and spend time exploring and understanding Testcontainers.cloud if I do not succeed otherwise. Thanks for the suggestion!Karyolymph

© 2022 - 2024 — McMap. All rights reserved.