How to start a full .NET Core Worker Service in an integration/ end-to-end test?
Asked Answered
T

4

9

TLDR: To integration test a .NET Core web application, we have the WebApplicationFactory. How do I do the same thing for a worker service ?

Given this worker service:

// Program.cs

using SampleWorker;

IHost host = Host.CreateDefaultBuilder(args)
    .ConfigureServices(services => { services.AddHostedService<Worker>(); })
    .Build();

host.Run();
// Worker.cs

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;

    public Worker(ILogger<Worker> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        // - connect to a middleware (data source)
        // - connect to database (data sink)
        // - received events from middleware, transform + persist in database
    }
}

I want to be able to write an end-to-end test which provides data source and sink via testcontainers (works) and the unit under test, the worker service, somehow directly via .NET Core. With an web application you would use the WebApplicationFactory<Program>. I am looking for a way to do the same with a Worker service.

Here is the test I'd like to be able to write:

public class E2eTest
{
    /// <summary>
    /// Tests the full round trip from an event being
    /// - picked up by the worker service
    /// - processed and transformed
    /// - persisted to the database.
    /// </summary>
    [Fact]
    public void EventIsProcessedAndWrittenToDatabase()
    {
        // arrange
        // - start middleware as data source (testcontainers)
        // - start database as data sink (testcontainers)
        
        // TODO there is no WorkerFactory.
        //      How can I start my SampleWorker with all the config in Program.cs ?
        var unitUnderTest = new WorkerFactory<Program>();
        
        // act
        // - publish an event to the middleware (data source)
        
        // assert
        // - check that there are entries in the database (data sink)
    }
}

I've got a similar version of this test already working by running the SamplerWorker also as a Docker container. This utilizes Testcontainer's ImageFromDockerfileBuilder. However, I want to be able to easily debug my unit under test which gets really complicated with my current Docker approach.

Any ideas? Thanks ๐Ÿ™‚

Terzetto answered 1/10, 2023 at 18:29 Comment(5)
If you extracted a method instead of implicit main and called it in a global test startup, would that not work? You might even run instance per test, when it comes to it. It's just regular C# code. Most of what the web server factory does is exposing in-process http client pseudo-connection. So you might need to do the same for your IO ports. โ€“ Hypogastrium
Basically what I'd do in your case is: Extract a startup class from your setup, extract a factory that creates a host builder with startup type as generic parameter, create a derived startup class for testing where you override the bare minimum of services you need to run your scenarios, call the factory with the test startup parameter from your global test setup and with the production startup from your main. โ€“ Hypogastrium
@Zdenฤ›kJelínek Thank you for sharing your approach. I admit I've been a framework user so far with focus on the application logic. I will try to follow your suggestions and see where I end up. Maybe you don't mind sharing some code example yourself ? ๐Ÿ™‚ โ€“ Terzetto
I'll do it if I find the time but I don't see that happening in the coming days. โ€“ Hypogastrium
Just for my understanding: why not use var sut = new WebApplicationFactory<Program>().Services.GetRequiredService<Worker>()? โ€“ Kessler
E
4

Turns out you can use WebApplicationFactory to test your worker service.

public class HostApplicationFactory<TEntryPoint> : WebApplicationFactory<TEntryPoint> 
    where TEntryPoint : class
{
    private readonly Action<IWebHostBuilder> _configuration;

    public HostApplicationFactory(Action<IWebHostBuilder> configuration)
    {
        _configuration = configuration;
    }

    protected override void ConfigureWebHost(IWebHostBuilder builder) =>
        _configuration(builder.Configure(_ => { }));

    public Task RunHostAsync()
    {
        var host = Services.GetRequiredService<IHost>();
        return host.WaitForShutdownAsync();
    }
}

After dissecting WebApplicationFactory, I found it was just easier/cleaner to leverage what it already offers rather than trying to create some kind of new factory for host applications. The trick to making this thing work is the builder.Configure(_ => { }), which prevents a startup error. I still don't completely understand what's happening here but another post mentions that a web host is needed and this creates an empty web app. It's buried pretty deep and is in some convoluted startup task logic.

Calling the service provider will start your host, so you can just wait for it to stop, or return a reference to IHost and stop in manually depending on what you need.

You can use it like this in your test:

await using HostApplicationFactory<Program> hostApplicationFactory =
    new(builder =>
    {
        builder.ConfigureTestServices(services =>
        {
            // Override any services here
        });
    });

await hostApplicationFactory.RunHostAsync();
Ezara answered 30/12, 2023 at 20:29 Comment(1)
thanks Eric! This is simpler I'd expected and works for my simple hello-world worker nicely! The only change I made was to pass in a cancellation token to RunHostAsync which is then passed down to WaitForShutdownAsync. With that my test was able to start & stop (via canceling the token) my demo worker. I'll now try to apply it to my big application. Thanks again ๐Ÿ‘๐Ÿฝ โ€“ Terzetto
S
1

TL;DR: Here's my current implementation in a gist.

Based loosely on Microsoft's WebApplicationFactory, I came up with this. I'm sure there are scenarios it doesn't handle and there is plenty of room for improvement, but it gets me what I wanted: a worker service running in-process just like with WebApplicationFactory. This is using the HostApplicationBuilder, while you're using IHostBuilder. However, the code is easily adaptable to IHostBuilder if you want to continue using it. The gist linked above does handle IHostBuilder properly.

Create a discoverable method to hook into

Microsoft's testing framework looks for methods named BuildWebHost, CreateWebHostBuilder, and CreateHostBuilder on the assembly entrypoint. The framework will invoke these methods to get what they need to create an in-process host. However, in a worker service our entry point is Program.cs and the only declared method is <Main>$. So we'll need to give ourselves a method to retrieve a builder. We'll also declare Program as public; that's important later on.

Our Program.cs becomes

var builder = CreateHostBuilder(new HostApplicationBuilderSettings { Args = args });
// If there's any builder setup you don't want in the tests, such as auth, do it here.
var host = builder.Build();
host.Run();


public sealed partial class Program
{
    private static HostApplicationBuilder CreateHostBuilder(HostApplicationBuilderSettings settings)
    {
        var builder = Host.CreateApplicationBuilder(settings);
        // The rest of your setup. builder.Services.AddStuff(), etc.
        // Any builder setup you want to make it into the tests
        // MUST be done in here.
        ...
    }
}

Create a factory to spin up a host

Now, we need a factory which will take a generic parameter, create our builder, and start up a host. I created a file called WorkerServiceFactory.cs for this. We'll also want a second class to delay our host startup so that IAsyncLifetime will work properly with our factory. I've named this TestWorkerService.cs We'll take it step by step here, with the full code provided below.

NOTE: You could do a fancier factory method lookup which would handle more scenarios. The gist I linked above does exactly that. I've opted to keep it simpler here.

TestWorkerService.cs

All of our work here can be done in the constructor. We receive an argument with the builder from which to create our host, and an argument which will configure the builder. We then build the host and start it, being sure not to block the thread waiting for the host to complete.

public sealed class TestWorkerService : IDisposable
{
    internal readonly IHost Host;

    internal TestWorkerService(
        HostApplicationBuilder builder,
        Action<HostApplicationBuilder> configure
    )
    {
        configure(builder);
        Host = builder.Build();
        Host.StartAsync().GetAwaiter().GetResult();
    }

    public void Dispose()
    {
        Host.Dispose();
    }
}

WorkerServiceFactory.cs

First, we need to discover our application's entrypoint.

var entryPoint = typeof(T).Assembly.EntryPoint!.DeclaringType!;

Then, we'll find the method we created earlier in our Program.cs. I'm using a property (provided below) for the method name so that you could change it if you wanted to. You'd just need to change the method name in your Program.cs, then override the CreateBuilderMethodName property.

var hostBuilderFactory = entryPoint.GetMethod(
    CreateBuilderMethodName,
    BindingFlags.Static | BindingFlags.NonPublic
)!;

We'll construct the settings to pass to the builder, setting the environment name to a variable named Environment (provided below). Then, we invoke the factory method with these settings and cast the return type to HostApplicationBuilder.

var settings = new HostApplicationBuilderSettings
{
    ApplicationName = entryPoint.Assembly.GetName().Name,
    EnvironmentName = Environment
};
_builder = (HostApplicationBuilder)hostBuilderFactory.Invoke(null, new object?[] { settings })!;

Now, we'll create a field and property to hold our TestWorkerService. Waiting to initialize _workerService until it's accessed will prevent our TestWorkerService from starting up our host until after InitializeAsync(), if we're using IAsyncLifetime. This allows, for example, a Docker container to be started and accessible when we call ConfigureHost.

protected virtual void ConfigureHost(HostApplicationBuilder builder) { }

private TestWorkerService? _workerService;
public TestWorkerService WorkerService =>
    _workerService ??= new TestWorkerService(_builder, ConfigureHost);

Here's the full code:

public abstract class WorkerServiceFactory<T> : IDisposable
{
    private readonly HostApplicationBuilder _builder;

    protected WorkerServiceFactory()
    {
        var entryPoint = typeof(T).Assembly.EntryPoint!.DeclaringType!;
        var hostBuilderFactory = entryPoint.GetMethod(
            CreateBuilderMethodName,
            BindingFlags.Static | BindingFlags.NonPublic
        )!;
        var settings = new HostApplicationBuilderSettings
        {
            ApplicationName = entryPoint.Assembly.GetName().Name,
            EnvironmentName = Environment
        };
        _builder = (HostApplicationBuilder)
            hostBuilderFactory.Invoke(null, new object?[] { settings })!;
    }

    protected virtual void ConfigureHost(HostApplicationBuilder builder) { }

    public void Dispose()
    {
        _workerService?.Dispose();
        ServiceScope.Dispose();
        GC.SuppressFinalize(this);
    }

    public void Start()
    {
        _ = WorkerService;
    }

    protected virtual string CreateBuilderMethodName { get; } = "CreateHostBuilder";
    protected virtual string Environment { get; } = Environments.Development;

    public IServiceProvider Services => WorkerService.Host.Services;

    private TestWorkerService? _workerService;
    public TestWorkerService WorkerService =>
        _workerService ??= new TestWorkerService(_builder, ConfigureHost);

    // Optional: creates a service scope for you.
    /* public TService GetService<TService>()
        where TService : class => ServiceScope.ServiceProvider.GetRequiredService<TService>();

    private IServiceScope? _serviceScope;
    private IServiceScope ServiceScope => _serviceScope ??= Services.CreateScope();*/
}

Create a derived factory and perform any final configuration

All that's left to do is to create the factory you want to use in your tests. We'll inherit from our WorkerServiceFactory for this, passing our (now public) Program as the assembly marker.

public sealed class MyWorkerFactory : WorkerServiceFactory<Program>
{
    protected override void ConfigureHost(HostApplicationBuilder builder)
    {
        // Configure your test services via builder.Services.
        // Configure your config via builder.Configuration.
    }
}

You should now be able to use this just like you would the WebApplicationFactory, minus some more advanced features which you could add if you want.

Sedan answered 30/11, 2023 at 18:38 Comment(0)
R
0

There is one problem with this approach, you did not solve and we did. Problem is descibed as following: what if we want to use two or more Worker services, each of which has of course its own appsettings.json file. But current approach do not perform change of its ContentRootPath to assembly source root directory (project directory), but it takes it from Executable directory, which of course never has both files of appsettings.json from both worker services, because they have same names.

Indeed WebApplicationFactory has its own solution which based on directory upper traversal from assembly executable directory and then assign it to content root of builder. We have really smart solution.

private static string GetThisFilePath([CallerFilePath] string path = "")
{
    return path;
}

private static HostApplicationBuilder CreateHostBuilder(HostApplicationBuilderSettings settings)
{
    var path = GetThisFilePath();
    var builder = GetApplicationHostBuilder(settings, path);
}

public static HostApplicationBuilder GetApplicationHostBuilder(HostApplicationBuilderSettings settings, string path)
 {
     var directory = Path.GetDirectoryName(path);
     if (settings != null)
         settings.ContentRootPath = directory;
     return Host.CreateApplicationBuilder(settings);
 }

Note private static string GetThisFilePath([CallerFilePath] string path = "") method should be placed in Program.cs file to get desired effect. System.Runtime.CompilerServices CallerFilePath attribute has runtime capabilities to solve source path and working even in Release build, without any pdbs.

Reluctance answered 30/12, 2023 at 8:41 Comment(0)
C
0

Here is my solution in dotnet 7

_Factory is WebApplicationFactory

  var host = _Factory.Services.GetService<IHost>() ?? throw new Exception("host is null");
        await host.StartAsync(); // need to start the host first
        var hostedServices = host.Services.GetServices<IHostedService>();
        var mainBackgroundService = hostedServices.OfType<MainBackgroundService>().FirstOrDefault(); // to get your background service
        _Cts = new CancellationTokenSource(); // dispose _Cts later
try {
   await mainBackgroundService.StartAsync(_Cts.Token);
// start your test

}
finally{
   await mainBackgroundService.StopAsync(CancellationToken.None);
}

Coverup answered 15/1 at 20:22 Comment(0)

© 2022 - 2024 โ€” McMap. All rights reserved.