Using WebApplicationFactory to do E2E testing with .Net 6 (minimal api)

Question

Im trying to get E2E/UI testing (selenium, playwright) to work with my unit testing framework.

The basic idea it to use MSTest and the WebApplicationFactory to spin up a "real server" from within my unit tests. The reason for doing this would be to simply avoid having to deploy/release my application for testing (I guess this could be done by using containers etc, sadly.. Im not allowed to use containers). Im also thinking that doing it this wasy would be a "neat" way of mocking any code that calls external services, and be able to create multiple tests with different scenarios for those external calls.

I have searched the web for a way of doing this, but all I can find are posts about how to do this in previous .Net versions (2.1-5), but since .Net 6 the "startup ceremonial" code has changed and the standard way is now to use the minimal API.

Here is a blog post from Scott.H where he is basically doing exactly what Im planing to do, but with .Net 2.1: https://www.hanselman.com/blog/real-browser-integration-testing-with-selenium-standalone-chrome-and-aspnet-core-21

What I have done so far is creating a custom class that inherits from WebApplicationFactory.

basically:

class MyAppFactory : WebApplicationFactory<Program> {

}

And I can use that perfectly fine for integration testing. However.. the server thats initialized when using that class does not accept http-calls, so I cant reach that using a web browser, and neither can selenium.

I tried to follow Scotts blog post. But for some reason the:

protected override TestServer CreateServer(IWebHostBuilder builder)

Is never called.. (not sure if that has with minimal APIs and .Net 6 to do).

Has anyone managed to use the WebApplicationFactory and .Net 6 Minimal API to spin up an "actual server" in memory that accepts http-calls?

Guild

I am using .net 6, and I am setting up my controllers as normal classes, and I am able to use http calls in my integration tests - that might be obvious. I am just wondering,- do you have to use minimal APIs? – Uigur

@aIKid, no Im not forced to use the minimap apis, but since thats what the dotnet new templates "uses" as default, it would be nice to simply just use the default :). Are you able to use an actual browser to "visit" your pages/api as well? Or do you mean that your able to do http-class using the httpClient provided by the WebApplicationFactory? – Guild

That's a good point, I have not actually tried visiting it through browser. I had just been using the HttpClient. I'll try using the browsersometime! – Uigur

Answer 1

I faced the same problem while migrating to .NET 6 and found the solution thanks to this blog post from Marius Steinbach.

Steps:

• Make sure there is the Program definition in Program.cs file

var builder = WebApplication.CreateBuilder(args);

// adds services to the container
builder.Services.AddRazorPages();

var app = builder.Build();

// configures the HTTP request pipeline
app.UseExceptionHandler("/Error");
app.UseHsts();
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthorization();
app.MapRazorPages();

app.Run();

public partial class Program { }

• Create a new class WebApplicationFactoryFixture (sample)

public class WebApplicationFactoryFixture<TEntryPoint> : WebApplicationFactory<TEntryPoint>
    where TEntryPoint : class
 {
    public string HostUrl { get; set; } = "https://localhost:5001"; // we can use any free port

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.UseUrls(HostUrl);
    }

    protected override IHost CreateHost(IHostBuilder builder)
    {
        var dummyHost = builder.Build();

        builder.ConfigureWebHost(webHostBuilder => webHostBuilder.UseKestrel());

        var host = builder.Build();
        host.Start();

        return dummyHost;
    }
}

• Use this factory in the test class (sample)

public class SmokeSeleniumTest : IClassFixture<WebApplicationFactoryFixture<Program>>
{
    private readonly string _webUrl = "https://localhost:7112";

    public SmokeSeleniumTest(WebApplicationFactoryFixture<Program> factory)
    {
        factory.HostUrl = _webUrl;
        factory.CreateDefaultClient();

        var chromeOptions = new ChromeOptions();
        // ...
        WebDriver = new ChromeDriver(chromeDriverLocation, chromeOptions);
    }

    protected IWebDriver WebDriver { get; }

    [Theory]
    [InlineData("/", "Welcome")]
    [InlineData("/Index", "Welcome")]
    [InlineData("/Privacy", "Privacy Policy")]
    [InlineData("/Error", "Error.")]
    public void Get_EndpointsReturnSuccessAndCorrectContentType(string url, string expected)
    {
        // Arrange & Act
        WebDriver.Navigate().GoToUrl($"{_webUrl}{url}");

        // Assert
        WebDriver.FindElement(By.TagName("h1"), 30);
        WebDriver.FindElement(By.TagName("h1")).Text.Should().Contain(expected);
    }
}

It works!

Samples are validated through a CI pipeline in Azure DevOps in this repository.

Gifferd

I'm so glad you included the solution in your answer, because the blog posts is not available anymore. And the solution works for me! Thanks – Continent

Answer 2

It looks like you need to call CreateServer explicitly. This blog post shows how the author fixed this when migrating from 2.2 to 3.1 I don't know if there are better solutions in .NET 6, but this should at least fix your issue.

Gelb

Answer 3

The solution provided by @devpro works but it's not ideal, because you actually start two servers at once, one with kestrel which you can then interact with on localhost and one in memory, which you can debug, but they are separate instances! Which can be super confusing while debugging, beucase breakpoints will hit twice once with kestrel instance and once with in-memory.

Here is a version which works with one instance only, the only downside is that we need to use some reflection in order to start the underlying TeseServer, because it implements the IServer interface explicitly.

All the other steps should work as @devprov has explained them.

using Microsoft.AspNetCore.Connections;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.Server.Kestrel.Core;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace Simulation;

public class KestrelTestServer : TestServer, IServer
{
    private readonly KestrelServer _server;

    public KestrelTestServer(IServiceProvider serviceProvider) : base(serviceProvider)
    {
        // We get all the transport factories registered, and the first one is the correct one
        // Getting the IConnectionListenerFactory directly from the service provider does not work
        var transportFactory = serviceProvider.GetRequiredService<IEnumerable<IConnectionListenerFactory>>().First();

        var kestrelOptions = serviceProvider.GetRequiredService<IOptions<KestrelServerOptions>>();
        var loggerFactory = serviceProvider.GetRequiredService<ILoggerFactory>();
        _server = new KestrelServer(kestrelOptions, transportFactory, loggerFactory);
    }

    async Task IServer.StartAsync<TContext>(IHttpApplication<TContext> application, CancellationToken cancellationToken)
    {
        // We need to also invoke the TestServer's StartAsync method to ensure that the test server is started
        // Because the TestServer's StartAsync method is implemented explicitly, we need to use reflection to invoke it
        await InvokeExplicitInterfaceMethod(nameof(IServer.StartAsync), typeof(TContext), [application, cancellationToken]);

        // We also start the Kestrel server in order for localhost to work
        await _server.StartAsync(application, cancellationToken);
    }

    async Task IServer.StopAsync(CancellationToken cancellationToken)
    {
        await InvokeExplicitInterfaceMethod(nameof(IServer.StopAsync), null, [cancellationToken]);
        await _server.StopAsync(cancellationToken);
    }

    private Task InvokeExplicitInterfaceMethod(string methodName, Type? genericParameter, object[] args)
    {
        var baseMethod = typeof(TestServer).GetInterfaceMap(typeof(IServer)).TargetMethods.First(m => m.Name.EndsWith(methodName));
        var method = genericParameter == null ? baseMethod : baseMethod.MakeGenericMethod(genericParameter);
        var task = method.Invoke(this, args) as Task ?? throw new InvalidOperationException("Task not returned");
        return task;
    }
}

internal class WebApplicationFactoryWithKestrel : WebApplicationFactory<Program>
{
    private readonly int _port;

    public WebAppFactoryWithKestrel(int port)
    {
        _port = port;
    }

    protected override IHost CreateHost(IHostBuilder builder)
    {
        builder.ConfigureWebHost(webHostBuilder =>
        {
            webHostBuilder.UseKestrel(opt => opt.ListenLocalhost(_port));
            webHostBuilder.ConfigureServices(s => s.AddSingleton<IServer, KestrelTestServer>());
        });

        return base.CreateHost(builder);
    }
}

Abolition

Why do you need to start underlying TestServer? I tried without it and it worked ok. What am I missing? – Toner

Without the TestServer when you try to call the CreateClient method to communicate via the server inside tests, you will get an InvalidCastException as the testing framework will try to cast the KestrelServer int the TestServer – Abolition

I mean keep KestrelTestServer but start only underlying KestrelServer, do not use reflection to start TestServer. W/o KestrelTestServer I do get InvalidCastException exception, right. – Toner

If you won't start the underlaying TestServer and start only the KestrelServer you will get an InvalidOperationException: The server has not been started or no web application was configured. – Abolition

Hmmm, I don't get this error. m.b. because I don't call CreateClient. I access factory.Server directly. This getter ensures that server is created. – Toner

If you use any method which internally uses the private ApplicationWrapper? _application field, you will get that error, that's why you need to start the TestServer via reflection. If you find a better way please let me know. – Abolition