AspNetCore Integration Testing Multiple WebApplicationFactory Instances?
Asked Answered
G

4

13

Does any one know if it is possible to host multiple instances of WebApplicationFactory<TStartup>() in the same unit test?

I have tried and can't seem to get anywhere with this one issue.

i.e

_client = WebHost<Startup>.GetFactory().CreateClient();
var baseUri = PathString.FromUriComponent(_client.BaseAddress);
_url = baseUri.Value;

_client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(
    "Bearer", "Y2E890F4-E9AE-468D-8294-6164C59B099Y");

WebHost is just a helper class that allows me to build factory and then a client easily in one line.

Under the covers all it does is this:

new WebApplicationFactory<TStartup>() but a few other things too.

It would be nice if i could stand up another instace of a different web server to test server to server functionality.

Does anyone know if this is possible or not?

Gain answered 1/3, 2019 at 11:58 Comment(1)
I think you should change the accepted answer to the right one, which in this case is one with the most upvotes (mine)Inextirpable
I
30

Contrary to what the accepted answer states, it is actually pretty easy to test server to server functionality using two WebApplicationFactory instances:

public class OrderAPIFactory : WebApplicationFactory<Order>
{
    public OrderAPIFactory() { ... }
    protected override void ConfigureWebHost(IWebHostBuilder builder) { ... }
}

public class BasketAPIFactory : WebApplicationFactory<BasketStartup>
{
    public BasketAPIFactory() { ... }
    protected override void ConfigureWebHost(IWebHostBuilder builder) { ... }
}

Then you can instantiate the custom factories as follows:

[Fact] 
public async Task TestName()
{
    var orderFactory = new OrderAPIFactory();
    var basketFactory = new BasketAPIFactory();

    var orderHttpClient = orderFactory.CreateClient();
    var basketHttpClient = basketFactory.CreateClient();

    // you can hit eg an endpoint on either side that triggers server-to-server communication
    var orderResponse = await orderHttpClient.GetAsync("api/orders");
    var basketResponse = await basketHttpClient.GetAsync("api/basket");
}

I also disagree with the accepted answer about it necessarily being bad design: it has its use-cases. My company has a microservices infrastructure which relies on data duplication across microservices and uses an async messaging queue with integration events to ensure data consistency. Needless to say that messaging functionality plays a central role and needs to be tested properly. A test setup as described here is pretty useful in this situation. For example it allows us to thoroughly test how messages are being dealt with by a service that was down at the moment those messages were published:

[Fact] 
public async Task DataConsistencyEvents_DependentServiceIsDown_SynchronisesDataWhenUp()
{
    var orderFactory = new OrderAPIFactory();
    var orderHttpClient = orderFactory.CreateClient();

    // a new order is created which leads to a data consistency event being published,
    // which is to be consumed by the BasketAPI service 
    var order = new Order { ... };
    await orderHttpClient.PostAsync("api/orders", order);

    // we only instantiate the BasketAPI service after the creation of the order
    // to mimic downtime. If all goes well, it will still receive the 
    // message that was delivered to its queue and data consistency is preserved
    var basketFactory = new BasketAPIFactory();
    var basketHttpClient = orderFactory.CreateClient();

    // get the basket with all ordered items included from BasketAPI
    var basketResponse = await basketHttpClient.GetAsync("api/baskets?include=orders");
    // check if the new order is contained in the payload of BasketAPI
    AssertContainsNewOrder(basketResponse, order); 
}
Inextirpable answered 4/11, 2019 at 21:31 Comment(4)
Hmmm, does this scenario include one server calling the other? Like a proxy talking to an app server? I.e such as an api key check?Gain
That depends on what type of communication you want to use. Http requests might not work out of the box with WebApplicationFactory because I think it doesn't configure Kestrel, but looking at this related issue on github you can definitely get that to work too.Inextirpable
I tried what you did here but ConfigureWebHost never gets fired when instantiating the subclass of WebApplicationFactory. This means my server isn't actually built so it just fails fast from there.Airport
@FrankHale I'd have to check out your code to see whats going wrong, but we got it to work in our case, which is very similar! :)Inextirpable
B
7

It is possible to host multiple communicating instances of WebApplicationFactory in single integration test.

Let's say we have master service named WebApplication, which depends on utility service named WebService using named HttpClient with name "WebService".

Here is example of integration test:

[Fact]
public async Task GetWeatherForecast_ShouldReturnSuccessResult()
{
    // Create application factories for master and utility services and corresponding HTTP clients
    var webApplicationFactory = new CustomWebApplicationFactory();
    var webApplicationClient = webApplicationFactory.CreateClient();
    var webServiceFactory = new WebApplicationFactory<Startup>();
    var webServiceClient = webServiceFactory.CreateClient();
    
    // Mock dependency on utility service by replacing named HTTP client
    webApplicationFactory.AddHttpClient(clientName: "WebService", webServiceClient);

    // Perform test request
    var response = await webApplicationClient.GetAsync("weatherForecast");

    // Assert the result
    response.EnsureSuccessStatusCode();
    var forecast = await response.Content.ReadAsAsync<IEnumerable<WeatherForecast>>();
    Assert.Equal(10, forecast.Count());
}

This code requires CustomWebApplicationFactory class to be implemented:

// Extends WebApplicationFactory allowing to replace named HTTP clients
internal sealed class CustomWebApplicationFactory 
    : WebApplicationFactory<WebApplication.Startup>
{
    // Contains replaced named HTTP clients
    private ConcurrentDictionary<string, HttpClient> HttpClients { get; } =
        new ConcurrentDictionary<string, HttpClient>();

    // Add replaced named HTTP client
    public void AddHttpClient(string clientName, HttpClient client)
    {
        if (!HttpClients.TryAdd(clientName, client))
        {
            throw new InvalidOperationException(
                $"HttpClient with name {clientName} is already added");
        }
    }

    // Replaces implementation of standard IHttpClientFactory interface with
    // custom one providing replaced HTTP clients from HttpClients dictionary 
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        base.ConfigureWebHost(builder);
        builder.ConfigureServices(services =>
            services.AddSingleton<IHttpClientFactory>(
                new CustomHttpClientFactory(HttpClients)));
    }
}

And finally, CustomHttpClientFactory class is required:

// Implements IHttpClientFactory by providing named HTTP clients
// directly from specified dictionary
internal class CustomHttpClientFactory : IHttpClientFactory
{
    // Takes dictionary storing named HTTP clients in constructor
    public CustomHttpClientFactory(
        IReadOnlyDictionary<string, HttpClient> httpClients)
    {
        HttpClients = httpClients;
    }

    private IReadOnlyDictionary<string, HttpClient> HttpClients { get; }

    // Provides named HTTP client from dictionary
    public HttpClient CreateClient(string name) =>
        HttpClients.GetValueOrDefault(name)
        ?? throw new InvalidOperationException(
            $"HTTP client is not found for client with name {name}");
}

The complete code of example you may find here: https://github.com/GennadyGS/AspNetCoreIntegrationTesting

The pros of such approach are:

  • ability to test interactions between the services;
  • no need to mock internals of services so that you can consider them as black boxes;
  • tests are stable to any refactorings including changes in communication protocol;
  • tests are fast, self-contained, do not require any prerequisites and give predictable results.

The main cons of such approach is possible conflicting dependencies of participating services (e.g. different major versions of EFCore) in real world scenarios due to the fact that all services using in test are running in single process. There are several mitigations of such problem. One of them is to apply modular approach to services' implementations and load modules in runtime according to configuration file. This may allow to replace configuration file in tests, exclude several modules from loading and replace missing services with simpler mocks. The example of applying such approach you may find in branch "Modular" of the example repository above.

Beghard answered 28/11, 2020 at 14:12 Comment(1)
Fantastic! This is exactly what I needed. More details about how to get the HttpClient in the service under test: In its constructor, add the parameter IHttpClientFactory? _httpClientFactory = null to be injected into the service. The optional parameter ensures that no IHttpClientFactory has to be registered (which is usually the case when the program is started in normal mode), but if one is registered (like here for testing), it can be used: HttpClient httpClient = _httpClientFactory?.CreateClient("WebService") ?? new HttpClient();Courtly
R
0

No. It's not possible. WebApplicationFactory leans on xUnit's IClassFixture, which has to be applied at the class level, meaning you only get one bite at the apple. The WebApplicationFactory itself is capable of being customized per test, which fulfills most use cases where you're need a "different" one, but it doesn't help you wanting two totally separate active test servers at the same time.

However, that said, what you're wanting is a bad test design in the first place. The whole point of testing is to eliminate variables so you can actually ensure the piece of the SUT is actually working. Even in an integration testing environment, you're still just looking at one particular interaction between pieces of your application. Have two test servers, feeding off each other, effectively multiplies the variables giving you no assurance that either side is working correctly.

Rickeyricki answered 1/3, 2019 at 14:35 Comment(9)
Fair enough - i have two webapis one acts like a gatekeeper to the other and sends requests for looks ups and etc before letting a request through .... so in essence i am workig in the same domain and testing one peice of functionality. I see your point but for me i do need two test servers really. I will have to resort to mocking the call out to the other server! Is was a stretch tbhGain
That still doesn't necessitate two servers at once. Any app that depends on your "gatekeeper" service could care less that it's proxying requests to another service. They just need a response and that comes from the gatekeeper. As a result, tests for those only involve the gatekeeper service, not your other service. Then, all you need is tests for the gatekeeper service itself to ensure that it's proxies the requests correctly to the other service. In each case, a single test server will do.Rickeyricki
Hello again, @ChrisPratt. What you're doing here is applying unit testing principles to integration testing. The isolation you're suggesting is precisely what unit testing is for and precisely what integration testing is not for. Integration testing is the intentional testing against the underlying resources, why wouldn't this apply to other services?Micrometer
@jefffischer: Incorrect. Integration testing is about literally testing the integration between your various units. What you're describing is a systems test, and is totally different.Rickeyricki
I see what you're saying, but from what I've seen the connotation of "integration testing" has been replaced with functional testing on pretty much every team I've ever been on. You're saying your teams do not use integration testing to mean functional testing? Although I disagree with the terminology, that's all I've ever seen, @ChrisPratt.Micrometer
Not sure what you're talking about. Functional testing is a QA thing. You're literally testing a piece of functionality end to end like "a user can sign up". You fill the field, click the buttons, ensure the welcome email comes, etc. That is very high level testing. Integration testing is still pretty low level, one step above unit, to be exact. It's stuff like ensuring that when you call the CreateUser method, there's actually a call out to the DAL to write the data to the store. Not actually testing that it can do something like connect to a database, mind you.Rickeyricki
That would be a systems test.Rickeyricki
WebApplicationFactory does not lean on IClassFixture: it has no dependency on xUnit whatsoever, and it can easily be instantiated directly. The only advantage IClassFixture provides is that it'll call Dispose on the factory when the tests are all done.Malena
"...The WebApplicationFactory itself is capable of being customized per test..." I costs 1 second to have the factory instance at test level instead of class level. if you have lots of tests the test level instance would be a waste of time. Thus use the factory instance only on test level when you really need it that way.Zionism
K
0

I was based on Gennadii Saltyshchak's solution to create this, which is exaclty what I was looking for: Two servers communicating with one another via a fallback mechanism.

In this example one server runs on port 80 and the other on 82 and there is an api endpoint called fallback that calls the hello endpoint on the fallback server.

Full solution can be found here: https://github.com/diogonborges/integration-test-communicating-servers

public class Tests
{
    private HttpClient _port80Client;
    private HttpClient _port82Client;

    [SetUp]
    public void Setup()
    {
        // Create application factories for master and utility services and corresponding HTTP clients
        var port80Factory = new CustomWebApplicationFactory(80, 82);
        _port80Client = port80Factory.CreateClient();
        port80Factory.Server.Features.Set<IServerAddressesFeature>(new ServerAddressesFeature {Addresses = {"http://localhost:80"}});

        var port82Factory = new CustomWebApplicationFactory(82, 80);
        _port82Client = port82Factory.CreateClient();
        port82Factory.Server.Features.Set<IServerAddressesFeature>(new ServerAddressesFeature {Addresses = {"http://localhost:82"}});

        // Mock dependency on utility service by replacing named HTTP client
        port80Factory.AddHttpClient(Constants.Fallback, _port82Client);
        port82Factory.AddHttpClient(Constants.Fallback, _port80Client);
    }

    [Test]
    public async Task Port80_says_hello()
    {
        var response = await _port80Client.GetAsync("hello");

        var content = await response.Content.ReadAsStringAsync();
        Assert.AreEqual("hello from http://localhost:80", content);
    }
    
    [Test]
    public async Task Port80_falls_back_to_82()
    {
        var response = await _port80Client.GetAsync("hello/fallback");

        var content = await response.Content.ReadAsStringAsync();
        Assert.AreEqual("hello from http://localhost:82", content);
    }
    
    [Test]
    public async Task Port82_says_hello()
    {
        var response = await _port82Client.GetAsync("hello");

        var content = await response.Content.ReadAsStringAsync();
        Assert.AreEqual("hello from http://localhost:82", content);
    }

    [Test]
    public async Task Port82_falls_back_to_80()
    {
        var response = await _port82Client.GetAsync("hello/fallback");

        var content = await response.Content.ReadAsStringAsync();
        Assert.AreEqual("hello from http://localhost:80", content);
    }
}
Karolynkaron answered 16/2, 2022 at 13:13 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.