I am developing an integration solution that accesses a rate limited API. I am performing a variety of CRUD operations on the API using multiple HTTP verbs on different endpoints (on the same server though). I have been pointed towards Polly multiple times, but I haven't managed to come up with a solution that actually works.
This is what I have in my startup:
builder.Services
.AddHttpClient("APIClient", client =>
{
client.BaseAddress = new Uri(C.Configuration.GetValue<string>("APIBaseAddress"));
})
.AddTransientHttpErrorPolicy(builder =>
builder.WaitAndRetryAsync(new []
{
TimeSpan.FromSeconds(1),
TimeSpan.FromSeconds(5),
TimeSpan.FromSeconds(15),
}));
This is just resilience to retry in case of failure. I have a RateLimit policy in a singleton ApiWrapper class:
public sealed class ApiWrapper
{
private static readonly Lazy<ApiWrapper> lazy = new Lazy<ApiWrapper>(() => new ApiWrapper());
public static ApiWrapper Instance { get { return lazy.Value; } }
private IHttpClientFactory _httpClientFactory;
public readonly AsyncRateLimitPolicy RateLimit = Policy.RateLimitAsync(150, TimeSpan.FromSeconds(10), 50); // 150 actions within 10 sec, 50 burst
private ApiWrapper()
{
}
public void SetFactory(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
public HttpClient GetApiClient()
{
return _httpClientFactory.CreateClient("APIClient");
}
}
That policy is used in multiple other classes like this:
public class ApiConsumer
{
private HttpClient _httpClient = ApiWrapper.Instance.GetApiClient();
public async Task<bool> DoSomethingWithA(List<int> customerIDs)
{
foreach (int id in customerIDs)
{
HttpResponseMessage httpResponse = await ApiWrapper.Instance.RateLimit.ExecuteAsync(() => _httpClient.GetAsync($"http://some.endpoint"));
}
}
}
My expectation was that the rate limiter would not fire more requests than configured, but that does not seem to be true. From my understanding the way it works is that the rate limiter just throws an exception if there are more calls than the limit that has been configured. That's where I thought the Retry policy would come into play, so just try again after 5 or 15 seconds if it did not go through the limiter.
Then I played around a bit with Polly's Bulkhead policy, but as far as I can see that is meant to limit the amount of parallel executions.
I have multiple threads that may use different HttpClients (all created by the Factory like in the example above) with different methods and endpoints, but all use the same policies. Some threads run in parallel, some sequentially as I have to wait for their response before sending the next requests.
Any suggestions on how this can or should be achieved with Polly? (Or any other extension if there is good reason to)
Parallel.ForEachAsyncetc. – Tare