Long polling with Httpclient

Question

I am consuming one REST API (GET) using .Net HttpClient. I want to call this API with long polling.

I have few questions:

1. What is the best way to retrieve data using long polling?
2. Here is my use case - my application will consume this api with long polling and based on results I will perform some operation on different thread. Based on new response of long poll get, I will abort/complete old thread and start operation on new thread again. How to achieve this using Tasks?

Apteral

Answer 1

For the first question I've found this solution, works pretty well:

var url = "http://your.url";
using (var client = new HttpClient())
{
    client.Timeout = Timeout.InfiniteTimeSpan;
    var request = new HttpRequestMessage(HttpMethod.Get, url);
    using (var response = await client.SendAsync(
        request, 
        HttpCompletionOption.ResponseHeadersRead))
    {
        using (var body = await response.Content.ReadAsStreamAsync())
        using (var reader = new StreamReader(body))
            while (!reader.EndOfStream)
                Console.WriteLine(reader.ReadLine());
    }
}

Additory

Is this for the client or for the server? How would a server send data back to the client if this code is not for server – Ansate

THANK YOU SO MUCH BRO! This helped me out so much today. :) – Subsellium

@Subsellium no problem. I'm surprised this 5 yo answer is still valid :) – Additory

Me too honestly. I am dealing with a financial API that uses HTTP long polling to stream market data. It's rough. The provider is TradeStation - api.tradestation.com/docs/fundamentals/http-streaming – Subsellium

Answer 2

I am currently doing this. I have a client running as a Windows Service. At startup service will post a serviceID to the webserver for identification. Its basically just a GUID generated at startup if no existing ID was found.

After enrollment the service will then start a long polling http request at a longpolling web server resource. Web service will park the caller for at set amount of time. If no wakeup calls are received from other services the client request is then answered with a reason. Client will then immediately start a new long polling request after processing the response.

Web service also has a WakeUp resource used for other services to signal that a specific serviceID should receive an actual WakeUp. Longpolling and WakeUp functions share a list of serviceIDs to wake up. Depending on your goals you can have multiple wakup reasons depending on the complexity of your solution.

Windows Service is implemented as a multithreaded application. One general controlling thread that handles general Windows Service management stuff and signals worker threads if service receives OnStop events from the Service Control Manager. Its also the thread that sets every other thing up. Instantiates the other worker threads and starts them.

LongPolling thread which only job is to handle the result of the longpolling call. When a response is received it inspects the result and activates other threads based on the result and then starts a new longpolling request. This means it is able to receive new tasks while the other worker threads are processing their tasks.

And as said you can have multiple different worker threads that do different things. Moves files around, installs applications, deletes ilegal applications, makes backups of folders. The sky really is the limit.

This is pretty much the basic design of my system and it works pretty great. Its simple and easy to use.

Jochbed

Do you have your code example somewhere for reference? – Ansate

Not at the current time. Even just with boilerplate code scaffolding there is still a lot that goes on in a project like this. You can start by creating a Server project with a LongPolling Controller with a simple GET function and have a ConsoleApp consume that GET function. Let the function hang for 20 seconds and then return and make the ConsoleApp do another GET request. This is basically the start of a longpolling system. I'll see if I can make a very rudimentary client/server system and post to GitHub or something like that. – Jochbed

Answer 3

var socketsHandler = new SocketsHttpHandler
{
      PooledConnectionIdleTimeout = TimeSpan.FromHours(27),//Actually 5 mins can be idle at maximum. Note that timeouts longer than the TCP timeout may be ignored if no keep-alive TCP message is set at the transport level.
      MaxConnectionsPerServer = 10
};
client = new HttpClient(socketsHandler);

As you can see, although I set the idle timeout to 27 hours, but actually it just keep 5 mins alive.

So, finally I just call the target endpoint using the same HttpClient every 1 min. In this case, there is always an established connection. You could use netstat to check that.

Landry