Getting the Response of a Asynchronous HttpWebRequest

Question

Im wondering if theres an easy way to get the response of an async httpwebrequest.

I have already seen this question here but all im trying to do is return the response (which is usually json or xml) in the form of a string to another method where i can then parse it/ deal with it accordingly.

Heres some code:

I have these two static methods here which i think are thread safe as all the params are passed in and there are no shared local variables that the methods use?

public static void MakeAsyncRequest(string url, string contentType)
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.ContentType = contentType;
    request.Method = WebRequestMethods.Http.Get;
    request.Timeout = 20000;
    request.Proxy = null;

    request.BeginGetResponse(new AsyncCallback(ReadCallback), request);
}

private static void ReadCallback(IAsyncResult asyncResult)
{
    HttpWebRequest request = (HttpWebRequest)asyncResult.AsyncState;
    try
    {
        using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asyncResult))
        {
            Stream responseStream = response.GetResponseStream();
            using (StreamReader sr = new StreamReader(responseStream))
            {
                //Need to return this response 
                string strContent = sr.ReadToEnd();
            }
       }
       manualResetEvent.Set();
    }
    catch (Exception ex)
    {
        throw ex;
   }
}

Quadrilateral

The code you posted worked fine once I removed the extraneous manualResetEvent.Set(); - what's the problem you're having? – Hereabouts

@JamesManning Hi yeh that was a typo, I am after an easier way to get the result. What you have provided (Task<T>) is exactly along the lines. I've only just made the jump from synchronous requests, seems like theres a lot more going on. Thanks – Quadrilateral

Answer 1

Assuming the problem is that you're having a hard time getting to the returned content, the easiest path would likely be using async/await if you can use it. Even better would be to switch to HttpClient if you're using .NET 4.5 since it's 'natively' async.

Using .NET 4 and C# 4, you can still use Task to wrap these and make it a bit easier to access the eventual result. For instance, one option would be the below. Note that it has the Main method blocking until the content string is available, but in a 'real' scenario you'd likely pass the task to something else or string another ContinueWith off of it or whatever.

void Main()
{
    var task = MakeAsyncRequest("http://www.google.com", "text/html");
    Console.WriteLine ("Got response of {0}", task.Result);
}

// Define other methods and classes here
public static Task<string> MakeAsyncRequest(string url, string contentType)
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.ContentType = contentType;
    request.Method = WebRequestMethods.Http.Get;
    request.Timeout = 20000;
    request.Proxy = null;

    Task<WebResponse> task = Task.Factory.FromAsync(
        request.BeginGetResponse,
        asyncResult => request.EndGetResponse(asyncResult),
        (object)null);

    return task.ContinueWith(t => ReadStreamFromResponse(t.Result));
}

private static string ReadStreamFromResponse(WebResponse response)
{
    using (Stream responseStream = response.GetResponseStream())
    using (StreamReader sr = new StreamReader(responseStream))
    {
        //Need to return this response 
        string strContent = sr.ReadToEnd();
        return strContent;
    }
}

Hereabouts

A quick question, when accessing the MakeAsyncRequest() methods Result property how can i make sure that the result is always finished by the time i try use response. – Quadrilateral

The Result property will block until it's done if it isn't already. – Hereabouts

Ok great, so the requests will happen asynchronously and just the reading of the stream will be blocked. – Quadrilateral

@geepie - correct, whichever thread access the Result property (before it's ready) will block until the task either completes (and the result will return) or fails (and it'll throw the exception) – Hereabouts

request.Timeout = 20000; is useless in asyn get request – Ravine

This is not working for me, task is not waiting for get the response, its just running, it should wait unless it gets the response, i can see the response status "WaitingToActivation". – Hawkie

Answer 2

"Even better would be to switch to HttpClient if you're using .Net 4.5 since it's 'natively' async." - absolutely right answer by James Manning.

This question was asked about 2 years ago. Now we have .Net Framework 4.5, which provides powerful asynchronous methods. Use HttpClient. Consider the following code:

 async Task<string> HttpGetAsync(string URI)
    {
        try
        {
            HttpClient hc = new HttpClient();
            Task<Stream> result = hc.GetStreamAsync(URI);

            Stream vs = await result;
            StreamReader am = new StreamReader(vs);

            return await am.ReadToEndAsync();
        }
        catch (WebException ex)
        {
            switch (ex.Status)
            {
                case WebExceptionStatus.NameResolutionFailure:
                    MessageBox.Show("domain_not_found", "ERROR",
                    MessageBoxButtons.OK, MessageBoxIcon.Error);
                break;
                    //Catch other exceptions here
            }
        }
    }

To use HttpGetAsync(), make a new method that is "async" too. async is required, because we need to use "await" in GetWebPage() method:

async void GetWebPage(string URI)
        {
            string html = await HttpGetAsync(URI);
            //Do other operations with html code
        }

Now if you want to get web-page HTML source code asynchronously, just call GetWebPage("web-address..."). Even Stream reading is asynchronous.

NOTE: to use HttpClient .Net Framework 4.5 is required. Also you need to add System.Net.Http reference in your project and add also "using System.Net.Http" for easy access.

For further reading how this approach works, visit: http://msdn.microsoft.com/en-us/library/hh191443(v=vs.110).aspx

Use of Async: Async in 4.5: Worth the Await

Eyeopener

Answer 3

Once you go async, you can never go back. From there you only really have access to the async's callback. you can ramp up the complexity of this and do some threading & waithandles but that can be rather a painful endeavor.

Technically, you can also sleep the thread when you need to wait for the results, but I don't recommend that, you may as well do a normal http request at that point.

In C# 5 theyhave async/await commands that will make it easier to get the results of the async call to the main thread.

Marnie

the threading is what i have got to get my head around. Thanks for your input. – Quadrilateral

Answer 4

public static async Task<byte[]> GetBytesAsync(string url) {
    var request = (HttpWebRequest)WebRequest.Create(url);
    using (var response = await request.GetResponseAsync())
    using (var content = new MemoryStream())
    using (var responseStream = response.GetResponseStream()) {
        await responseStream.CopyToAsync(content);
        return content.ToArray();
    }
}

public static async Task<string> GetStringAsync(string url) {
    var bytes = await GetBytesAsync(url);
    return Encoding.UTF8.GetString(bytes, 0, bytes.Length);
}

Tapestry