How to properly make a http web GET request

Question

I am still new on c# and I'm trying to create an application for this page that will tell me when I get a notification (answered, commented, etc..). But for now I'm just trying to make a simple call to the api which will get the user's data.

I'm using Visual studio express 2012 to build the C# application, where (for now) you enter your user id, so the application will make the request with the user id and show the stats of this user id.

here is the code where I'm trying to make the request:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
//Request library
using System.Net;
using System.IO;

namespace TestApplication
{
    class Connect
    {
        public string id;
        public string type;

        protected string api = "https://api.stackexchange.com/2.2/";
        protected string options = "?order=desc&sort=name&site=stackoverflow";
        
        public string request()
        {
            string totalUrl = this.join(id);

            return this.HttpGet(totalUrl);
        }

        protected string join(string s)
        {
            return api + type + "/" + s + options;
        }

        protected string get(string url)
        {
            try
            {
                string rt;

                WebRequest request = WebRequest.Create(url);
                
                WebResponse response = request.GetResponse();

                Stream dataStream = response.GetResponseStream();

                StreamReader reader = new StreamReader(dataStream);

                rt = reader.ReadToEnd();

                Console.WriteLine(rt);

                reader.Close();
                response.Close();

                return rt;
            }

            catch(Exception ex)
            {
                return "Error: " + ex.Message;
            }
        }
        public string HttpGet(string URI)
        {
            WebClient client = new WebClient();

            // Add a user agent header in case the 
            // requested URI contains a query.

            client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");

            Stream data = client.OpenRead(URI);
            StreamReader reader = new StreamReader(data);
            string s = reader.ReadToEnd();
            data.Close();
            reader.Close();

            return s;
        }
    }
}

the class is an object and its being accessed from the form by just parsing it the user id and make the request.

I have tried many of the examples I have looked on google, but not clue why I am getting on all ways this message "�".

I am new in this kind of algorithm, if anyone can share a book or tutorial that shows how to do this kind of stuff (explaining each step), I would appreciate it

Drawn

see zetcode.com/csharp/httpclient very simple clear examples – Blackshear

Answer 1

Servers sometimes compress their responses to save on bandwidth, when this happens, you need to decompress the response before attempting to read the contents.

As of .NET 6, HttpWebRequest has been deprecated, and was the original example in this answer which I've left intact further down below.

It's highly recommended you use HttpClient moving forward.

Using HttpClient & AutomaticDecompression

public class HttpService
{
    private readonly HttpClient _client;

    public HttpService()
    {
        HttpClientHandler handler = new HttpClientHandler 
        { 
            AutomaticDecompression = DecompressionMethods.All 
        };
        
        _client = new HttpClient();
    }

    public async Task<string> GetAsync(string uri)
    {
        using HttpResponseMessage response = await _client.GetAsync(uri);

        return await response.Content.ReadAsStringAsync();
    }

    public async Task<string> PostAsync(string uri, string data, string contentType)
    {
        using HttpContent content = new StringContent(data, Encoding.UTF8, contentType);
        
        HttpRequestMessage requestMessage = new HttpRequestMessage() 
        { 
            Content = content,
            Method = HttpMethod.Post,
            RequestUri = new Uri(uri)
        };
        
        using HttpResponseMessage response = await _client.SendAsync(requestMessage);

        return await response.Content.ReadAsStringAsync();
    }
}

---

Using HttpWebRequest & AutomaticDecompression

Warning: The HttpWebRequest class used here is deprecated from .NET 6 and onwards. It's highly recommended to use HttpClient instead.

GET

public string Get(string uri)
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
    request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;

    using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
    using(Stream stream = response.GetResponseStream())
    using(StreamReader reader = new StreamReader(stream))
    {
        return reader.ReadToEnd();
    }
}

---

GET async

public async Task<string> GetAsync(string uri)
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
    request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;

    using(HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync())
    using(Stream stream = response.GetResponseStream())
    using(StreamReader reader = new StreamReader(stream))
    {
        return await reader.ReadToEndAsync();
    }
}

---

POST
Contains the parameter method in the event you wish to use other HTTP methods such as PUT, DELETE, ETC

public string Post(string uri, string data, string contentType, string method = "POST")
{
    byte[] dataBytes = Encoding.UTF8.GetBytes(data);

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
    request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
    request.ContentLength = dataBytes.Length;
    request.ContentType = contentType;
    request.Method = method;

    using(Stream requestBody = request.GetRequestStream())
    {
        requestBody.Write(dataBytes, 0, dataBytes.Length);
    }

    using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
    using(Stream stream = response.GetResponseStream())
    using(StreamReader reader = new StreamReader(stream))
    {
        return reader.ReadToEnd();
    }
}

    

---

POST async
Contains the parameter method in the event you wish to use other HTTP methods such as PUT, DELETE, ETC

public async Task<string> PostAsync(string uri, string data, string contentType, string method = "POST")
{
    byte[] dataBytes = Encoding.UTF8.GetBytes(data);

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
    request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
    request.ContentLength = dataBytes.Length;
    request.ContentType = contentType;
    request.Method = method;

    using(Stream requestBody = request.GetRequestStream())
    {
        await requestBody.WriteAsync(dataBytes, 0, dataBytes.Length);
    }

    using(HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync())
    using(Stream stream = response.GetResponseStream())
    using(StreamReader reader = new StreamReader(stream))
    {
        return await reader.ReadToEndAsync();
    }
}

---

Ahwaz

fyi you may want to show an example of how to parse the html string +1 for clean code by the way.. – Duwalt

thank you, i did not know about the decompressing, i'm a php/nodejs developer and this is the first time i start developing on desktop apps. – Drawn

Your welcome, you may want to have a look at 'Newtonsoft.Json' to deserialize the JSON response that you retrieve. – Ahwaz

is there any chance to async version – Cacology

@ahmadmolaie Added them, as well as how to do POST requests – Ahwaz

I want to submit form data, how to call the POST method? ( i mean where i put my post parameter values?) – Isochronism

how to set POST params to data string here? I tried 'string postData = WebUtility.UrlEncode("email") + "=" + WebUtility.UrlEncode(email); postData += "&" + WebUtility.UrlEncode("password") + "=" + WebUtility.UrlEncode(password);' – Crowboot

Your post data looks fine, what are you setting for content type – Ahwaz

hello there! i used your answer and made a wrapper that will also help you in handling the query strings automatically gist.github.com/earlpeterg/d3da31c2cf1d029215d5229c4ff74b74 HttpHelper.Get("https://api.example.com/status", new { userId = "1234" }); will result to api.example.com/status?userId=1234 – Saundrasaunter

Answer 2

Another way is using 'HttpClient' like this:

using System;
using System.Net;
using System.Net.Http;

namespace Test
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Making API Call...");
            using (var client = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate }))
            {
                client.BaseAddress = new Uri("https://api.stackexchange.com/2.2/");
                HttpResponseMessage response = client.GetAsync("answers?order=desc&sort=activity&site=stackoverflow").Result;
                response.EnsureSuccessStatusCode();
                string result = response.Content.ReadAsStringAsync().Result;
                Console.WriteLine("Result: " + result);
            }
            Console.ReadLine();
        }
    }
}

Check HttpClient vs HttpWebRequest from stackoverflow and this from other.

Update June 22, 2020: It's not recommended to use httpclient in a 'using' block as it might cause port exhaustion.

private static HttpClient client = null;
    
ContructorMethod()
{
   if(client == null)
   {
        HttpClientHandler handler = new HttpClientHandler()
        {
            AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
        };        
        client = new HttpClient(handler);
   }
   client.BaseAddress = new Uri("https://api.stackexchange.com/2.2/");
   HttpResponseMessage response = client.GetAsync("answers?order=desc&sort=activity&site=stackoverflow").Result;
   response.EnsureSuccessStatusCode();
   string result = response.Content.ReadAsStringAsync().Result;
            Console.WriteLine("Result: " + result);           
 }

If using .Net Core 2.1+, consider using IHttpClientFactory and injecting like this in your startup code.

 var timeout = Policy.TimeoutAsync<HttpResponseMessage>(
            TimeSpan.FromSeconds(60));

 services.AddHttpClient<XApiClient>().ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler
        {
            AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
        }).AddPolicyHandler(request => timeout);

Minotaur

Thank you! Very useful for me. I just modified a bit by enclosing the response and the content in "using" statement: – Cariotta

Per aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong, never wrap the HttpClient in a using statement. – Baggott

@sfors Never say never. Look at the code. The HttpClient instance is used exactly once for the life of the program and is disposed just before the program exits. That is completely correct and appropriate. – Constance

I'm not sure how you can dispute that article and others on how to properly create an instance of the HttpClient. Using a private static variable, which doesn't get disposed. Because of this, as quoted in that article: (regarding not using dispose)... "But HttpClient is different. Although it implements the IDisposable interface it is actually a shared object. This means that under the covers it is reentrant) and thread safe. Instead of creating a new instance of HttpClient for each execution you should share a single instance of HttpClient for the entire lifetime of the application." – Baggott

I realise my comment is 2 years too late, but Todd wasn't disputing the article. Todd was simply saying that given the full program example, a single HttpClient is used for the life of the application. – Trichinopoly

Answer 3

Simpliest way for my opinion

  var web = new WebClient();
  var url = $"{hostname}/LoadDataSync?systemID={systemId}";
  var responseString = web.DownloadString(url);

OR

 var bytes = web.DownloadData(url);

Aniakudo

Answer 4

var request = (HttpWebRequest)WebRequest.Create("sendrequesturl");
var response = (HttpWebResponse)request.GetResponse();
string responseString;
using (var stream = response.GetResponseStream())
{
    using (var reader = new StreamReader(stream))
    {
        responseString = reader.ReadToEnd();
    }
}

Thibaud

code doesn't dispose objects; could be a memory leak. Needs using statements. – Earley

You cannot assign <null> to an implicitly-typed variable! – Keenan

Its only declare null.i know that.i remove null. – Thibaud

Answer 5

Adding to the responses already given, this is a complete example hitting JSON PlaceHolder site.

using System;
using System.Net.Http;
using System.Threading.Tasks;
using Newtonsoft.Json;

namespace Publish
{
    class Program
    {
        static async Task Main(string[] args)
        {
            
            // Get Reqeust
            HttpClient req = new HttpClient();
            var content = await req.GetAsync("https://jsonplaceholder.typicode.com/users");
            Console.WriteLine(await content.Content.ReadAsStringAsync());

            // Post Request
            Post p = new Post("Some title", "Some body", "1");
            HttpContent payload = new StringContent(JsonConvert.SerializeObject(p));
            content = await req.PostAsync("https://jsonplaceholder.typicode.com/posts", payload);
            Console.WriteLine("--------------------------");
            Console.WriteLine(content.StatusCode);
            Console.WriteLine(await content.Content.ReadAsStringAsync());
        }
    }

    public struct Post {
        public string Title {get; set;}
        public string Body {get;set;}
        public string UserID {get; set;}

        public Post(string Title, string Body, string UserID){
            this.Title = Title;
            this.Body = Body;
            this.UserID = UserID;
        }
    }
}

Isagoge