How Decompress Gzipped Http Get Response in c#

Question

Want to Decompress a Response which is GZipped Getting from an API.Tried the Below Code ,It Always return Like:-

\u001f�\b\0\0\0\0\0\0\0�Y]o........

My code is:

 private string GetResponse(string sData, string sUrl)
 {
      try
      {
           string script = null;
           try
           {
                string urlStr = @"" + sUrl + "?param=" + sData;

                Uri url = new Uri(urlStr, UriKind.Absolute);

                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
                request.Method = "GET";
                request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;

                using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
                using (StreamReader reader = new StreamReader(response.GetResponseStream()))
                {
                     script = reader.ReadToEnd();
                }      
           }
           catch (System.Net.Sockets.SocketException)
           {
                // The remote site is currently down. Try again next time. 
           }
           catch (UriFormatException)
           {
                // Only valid absolute URLs are accepted 
           }

           return script;
      }
      catch (Exception ex)
      {
           throw new Exception(ex.ToString());
      }
 }

I Found the Above Code from many References for Automatic Decompression.But Eventually,it doesn't Work for me.So as to Unzip the zipped Data I tried the Below Function,

 private string DecompressGZIP(string compressedText)
 {
      byte[] gZipBuffer = Convert.FromBase64String(compressedText);
      using (var memoryStream = new MemoryStream())
      {
           int dataLength = BitConverter.ToInt32(gZipBuffer, 0);
           memoryStream.Write(gZipBuffer, 4, gZipBuffer.Length - 4);

           var buffer = new byte[dataLength];

           memoryStream.Position = 0;
           using (var gZipStream = new GZipStream(memoryStream, CompressionMode.Decompress))
           {
                gZipStream.Read(buffer, 0, buffer.Length);
           }

           return Encoding.UTF8.GetString(buffer);
      }
 }

But,it also Failed in the First Line of code itself Because of the Following Exception:

System.FormatException: 'The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters. '

As i am a Beginner,Hope You guys will Guide Me .....Thanks in advance....

Madalena

What? Where did you find that second code block? Why are you reading the response as a string? Why are you base64 decoding the result of that? That's not decompression, which request.AutomaticDecompression = DecompressionMethods.GZip should already do for you anyway. Please read How to Ask and start over, creating a minimal reproducible example. And You Don't Have to Capitalize Random Words in English. – Debauchee

Possible duplicate of Does .NET's HttpWebResponse uncompress automatically GZiped and Deflated responses? – Fannie

Answer 1

Just Change My Function as Follows,Which is Perfectly Working For me:

private JObject PostingToPKFAndDecompress(string sData, string sUrl)
        {
            var jOBj = new JObject();
            try
            {

                try
                {
                    string urlStr = @"" + sUrl + "?param=" + sData;


                    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlStr);
                    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                    Stream resStream = response.GetResponseStream();

                    var t = ReadFully(resStream);
                    var y = Decompress(t);

                    using (var ms = new MemoryStream(y))
                    using (var streamReader = new StreamReader(ms))
                    using (var jsonReader = new JsonTextReader(streamReader))
                    {
                        jOBj = (JObject)JToken.ReadFrom(jsonReader);
                    }


                }
                catch (System.Net.Sockets.SocketException)
                {
                    // The remote site is currently down. Try again next time. 
                }

            }
            catch (Exception ex)
            {
                throw new Exception(ex.ToString());
            }
            return jOBj;
        }

        public static byte[] ReadFully(Stream input)
        {
            byte[] buffer = new byte[16 * 1024];
            using (MemoryStream ms = new MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
        }

        public static byte[] Decompress(byte[] data)
        {
            using (var compressedStream = new MemoryStream(data))
            using (var zipStream = new GZipStream(compressedStream, CompressionMode.Decompress))
            using (var resultStream = new MemoryStream())
            {
                zipStream.CopyTo(resultStream);
                return resultStream.ToArray();
            }
        }

Madalena

A convoluted answer for a convoluted question. The comments on the question point out an even better solution. Here is a clarification: var clientHandler = new HttpClientHandler() { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate }; var client = new HttpClient(clientHandler); will take care of decoding the gzipped stream – Undecided

Yep - in theory what you say SHOULD work. But it doesn't ALWAYS especially when dealing with Http 301, 302, et. al. ad nauseam re-directs. Then - for some obscure reason - the decompression is 'missed' and you have to end up doing it yourself! – Decuple

Found the response was encoded with Brotoli (?? whatever the eff that is ??) Chimps to the fore! – Decuple

@Undecided You should add your comment as an answer, as it leads to much more concise code. – Cordless

@BerendEngelbrecht I did, feel free to edit if you can make it even better. – Undecided

Answer 2

This is the essential bit which will take care of decoding the gzipped stream:

var clientHandler = new HttpClientHandler() { AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate }; 
var client = new HttpClient(clientHandler); 

Undecided

You are awesome!! – Unclear

Does not work under blazor you will get the error System.PlatformNotSupportedException: Operation is not supported on this platform. – Fedora

Answer 3

Here's my solution for Gzip, Deflate, Brotli or All using AutomaticDecompression.

NOTE: You don't have to send a header necessarily.

string url = "https://my.example.com";

using HttpClientHandler handler = new HttpClientHandler();

/// Set the AutomaticDecompression property to DecompressionMethods.GZip
handler.AutomaticDecompression = DecompressionMethods.GZip;
// handler.AutomaticDecompression = DecompressionMethods.Deflate;
// handler.AutomaticDecompression = DecompressionMethods.Brotli;
// handler.AutomaticDecompression = DecompressionMethods.All;

/// Create an instance of HttpClient with the handler
using HttpClient client = new HttpClient(handler);

/// Send a GET request to a URL that returns compressed content
using HttpResponseMessage response = await client.GetAsync(url);

/// Check if the response is successful
if (response.IsSuccessStatusCode)
{
    /// Read the response content as a string
    /// The content will be automatically decompressed by the handler
    string content = await response.Content.ReadAsStringAsync();

    Console.WriteLine(content);

    HtmlDocument htmlDocument = new();
    htmlDocument.LoadHtml(content);

    List<HtmlNode> allRows = htmlDocument.DocumentNode.Descendants("div").ToList();
    Console.WriteLine(allRows[0]);
}
else
{
    /// Handle the error
    _logger.LogError($"Error accessing the URL. Is TSETMC server down? Status: {response.StatusCode}");
}

Peadar