How to enable gzip compression in Simple OData Client?

Question

I'm trying to perform a query that retrieves information from a CRM Dynamics with the Simple OData Client library like this (C#):

var settings = new ODataClientSettings(resource + "/api/data/v8.0/");
settings.BeforeRequest = (request) =>
    {
        request.Headers.Clear();
        request.Headers.Add("Authorization", accesstoken.AccessTokenType + " " + accesstoken.AccessToken);
    };

settings.PayloadFormat = ODataPayloadFormat.Json;
var client = new ODataClient(settings);
var annotations = new ODataFeedAnnotations();

var transactions = await client.For("mss_transaccions").FindEntriesAsync(annotations);
while (annotations.NextPageLink != null)
{
    transactions = transactions.Union(await client.For("mss_transaccions").FindEntriesAsync(annotations.NextPageLink, annotations));
}

While this works, it is extremely slow because my query on the mss_transaccions table has 7200 entities. I'm looking at the output in Fiddler and I can see that it is trying to download about 20 MB of information.

I tried to run the same query on Google Chrome, and I could see that by default the obtained response is compressed in gzip format, going from 20MB to some mere 500KB. So I'm deducing that Simple OData Client is not doing any kind of compression, and that's why it is so tremendously slow.

In addition, the request from OData Simple Client asks for metadata information, which adds another 4MB, while Chrome or a simple HttpClient request do not need to make that call.

Is there anything I can do to improve that and enable compression?

Thank you.

Browder

I have little experience with the OData client, but is there a way you can check/set/modify the "Accept-Encoding: gzip, deflate" header on the outgoing request? That should at least have the server kick in and compress it. It would then be hoping that the client uses some stream that decompresses it. – Manners

@MarvinSmit That is why this is not an OData question but a Simple.Odata question - which is a specific client library. Given that the outgoing request is managed by a library, modifying it is not as trivial as it sounds (if the library does not support it). Technially, yes, one "just" has to subclass the HttpClient and make it submit the compression header info.... but how to do that in SImple.Odata? I am in the same boat ;) – Florrieflorry

given your code above, i would expect a "beforesend"->"Headers.Add" of the accept type and give it a spin? – Manners

Answer 1

I've been able to finally enable compression and speed up the overall process. The whole discussion can be found here: https://github.com/object/Simple.OData.Client/issues/238

To lay it down simply and quickly, you just need to modify the message handler in the ODataSettings instance with the following piece of code:

settings.OnApplyClientHandler = handler =>
            {
                handler.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
            };

Now the httpRequests are sent as gzip,deflate and decompressed on response properly.

Browder

Answer 2

In your BeforeRequest action, add the Accept-Encoding header as follows:

settings.BeforeRequest = (request) =>
{
    // ... other headers as above
    request.Headers.Add("Accept-Encoding", "gzip");
};

Thereabout

If I add "Accept-Encoding", "gzip" to request headers I get '', hexadecimal value 0x1F, is an invalid character. Line 1, position 1. error – Neace

Answer 3

If you inspect which headers is sending Chrome and try to replicate them in C#?

Also, If I have to access CRM from C#, I would use the Microsoft.Xrm.Sdk as opposed to OData. You have loads of proxy types and requests that will allow you to write code much more cleaner. OData has other limitations which QueryExpressions / CRM LINQ / FetchXml don't have too.

OData would make more sense for JS code (i.e. requests from a CRM form).

Weigand