How to send big data via SignalR in .NET client

Question

We have a .NET client, which use SignalR to call Server method, but the parameter seems very big, for such scenario how to fix it?

Client code:

public async Task FooAsync()
{
    var hubConnection = new HubConnection(...);
    await hubConnection.Start();

    var hubProxy = hubConnection.CreateHubProcx("ValueHub");
    //the content is very long, about 11776065 bytes (11MB)
    var content = File.ReadAllText(...);
    hubProxy.Invoke("Send", content);
    ...
}

Server code:

[HubName("ValueHub")]
public class ValueHub : Hub
{
    public void Send(string json)
    {

    }
}

From the exception stack and source code, I found the SignalR internally use HttpClient with the FormUrlEncodedContent type HttpContent, and maybe the limitation came from here.

System.UriFormatException was unhandled
  HResult=-2146233033
  Message=Invalid URI: The Uri string is too long.
  Source=System
  StackTrace:
       at System.UriHelper.EscapeString(String input, Int32 start, Int32 end, Char[] dest, Int32& destPos, Boolean isUriString, Char force1, Char force2, Char rsvd)
       at System.Uri.EscapeDataString(String stringToEscape)
       at System.Net.Http.FormUrlEncodedContent.Encode(String data)
       at System.Net.Http.FormUrlEncodedContent.GetContentByteArray(IEnumerable`1 nameValueCollection)
       at System.Net.Http.FormUrlEncodedContent..ctor(IEnumerable`1 nameValueCollection)
       at Microsoft.AspNet.SignalR.Client.Http.DefaultHttpClient.Post(String url, Action`1 prepareRequest, IDictionary`2 postData, Boolean isLongRunning)
       at Microsoft.AspNet.SignalR.Client.Transports.HttpBasedTransport.Send(IConnection connection, String data, String connectionData)
       at Microsoft.AspNet.SignalR.Client.Transports.AutoTransport.Send(IConnection connection, String data, String connectionData)
       at Microsoft.AspNet.SignalR.Client.Connection.Send(String data)
       at Microsoft.AspNet.SignalR.Client.Hubs.HubProxy.Invoke[T](String method, Object[] args)
       at Microsoft.AspNet.SignalR.Client.Hubs.HubProxy.Invoke(String method, Object[] args)

Any good suggestions over this problem?

Losel

Honestly I would not do it in a hub. Use the hub as a messaging framework to inform the client that new data has arrived. The client then can fetch the data via web api / webmethod or any other transportation technique. – Candicecandid

This might help you #7044066 – Urinal

What is the use case? – Tauromachy

#13426514 – Mudcat

Answer 1

As you have already gathered - this data is too much for SIGNALR by it's own design.

Would it not be a better idea to rather have another process that does this with a normal REST API (GET/POST). Perhaps a message indicating to the user that this needs to be done, as this feels very 'BATCH' like.

Secondly, if it a requirement (possible wrong tool for the job), have you considered compression.

Boutonniere

What about smaller files? I want to send 60KB thumbnails. I can tell my server to accept such a long query string(and it will be quite fast anyway), but SignalR's uri encoding will still block it :S. – Tic

Answer 2

You can add a line that makes the message size 'infinite 'in your Startup.cs by setting the MaxIncomingWebSocketMessageSize to null:

    public class Startup
    {
        public void Configuration(IAppBuilder app)
        {
        app.MapSignalR();
        GlobalHost.Configuration.MaxIncomingWebSocketMessageSize = null;
        }
    }
} 

Mine works with ~200kb of data, 10 messages send consistently. I don't know how well it works if there is more data send per second though.

Subjunction

Didn't wonk for me. May because it's an owin app with Web Api and Signalr and the configuration has to be set up differently – Stereoscopic

In case SignalR is inside an Owin App, the above answer has to be combined with this jerodkrone.com/… in order to work. So if the hub is using a custom configuration like var hubconfig = new HubConfiguration(); and a custom resolver like hubconfig.Resolver = new AutofacDependencyResolver(lifetimeScope); then the following 2 lines need to be added in StartUp: GlobalHost.DependencyResolver = hubconfig.Resolver; GlobalHost.Configuration.MaxIncomingWebSocketMessageSize = null; – Stereoscopic

For dot net core use - - - - - services.AddSignalR(configure => { configure.MaximumReceiveMessageSize = null; }); – Rabah

That is not recommended by Microsoft: "Client messages: For long polling or server side events, the client can't send messages larger than 1 MB." learn.microsoft.com/en-us/azure/azure-signalr/… – Secondguess

Answer 3

its easy to do it..

• split files into byte Array chunks (my tests shows max 10kb per chunk is enough)

• send chunks to client with an invoke like (calculate totalBytes and the chunk data you have) :

  hubProxy.Invoke("Send", chunk,currentBytes,totalBytes);
  

• get the chunks from client and create a byte array and append each chunk, signalr sends files syncronously, this means data will be received as your send order
• you have totalbytes and currentBytes data, now you know all data received, save this byte array to a file with stream or whatever you like..

Ri

You would need to send sequence information along with it as I read that SignalR does not guarantee the order of messages. – Borak

probably sending the stream position number may define the sequence information for each chunk – Rehabilitate

Answer 4

add this to web config
<httpRuntime maxRequestLength="1048576" executionTimeout="600" />

then add this code to Startup.cs

public class Startup
{
    public void Configuration(IAppBuilder app)
    {
        app.MapSignalR();
        GlobalHost.Configuration.MaxIncomingWebSocketMessageSize = null;
    }
}

Lankford

"Client messages: For long polling or server side events, the client can't send messages larger than 1 MB." learn.microsoft.com/en-us/azure/azure-signalr/… – Secondguess

Answer 5

Since aspnetcore3.1+ we have SignalR streaming. If you're reading a file to a large string, like your example, I would use the channel.

NOTE: If you use MessagePack you can compress your SignalR connections and send less data on the wire.

Contrary to the old accepted answer, SignalR is now capable of sending large data streams efficiently.

Client code:

public async Task FooAsync()
{
    var hubConnection = new HubConnection(...);
    await hubConnection.Start();

    var hubProxy = hubConnection.CreateHubProcx("ValueHub");
    var channel = Channel.CreateBounded<string>(1);
    await hubProxy.Invoke("Send", channel.Reader);

    var content = File.ReadAllText(...);
    await channel.Writer.WriteAsync(content);
    channel.Writer.Complete();
    ...
}

Server code:

[HubName("ValueHub")]
public class ValueHub : Hub
{
    public void Send(ChannelReader<string> stream)
    {
        while (await stream.WaitToReadAsync())
        {
            while (stream.TryRead(out string item))
            {
                // do something with the stream item
               
            }
        }
    }

Secondguess