What is the best way to load test a SignalR hubs application?

Question

I would like to know some of the different methods that have been used to test a SignalR hubs-based application.

Staves

Answer 1

In short, if using Hubs, using the .Net client will suffice.

In my case, I have a newsfeed hub that dishes out client-specific data based on the user's profile ID. In my test case, I load up a bunch of profile ID's (6000 in this case), invoke a hub method called JoinNewsfeed() along with the client-specific connection ID and profile ID. Every 100ms a new connection is established.

    [TestMethod]
    public void TestJoinNewsfeedHub()
    {
        int successfulConnections = 0;

        // get profile ID's
        memberService = new MemberServiceClient();
        List<int> profileIDs = memberService.GetProfileIDs(6000).ToList<int>();

        HubConnection hubConnection = new HubConnection(HUB_URL);
        IHubProxy newsfeedHub = hubConnection.CreateProxy("NewsfeedHub");


        foreach (int profileID in profileIDs)
        {
            hubConnection.Start().Wait();
            //hubConnection = EstablishHubConnection();
            newsfeedHub.Invoke<string>("JoinNewsfeed", hubConnection.ConnectionId, profileID).ContinueWith(task2 =>
            {
                if (task2.IsFaulted)
                {
                    System.Diagnostics.Debug.Write(String.Format("An error occurred during the method call {0}", task2.Exception.GetBaseException()));
                }
                else
                {
                    successfulConnections++;
                    System.Diagnostics.Debug.Write(String.Format("Successfully called MethodOnServer: {0}", successfulConnections));

                }
            });

            Thread.Sleep(100);

        }

        Assert.IsNotNull(newsfeedHub);
    }

Performance metrics listed here do the trick on the server. To ensure that a client has in fact connected and the process of populating client object on the server has successfully completed, I have a server-side method that invokes a client-side function with the number and list of connected clients derived from the connected client collection.

Staves

Answer 2

@ElHaix From what I have seen in my own tests your method is not creating a new connection, but reusing the existing connection. As you loop over the collection of profileIDs you should see that hubConnection.ConnectionID stays the same. In order to create a new connection you would need to create an instance of HubConnection inside the foreach loop.

        int successfulConnections = 0;
        const int loopId = 10;

        Console.WriteLine("Starting...");
        for (int i = 1; i <= loopId; i++)
        {
            Console.WriteLine("loop " + i);

            var hubConnection = new HubConnection(HUB_URL);
            IHubProxy chatHub = hubConnection.CreateProxy(HUB_NAME);

            Console.WriteLine("Starting connection");
            hubConnection.Start().Wait();
            Console.WriteLine("Connection started: " + hubConnection.ConnectionId);

            chatHub.Invoke("Register", "testroom").ContinueWith(task2 =>
            {
                if (task2.IsFaulted)
                {
                    Console.WriteLine(String.Format("An error occurred during the method call {0}", task2.Exception.GetBaseException()));
                }
                else
                {
                    Console.WriteLine("Connected: " + hubConnection.ConnectionId);
                    successfulConnections++;
                }
            });

            Thread.Sleep(1000);
        }

Creel

Which packages did you add? – Cheremkhovo

Answer 3

Crank can only test PersistenConnections but since you're looking to test the SignalR Hub itself, you can use Tresi. It's a commercial application though. Here are some links on how the performance test is conducted for different types of loads

1. Constant Load
2. Gradually Incrementing Load
3. Burst Load

Performance Counter Settings for HTTPWebRequests such as Created/Sec, Aborted/Sec, Average Lifetime etc. are displayed during the execution of the load test. It also shows other computed metrics such as Throughput/user Shown below is screenshot of the application

Munford

Answer 4

I haven't done much performance testing on SignalR, but the project provides a useful tool - Crank - for generating client load.

The project wiki also has some guidance on useful performance counters and settings

Heartache

From my understanding, Crank isn't for use with hubs. Anything else? – Staves

Answer 5

Make your own script using Gatling tool for virtual user creation(multithreading) and use java signalr client. please specify in comment if you want to know how to attach your custom script written in java or scala to gatling virtual users. Tell me if you want test plan for the signalr performance testing as it is key point in conducting tests.

Shortbread