How to speed up Azure Storage Queue

Question

I have tried everything I can think of to increase the speed of inserts. Which is really only a couple of things with no improvement.

I have chunks of identifiers (Int64) that I need to send to a queue so that my multiple worker roles can work on it without having to worry about concurrency.

I have tried a foreach loop (both with .ToString() and BitConverter.GetBytes()):

foreach(long id in ids) {
    queue.AddMessage(new CloudQueueMessage(id.ToString() /*BitConverter.GetBytes(id)*/));
}

And a Parallel .ForAll<T>():

ids.AsParallel().ForAll(id => queue.AddMessage(new CloudMessage(id.ToString())));

Both from local and a WorkerRole inside the same data center, the inserts max out at 5 per second, and average 4.73 per second.

Am I doing something wrong?

Fiume

How big is the message you're trying to add? – Venavenable

@PeterRitchie 2 bytes or 6 bytes depending on the method (BitConverter vs ToString) – Fiume

Answer 1

Try disabling Nagle on the tcp stack, as this buffers small packets, resulting in upwards of 1/2-second delay shipping your content. Put this in your role start code:

ServicePointManager.UseNagleAlgorithm = false; 

Zug

And this blog post to augment David's answer: blogs.msdn.com/b/windowsazurestorage/archive/2010/06/25/… :) – Bangweulu

You should look at Sandrino di Mattia's answer here: #12750802. He goes deeper into David's response. – Krever

HUGE Difference, 4.73 to 250 (eyeball estimation)/second – Fiume

Nagle shouldn't' slow things down that much... (30 bytes of message payload a second?) – Venavenable

Answer 2

I used a different approach to pump 1500 messages per second from my laptop in a corporate network and hit the 2000 message limit on a VM co-located with the storage account.

I used an async parallel partitioner in combination with some tuning of the default connection limit and partition count.

ServicePointManager.DefaultConnectionLimit = 1000;

public static async Task SendMessagesAsync(CloudQueue queue, IEnumerable<string> messages)
{
    await Task.WhenAll(
            from partition in Partitioner.Create(messages).GetPartitions(500)
            select Task.Run(async delegate
            {
                using (partition)
                    while (partition.MoveNext())
                        await queue.AddMessageAsync(new CloudQueueMessage(partition.Current));
            }));
}

Nagle on or off had no impact on performance.

Reverence