Do Webjobs automatically renew leases on Azure Queue messages?

Question

When Webjobs get a message from a queue on Azure Storage via QueueTrigger, it leases the message (makes it invisible). If the triggering function (of webjob) takes a long time to process the message, is this lease automatically extended? Or should I handle that in the function?

On this link Windows Azure Queues: Improved Leases, Progress Tracking, and Scheduling of Future Work, the author states that "A lease on the message can be extended by the worker that did the original dequeue so that it can continue processing the message"

Note: I've tried a webjob (with a QueueTrigger) which waits for 20 minutes.

//Write Log
Thread.Sleep(1200000);
//Write Log

It is completed successfully. And during this time no other webjob instance try to attempt for the same queue item (It did not become visible). Therefore it seems that an auto-renew mechanism for leases exists. Anyhow I am waiting for an answer from a Microsoft employee or with an official link (msdn, azure, ...).

Mezzotint

Answer 1

Yes, your lease is automatically extended. It is 10 minutes each time.

See this answer here [1] by a Microsoft employee referring to the docs and comments on azure.microsoft.com [2].

EDIT (long answer)

In addition, an examination of the source code, starting with the QueueListener class at https://github.com/Azure/azure-webjobs-sdk/blob/cfc875a7f00e595410c0603e6ca65537025490a9/src/Microsoft.Azure.WebJobs.Host/Queues/Listeners/QueueListener.cs indicates the same.

The code in QueueListener has the relevant parts on line 138, where the 10 minute visibilityTimeout variable is defined:

TimeSpan visibilityTimeout = TimeSpan.FromMinutes(10); // long enough to process the job

That variable is then passed along to ProcessMessageAsync, which starts a timer defined in the method CreateUpdateMessageVisibilityTimer with that same value. The 10 minute value is used to determine when the first and next update the visibility timeout also (by halving it and creating an instance of the LinearSpeedupStrategy class).

Eventually, in the class UpdateQueueMessageVisibilityCommand [3], you will find that the UpdateMessageAsync method on the queue is called with that same 10 minute renewal.

The LinearSpeedupStrategy will renew again after 5 minutes, unless the renewal failed in which case it will try again after 1 minute (as defined in QueueListener).

[1] Azure Storage Queue and multiple WebJobs instances: will QueueTrigger set the message lease time on triggered?

[2] https://azure.microsoft.com/en-us/documentation/articles/websites-dotnet-webjobs-sdk-get-started/

[3] https://github.com/Azure/azure-webjobs-sdk/blob/cfc875a7f00e595410c0603e6ca65537025490a9/src/Microsoft.Azure.WebJobs.Host/Queues/Listeners/UpdateQueueMessageVisibilityCommand.cs

Nial

One of the answers in that question belongs to me. Which means I saw that question and obviously the other answer. And I also read the link given and did that example long before. I couldn't find anything in those suggesting "your lease is automatically extended. It is 10 minutes each time". Maybe I miss something. Could you specify the paragraph which you conclude this result? – Mezzotint

@NuriTasdemir I am sorry, I didn't look at names of posters. You're correct, the article does not mention the extension of the time and how long it is. So, I looked at the source code on GitHub (github.com/Azure/azure-webjobs-sdk/blob/master/src/…). The 10 minute window is hard-coded on line 138. That value is then passed along some method calls and used again on line 250 in a timer that will update the message to reset the visibility timeout. – Nial

@NuriTasdemir It's a bit complicated because it will start resetting the visibility timeout halfway through. So, after the first 5 minutes of processing time, it will extend it for 10 minutes, etc. There's the use of UpdateQueueMessageVisibilityCommand via LinearSpeedupStrategy etc. But the bottom line is that the time is extended on your WebJob's behalf for 10 minutes each time. – Nial

Answer 2

You can use method(Java code):

queue.retrieveMessage()

to get a message from a queue on azure storage. It will be visible after 30 seconds by default.

If you want to extend the lease, you can use code below:

CloudQueueMessage updateMessage = queue.retrieveMessage();
EnumSet<MessageUpdateFields> updateFields = EnumSet.of(MessageUpdateFields.CONTENT, MessageUpdateFields.VISIBILITY);
queue.updateMessage(updateMessage, 60, updateFields, null, null);

This means your message will be able to be processed for another 60 seconds.

Beem

I konow that. However, in my case I am getting queue message through QueueTrigger attribute of webjob sdk (c#) as in "public static async Task ProcessQueueMessage ([QueueTrigger("queue_name")] BlobInformation blobInfo)". I guess in this case, that is handled behind the scenes. And I am asking just to be sure. In my trials, webjob process took as long as 90 seconds and the queue message did not become visible in the queue. – Mezzotint

I think you can read article How to use Azure queue storage with the WebJobs SDK. There is a "Polling algorithm" in webjob sdk. It seems your webjob will be continued to process until it reaches the maximum wait time. Can you check you message max wait time? – Beem

@AlexChen-WX The polling algorithm described in that article relates only to how often the WebJob will check for new queue messages. Because each check incurs a storage transaction, you can use this to keep your costs under control. – Nial