QueueTrigger Attribute Visibility Timeout

Question

If I were to get a message from queue using Azure.Storage.Queue

queue.GetMessage(TimeSpan.FromMinutes(20));

I can set the visibility timeout, however when trying to use Azure.WebJobs (SDK 0.4.0-beta) attributes to auto bind a webjob to a queue

i.e.

public static void ProcessQueueMessage([QueueTrigger("myqueue")] string message){
       //do something with queue item
}

Is there a way to set the visibility timeout on the attribute? There does not seem to be an option in JobHostConfiguration().Queues. If there is no way to override, is it the standard 30 seconds then?

Ferruginous

Answer 1

In the latest v1.1.0 release, you can now control the visibility timeout by registering your own custom QueueProcessor instances via JobHostConfiguration.Queues.QueueProcessorFactory. This allows you to control advanced message processing behavior globally or per queue/function.

For example, to set the visibility for failed messages, you can override ReleaseMessageAsync as follows:

protected override async Task ReleaseMessageAsync(CloudQueueMessage message, FunctionResult result, TimeSpan visibilityTimeout, CancellationToken cancellationToken)
{
    // demonstrates how visibility timeout for failed messages can be customized
    // the logic here could implement exponential backoff, etc.
    visibilityTimeout = TimeSpan.FromSeconds(message.DequeueCount);

    await base.ReleaseMessageAsync(message, result, visibilityTimeout, cancellationToken);
}

More details can be found in the release notes here.

Rabaul

the parameter value for visibilityTimeout always defaults as 0. I've tried setting it via _queue.UpdateMessage(message, new TimeSpan(0, 2, 0), MessageUpdateFields.Visibility); but that doesn't seem to effect the value. any idea how that parameter gets set in the first place or how to effect it? – Bowrah

In the 1.1.2 version of the SDK the visibilityTimeout is hardcoded to 10 min, but after 5 min the lease is extended: github.com/Azure/azure-webjobs-sdk/blob/master/src/… – Coaxial

@john I just verified myself in a brand new 1.1.2 SDK app that the timeout when overridden as you've done above is applied. I also used 1 minute. Before I added the custom queue processor override, the message was retried 5 times in quick succession. With the override, it was retried once every minute. So rest assured that it works. I recommend you log a bug in our repo github.com/Azure/azure-webjobs-sdk/issues with complete code and repro steps so we can figure out where you might be going wrong. – Rabaul

@Rabaul I've removed my previous comments after finding out that somebody had accidentally removed the assignment of the custom factory during a bad git merge. Sorry to waste your time. – Linen

Answer 2

I have the same question and haven't found answer yet. But, to answer a part of your question, the default lease is 10 minutes.

Quoting the Azure Website: "When the method completes, the queue message is deleted. If the method fails before completing, the queue message is not deleted; after a 10-minute lease expires, the message is released to be picked up again and processed. This sequence won't be repeated indefinitely if a message always causes an exception. After 5 unsuccessful attempts to process a message, the message is moved to a queue named {queuename}-poison. The maximum number of attempts is configurable."

Link: http://azure.microsoft.com/en-us/documentation/articles/websites-dotnet-webjobs-sdk-get-started/ Section: ContosoAdsWebJob - Functions.cs - GenerateThumbnail method

Hope this helps!

Imine

thanks for seeing that 10 minute limit - I missed that. If I don't get any other response about whether the visibility timeout can be overridden then I'll take it as it can't be done and mark your answer as the correct answer at this points. Thanks – Ferruginous