Azure Service Bus queue message handling

Question

So I have an azure function acting as a queue trigger that calls an internally hosted API.

There doesn't seem to be a definitive answer online on how to handle a message that could not be processed due to issues other than being poisonous.

An example:

My message is received and the function attempts to call the API. The message payload is correct and could be handled however the API/service is down for whatever reason (this time will likely be upwards of 10 minutes). Currently what happens is the message delivery count is reaching its max(10) and then getting pushed to the dead letter queue, which in turn happens for each message after.

I need a way to either not increment the delivery count or reset it upon reaching max. Alternatively I could abandon the peek lock on the message without increment the delivery count as I want to stop processing any message on the queue until the API/service is back up and running. This way I would ensure that all messages that can be processed will be and will not fall on the dead letter because of connection issues between services.

Any ideas on how to achieve this?

Mecklenburg

Have you solved this issue, do you need further assistance? – Transarctic

I apologise for not replying. I think in case of service/api downtime we are going to delay the trigger retrying by sleeping the thread and upping the delivery count. This way if it is down it was not instantly reach max delivery count and fall in dead letter. Thanks for your help – Mecklenburg

Answer 1

Currently what happens is the message delivery count is reaching its max(10) and then getting pushed to the dead letter queue, which in turn happens for each message after.

As this document states about Exceeding MaxDeliveryCount:

Queues and subscriptions have a QueueDescription.MaxDeliveryCount/SubscriptionDescription.MaxDeliveryCount setting; the default value is 10. Whenever a message has been delivered under a lock (ReceiveMode.PeekLock), but has been either explicitly abandoned or the lock has expired, the message's BrokeredMessage.DeliveryCount is incremented. When the DeliveryCount exceeds the MaxDeliveryCount, the message gets moved to the DLQ specifying the ``MaxDeliveryCountExceeded``` reason code.

This behavior cannot be turned off, but the MaxDeliveryCount can set to a very large number.

According to your requirement, I assumed that you could follow the approaches below to achieve your purpose:

• For receiving messages under ReceiveMode.PeekLock

  You could specify the Maximum Delivery Count between 1 and 2147483647 under the "SETTINGS > Properties" of your service bus queue on Azure Portal.

• For receiving messages under ReceiveMode.ReceiveAndDelete

You could try-catch the exception when your API/service is down, then you could re-send the message to your queue.

Transarctic

I have thought about catching the exception and pushing message to the back of the queue. However this raises another problem that the messages will be out of sync and the data being pushed to the api out of date. – Mecklenburg

I think your concern is necessary, but the service bus would not delivery your messages to DLQ, and at this point, the main issue relates to the availability of your API/service. You need to monitor the health of your API/service. – Transarctic