Deleting message from SQS FIFO queue: The receipt handle has expired

Question

I switched to a FIFO queue and I got this error message when I tried to delete a message from the queue

Value {VALUE} for parameter ReceiptHandle is invalid. Reason: The receipt handle has expired.

It appears that the error happens because I tried to delete the message after visibility timeout has expired. I changed the default visibility timeout 0 to the maximum, 12 hours, this solved partially the issue. Sometimes it could happens that a message still in my queue for longer than 12 hours before I could perform it and than delete it, so I will get the error again. Is there any solution to increase the visibility timeout for more than 12 hours or to bypass this error by another way?

Lolanthe

Answer 1

You can do it in AWS Console, but the trick is, you have to do it while the Polling progress is still active.
For example, when you poll for 10 seconds, and 10 messages, you need to delete the message within 10 seconds or before 10th messages arrived, whichever comes first, after the polling stopped, your window of deletion closed.

You get error when polling stopped

Adjust polling duration, and message count

While polling, select the message and delete

Message deleted successfully.

Dorser

This does not work for me. I still get the error message even if I do the delete within the polling window. – Aras

Same in my case, messages are not getting deleted, I have set it to 1 hour. Message is getting processed in few mins. – Sperrylite

Works. But want to understand why this works? What happens here? – Bennion

Answer 2

TLDR: You want to look into the ChangeMessageVisibility API.

Details

The reason for visibility timeout is to make sure the process handling the message hasn't unexpectedly died, and allow the message to be processed by a different worker.

If your process needs to take longer than the configured visibility timeout, it essentially needs to send some signal to SQS that says "I'm still alive and working on this message". That's what ChangeMessageVisibility is for.

If you have wide variability in the time required to consume and process a message, I suggest setting a small-ish default visibility timeout and having your workers emit a "heartbeat" (using ChangeMessageVisibility) to indicate they're still alive and working on the message. That way you can still recover relatively quickly when a worker legitimately fails.

Note there is also ChangeMessageVisibilityBatch for doing this on batches of messages.

Benniebenning

Please be aware that you cannot extend beyond 12 hours if you have already set the max to be 12 hours. – Valora

Answer 3

Try increasing the value of VisibilityTimeout parameter in sqs.receive_message() for the message you wish to delete using ReceiptHandle

Nimwegen

Thanks man, that was it. I thought VisibilityTimeout was only set on the Queue itself but it turned out the VisibilityTimeout on the boto3 call overrides that and makes the message undeletable. Changing it as you suggested solved the problem – Greene

Answer 4

setting VisibilityTimeout greater than 0 will work

Bodily

Answer 5

change VisibilityTimeout:0 to VisibilityTimeout:60 it's working

const params = {
        AttributeNames:[
            "SentTimestamp"
        ],
        MaxNumberOfMessages:10,
        MessageAttributeNames:[
            "All"
        ],
        QueueUrl:queueURL,
        VisibilityTimeout:60,
        WaitTimeSeconds:0,
    };
    sqs.receiveMessage(params,function (err,data) {
        console.log(data);
        if (err) {
            console.log("Receive Error", err);
        }else if (data.Messages) {
            let deleteParams = {
              QueueUrl: queueURL,
              ReceiptHandle: data.Messages[0].ReceiptHandle
            };
            sqs.deleteMessage(deleteParams, function(err, data) {
                  if (err) {
                    console.log("Delete Error", err);
                  } else {
                    console.log("Message Deleted", data);
                  }
            });
         }
    });

Gastrectomy

Answer 6

Why increasing the visibility timeout works? So basically the visibility timeout is the time during which the message is invisible to others AFTER being read from the queue. So when the timeout is 0, the moment the message is read, the next moment it's reborn to be read again by any producer. So why can't you delete it? Because the original receipt handle is immediately obsolete/invalid that's why you see Receipt handle expired error message.

That's why you should have non-zero visibility timeout to allow sufficient time to process and delete a msg.

Spirit

Now the impact of all this can also happen in Standard Queue, but it's just less noticeable due to lack of strict ordering. Standard Queue aims for "Atleast One" delivery. – Spirit