How to prevent AWS SQS from deleting a message when Lambda function triggered fails to process that message?

Question

I have deployed a AWS Lambda function that triggers when a SQS queue receives a message. The function makes a request to a Rest API and if the response is not Ok the SQS message needs to be processed again.

That's why I need to resend the message to the queue but I would prefer to delete the SQS messages programatically, although I can't find how to configure SQS. I have tried message retention but it seems the trigger event causes the message being deleted anyway.

Other possible options could be back up the message in S3 or persisting it in DynamoDB but I wonder if there's a better option.

Any insights on this question would be very helpful.

Goofy

Answer 1

From AWS Lambda Retry Behavior - AWS Lambda:

If you configure an Amazon SQS queue as an event source, AWS Lambda will poll a batch of records in the queue and invoke your Lambda function. If the invocation fails or times out, every message in the batch will be returned to the queue, and each will be available for processing once the Visibility Timeout period expires. (Visibility timeouts are a period of time during which Amazon Simple Queue Service prevents other consumers from receiving and processing the message).

Once an invocation successfully processes a batch, each message in that batch will be removed from the queue. When a message is not successfully processed, it is either discarded or if you have configured an Amazon SQS Dead Letter Queue, the failure information will be directed there for you to analyze.

So, it seems (from reading this) that a simple option would be set a high visibility timeout on the queue and then raise an error if the function cannot process the message. This message will remain invisible for the configured timeout period, then would reappear on the queue for processing. If it exceeds the permitted number of retries, it would be deleted or moved to a Dead Letter Queue (if configured).

Lectra

I maybe missing something but did you mean to raise an error for single message failure out of the batch? If so, raising an error will fail the whole invocation(let say 10 msg), and then all the 10 messages having the visibility timeout of lets say an hour won't be visible as a group for retry, when originally you want just to make the msg that did the problem invisible for a while... – Subaltern

@Subaltern It is now possible to fail specific messages in a batch. See: AWS Lambda now supports partial batch response for SQS as an event source – Lectra

Thanks will try it out, however how did your previous answer works without this feature? Meaning how the visibility timeout will manage to solve failure in batch process of specific messages? Maybe the batch taking each time another group of messages so we are solving this just by statistics\luck? – Subaltern

I don't think it was previously possible to fail one message in a batch, unless you set the batch size to 1. – Lectra

Answer 2

There is a lambda-powertools library created and maintained by AWSLabs and one of the feature is batch processing.

The batch processing utility handles partial failures when processing batches from Amazon SQS, Amazon Kinesis Data Streams, and Amazon DynamoDB Streams.

Check out the documentation here. This is the python version, but there are versions for other environments.

Smokedry

Answer 3

So after some research I found the following:

1. Frankly there was an workaround options to selectively filter out messages processed as good ones from a batch - before aws implemented it.

   Kindly refer to approaches 1-3 demonstrated in here

2. As for using aws's implementation use approach No.4

Subaltern

@JourneyToJsDude: fyi – Subaltern