how to implement exponential backoff in amazon sqs without writing code

Question

I have a simple task that requires a 3rd party. When a request comes, I push it to an amazon sqs queue, pull it in a worker and call the 3rd party. In case of a time out, I want to implement an exponential backoff (try again in 2 secs, then 4 then 8, then...) with a max retry.

Using python, boto -> sqs

I've looked for built in parameters to allow me to do so with as little code as possible (ideally, no code at all).

Something like

from boto import sqs

def handle_message(message):
    try:
      # send a post to api
    except TimeOut, err:
      # send_back_to_itself in 2/4/8 sec
      if delay < DELAY_LIMIT:
          queue.write(message, delay=secs)

Darrendarrey

You're asking how to implement backoff/retry for the HTTP POST to the 3rd-party API, right? (not via boto3 to SQS) There are NPM modules that do this, for example: github.com/FGRibreau/node-request-retry – Longlongan

You would need to delete the message before writing it back to the queue, which is risky as it's not an atomic update. – Paracelsus

Answer 1

According to the AWS SDK docs, you get exponential backoff for free if you're using one of the official SDK libraries. So it seems all you need to do is set your max_attempts count, and everything after the first attempt will back off exponentially:

import boto3
from botocore.config import Config

config = Config(
    retries = dict(
        max_attempts = 10   # docs say default is 5
    )
)

sqs = boto3.client('sqs', config=config)
sqs.receive_message(QueueUrl='https://sqs.us-east-1.amazonaws.com/<your-account-num>/foobar')

Goldfilled

OP is asking for waiting for an exponentially increasing amount of time before reprocessing the SQS message, not for exponentially waiting against responses from the SQS API. – Paracelsus

Any code examples to configure this in spring boot java? – Bartle

Answer 2

I don't think "no code at all" is possible as of summer 2021. But there is a great blog post on how to do this with coding examples https://aws.amazon.com/blogs/compute/using-amazon-sqs-dead-letter-queues-to-replay-messages/

Code sample from the link above:

def handler(event, context):
    """Lambda function handler."""
    for record in event['Records']:
        nbReplay = 0
        # number of replay
        if 'sqs-dlq-replay-nb' in record['messageAttributes']:
            nbReplay = int(record['messageAttributes']['sqs-dlq-replay-nb']["stringValue"])

        nbReplay += 1
        if nbReplay > config.MAX_ATTEMPS:
            raise MaxAttempsError(replay=nbReplay, max=config.MAX_ATTEMPS)

        # SQS attributes
        attributes = record['messageAttributes']
        attributes.update({'sqs-dlq-replay-nb': {'StringValue': str(nbReplay), 'DataType': 'Number'}})

        _sqs_attributes_cleaner(attributes)

        # Backoff
        b = backoff.ExpoBackoffFullJitter(base=config.BACKOFF_RATE, cap=config.MESSAGE_RETENTION_PERIOD)
        delaySeconds = b.backoff(n=int(nbReplay))

        # SQS
        SQS.send_message(
            QueueUrl=config.SQS_MAIN_URL,
            MessageBody=record['body'],
            DelaySeconds=int(delaySeconds),
            MessageAttributes=record['messageAttributes']
        )

Acidophil

An answer should provide some additional insight, not only a link. In my opinion "I think impossible." is not a contribution. That leaves this post a link-only answer, which risks being flagged as "not an answer" and being deleted. – Lot