Adding SQS redrive policy using AWS CLI command

Question

I am trying to set redrive policy for SQS using the AWS CLI Command below , but seeing an error related to redrive JSON. Can you please let me know how I can fix this?

redrive_policy="{\"RedrivePolicy\":{\"deadLetterTargetArn\":\"$dlq_arn\",\"maxReceiveCount\":\"15\"}}"

AWS CLI COMMAND

aws sqs set-queue-attributes --queue-url https://queue.amazonaws.com/12345678/test-queue --attributes $redrive_policy --region=us-east-1

Error Message

Parameter validation failed: Invalid type for parameter Attributes.RedrivePolicy, value: OrderedDict([(u'deadLetterTargetArn', u'arn:aws:sqs:us-east-1:12345678:dlq'), (u'maxReceiveCount', u'15')]), type: , valid types:

Lozenge

Answer 1

Have you tried just creating the JSON in a separate file and passing it as an argument to your AWS CLI command? I find it's difficult to get all of the escaping correct when passing the JSON as a parameter. So you'd basically do it as the example shows in the AWS documentation:

https://docs.aws.amazon.com/cli/latest/reference/sqs/set-queue-attributes.html#examples

1. So first you'd create a new file called "set-queue-attributes.json" like so:

    {
      "DelaySeconds": "10",
      "MaximumMessageSize": "131072",
      "MessageRetentionPeriod": "259200",
      "ReceiveMessageWaitTimeSeconds": "20",
      "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
      "VisibilityTimeout": "60"
    }

2. Then run the command like this:

aws sqs set-queue-attributes --queue-url https://sqs.us-east-1.amazonaws.com/80398EXAMPLE/MyNewQueue --attributes file://set-queue-attributes.json --region=us-east-1

Duck

Thanks , I just figured that I missed the double quotes for redrive policy. It worked fine after I added that – Lozenge

Any tips on how to remove it afterwards? I tried omitting the RedrivePolicy field from the JSON but no luck so far... – Perrie

@PunterVicky - I am trying to do the exact thing with the Policy attribute. Getting an identical error. What did you fix? – Andresandresen

@Andresandresen it’s been quite a while but I believe I removed the double quotes when assigning the value to redrive_policy variable – Lozenge

I got it figured out. It is the double quotes - need to use single quotes wrapping escaped json. Double quotes remove the escaping and turn it back into proper json. – Andresandresen

Answer 2

if you want to run in the same command you can use this example:

aws sqs set-queue-attributes \
--queue-url https://sqs.us-east-1.amazonaws.com/80398EXAMPLE/MyNewQueue \
--attributes '{
    "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
    "MessageRetentionPeriod": "259200",
    "VisibilityTimeout": "90"
}'

Seven

Answer 3

Three Methods to achieve this:

Note: The solutions also work on any other AWS CLI commands that require a stringified JSON

1. Using the Command-line JSON processor jq (Recommended)

This method is recommended because of many reasons:

• I've found jq a handy tool to use when working with AWS CLI as the need to stringify JSON comes up quite frequently.

Install for Ubuntu: sudo apt install jq

Basic Options:

• jq -R: Returns the stringified JSON
• jq -c: Eliminates spacing and newline characters

The benefit is that you can write JSON as JSON and Pipe the result into the jq -R command.

Method 1:

aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
    --attributes RedrivePolicy=$(echo '{"maxReceiveCount":500,"deadLetterTargetArn":"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue"}' | jq -R)

OR if you have a sqs-redrive-policy.json file:

Method 2:

In sqs-redrive-policy.json,

{
  "maxReceiveCount": 500,
  "deadLetterTargetArn": "arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue"
}

Run in Command Line:

   aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \ 
    --attributes RedrivePolicy=$(cat ~/path/to/file/sqs-redrive-policy.json | jq -c | jq -R)

• As you can see the second benefit is that you can isolately modify only the --redrive-policy without having to touch any of the other attributes.

Common Confusion: A confusion is the name set-queue-attributes (it would be better named put-queue-attributes). as it doesn't overwrite all attributes but only overwrites the attributes mentioned with the command. So if you already set a Policy attribute earlier during create-queue, this will not overwrite the Policy to null. In other words, this is safe to use.

---

2. Using a stringified JSON

This is a pain to be honest, and I avoid this.

aws sqs set-queue-attributes \
--queue-url "https://sqs.us-east-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
--attributes '{
    "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue\",\"maxReceiveCount\":\"500\"}",
}'

3. Use a filePathURL to the JSON file for attributes.json NOT sqs-redrive-policy.json

This is my last preference.

Reason:

• This means setting all the attributes specified in the attributes.json file again at a single go.
• Doesn't escape the pain of writing stringified JSON as text.

In attributes.json,

{
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:IAMEXAMPLE12345678:ExampleDeadLetterQueue\", \"maxReceiveCount\":\"5\"}"
}

Run in command line:

aws sqs set-queue-attributes \
    --queue-url "https://sqs.ap-south-1.amazonaws.com/IAMEXAMPLE12345678/ExampleQueue" \
    --attributes file:///home/yourusername/path/to/file/attributes.json

---

About