Set SQS policy document with AWS CLI

Question

I am trying to set the policy of an SQS queue to allow SNS topics to send messages.

I have tried following this, this and this

I am using this command

policy=$(getPolicy $queueArn $topicArn)
echo  $policy
aws sqs set-queue-attributes --queue-url $queueUrl --attributes "{\"Policy\":${policy}}"

the getPolicy function builds a policy document

function getPolicy() {
    queueArn=$1
    topicArn=$2

    policy="{\"Version\":\"2012-10-17\",\"Id\":\"policy12345\",\"Statement\":[{\"Sid\":\"stmt12345\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::${account}:root\"},\"Action\":\"SQS:*\",\"Resource\":\"${queueArn}\"},{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"sns.amazonaws.com\"},\"Action\":\"sqs:SendMessage\",\"Resource\":\"${queueArn}\",\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"${topicArn}\"}}}]}"
    echo $policy
}

I am getting a parameter validation failed. Invalid type for parameter Attributes.Policy.

The policy that is being generated is escaped and appears to be a valid policy statement.

This is passing json as the argument to the --attributes parameter.

If I switch it up to use the shorthand syntax -

aws sqs set-queue-attributes --queue-url $queueUrl --attributes Policy=${policy}

I get a different error:

Diffident

Can you perhaps surround the policy single quotes, which would avoid the need to escape the double-quotes? I know that on my Mac with zsh it prefers have parameters wrapped in single quotes. Failing that, can you possible put echo at the front of the set-queue-attributes line and show us the full text of the command that is being sent to AWS, so that we can try to reproduce your situation? – Tsunami

I got it figured out. You were spot on with the single ' vs double " quotes. I had escaped json inside of double quotes so my variables would be expanded. Turns out bash also escapes items inside double quotes but single quotes the string kept. I'll post an answer explaining it. – Diffident

Answer 1

So @John Rotenstein was spot on with the problem.

I am going to post this for anyone else that comes across this problem.

The root problem is single quotes ' vs double quotes " in a bash script - or more specifically my naivety when it comes to their use.

I am trying to dynamically generate a json policy document for my SQS queues to allow the various SNS topics to publish to them. The CLI example uses a json file - but this will not work because I want to specify the ARNs of the SNS / SQS resources while my script is executing.

Attributes parameter want a string with a json object; the properties need to be escaped json.

{
  "DelaySeconds": "10",
  "MaximumMessageSize": "131072",
  "MessageRetentionPeriod": "259200",
  "ReceiveMessageWaitTimeSeconds": "20",
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:80398EXAMPLE:MyDeadLetterQueue\",\"maxReceiveCount\":\"1000\"}",
  "VisibilityTimeout": "60"
}

My problem is I had a bash function returning the "escaped" json wrapped in double quotes. I needed that to expand my variables.

function getPolicy() {
    queueArn=$1
    topicArn=$2

    policy="{\"Version\":\"2012-10-17\",\"Id\":\"policy12345\",\"Statement\":[{\"Sid\":\"stmt12345\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::${account}:root\"},\"Action\":\"SQS:*\",\"Resource\":\"${queueArn}\"},{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"sns.amazonaws.com\"},\"Action\":\"sqs:SendMessage\",\"Resource\":\"${queueArn}\",\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"${topicArn}\"}}}]}"
    echo $policy
}

The problem is double quotes escape certain characters as explained on Difference between single and double quotes in Bash.

So my escaped json was being turned back into a proper json before being sent.

The solution was to use single quotes in my function.

function buildPolicy() {
    queueArn=$1
    topicArn=$2

    policy='{\"Version\":\"2012-10-17\",\"Id\":\"Policy1564523767951\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::'$account':root\"},\"Action\":\"SQS:*\",\"Resource\":\"\"},{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"sns.amazonaws.com\"},\"Action\":\"sqs:SendMessage\",\"Resource\":\"'$queueArn'\",\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"'$topicArn'\"}}}]}'
    echo $policy
}

You have to stop and start the string to allow for your variable expansion.

Then setting the attribute:

function setPolicy() {
    url=$1
    topic=$2

    arn=$(getQueueArn $url)
    policy=$(buildPolicy $arn $topic)
    aws sqs set-queue-attributes --queue-url $url --attributes "{ \"Policy\" : \"${policy}\" }"
    echo "${arn} policy set"
}

You can use the double quotes here to expand the $policy variable - but that will be escaped json.

It is a trivial mistake but honestly I spent a way too long on it and hopefully someone else will benefit from this.

Diffident