aws cli command to subscribe to a topic with filters

Question

I'm trying to write a cross account aws cli command to subscribe to a topic and create a filter for that subscription at the same time. Below is how my command looks like.

aws sns subscribe --topic-arn arn:aws:sns:region:accountId:my_topic --protocol sqs --notification-endpoint arn:aws:sqs:region:differentAccountId:my_sqs_queue --attributes "{'RawMessageDelivery': 'true', 'FilterPolicy': '{\"filter\": [\"value1\", \"value2\"]}'}"

I'm getting below error when I run this.

Unknown options: --attributes, [\value1\,, \value2\]}'}, {'RawMessageDelivery': 'true', 'FilterPolicy': '{" filter\:

I've access to admin access both the aws accounts. Any suggestions on what I'm doing wrong?

EDIT: I'm running this in VS Code powershell terminal in windows.

Archive

The problem appears to be related to how the AWS CLI is interpreting the backslashes in the command. Are running that command on Windows or Linux? Instead of using JSON formatting, you could try using the "shorthand syntax" of KeyName1=string,KeyName2=string. – Dyslalia

@JohnRotenstein: Thank you for your time to answer. I'm using powershell in Windows to run this command. I updated the command to use shorthand syntax like --attributes "RawMessageDelivery=true" and still getting the same error. I tried --attributes "RawMessageDelivery=\"true\"" and --attributes "RawMessageDelivery='true'" but it threw a similar error Unknown options: --attributes, RawMessageDelivery=true – Archive

Answer 1

There's probably an easier way to do it (eg using --cli-input-json and providing JSON in a file), but I got this working:

aws sns subscribe \
  --topic-arn arn:aws:sns:region:accountId:my_topic \
  --protocol sqs \
  --notification-endpoint arn:aws:sqs:region:differentAccountId:my_sqs_queue \
  --attributes '{\"RawMessageDelivery\": \"true\", \"FilterPolicy\": \"{\\\"filter\\\": [\\\"value1\\\", \\\"value2\\\"]}\"}'

The problem was the JSON included in a string, which needed \" to be escaped as \\\".

Dyslalia

Thanks for this. The documentation is woeful. I didn't need quite so many escapes in mine... --attributes '{"FilterPolicy":"{\"filter\":[\"value1\",\"value2\"]}"}' – Kanchenjunga

I still get the error even with --attributes '{\"RawMessageDelivery\": \"true\"}' – Compony

Whatever phil has mentioned is right, I have used below syntax --attributes '{ "RawMessageDelivery" : "true", "FilterPolicyScope" : "MessageBody", "FilterPolicy" : " {\"name\": [{ \"anything-but\": [\"Pop\"] }] }" }' – Karlkarla

Answer 2

This Github repo has an example: https://github.com/Haple/sns-sqs-subscribe

#!/bin/sh

# SETUP

queue_arn=$(awslocal sqs create-queue --queue-name test_queue --output text)

echo "Queue ARN: $queue_arn"

topic_arn=$(awslocal sns create-topic --name test_topic --output text)

echo "Topic ARN: $topic_arn"

subscription_arn=$(awslocal sns subscribe \
    --topic-arn "$topic_arn" \
    --protocol sqs \
    --notification-endpoint "$queue_arn" \
    --output text)

echo "Subscription ARN: $subscription_arn" 

awslocal sns set-subscription-attributes \
    --subscription-arn "$subscription_arn" \
    --attribute-name FilterPolicy \
    --attribute-value "{ \"EVENT_TYPE\": [\"SUCCESS\"] }"

# TEST

awslocal sns publish \
    --topic-arn "$topic_arn" \
    --message "SUCCESS PAYLOAD (SHOULD GO TO THE QUEUE)" \
    --message-attributes '{"EVENT_TYPE" : { "DataType":"String", "StringValue":"SUCCESS"}}'

awslocal sns publish \
    --topic-arn "$topic_arn" \
    --message "ERROR PAYLOAD (SHOULD NOT GO TO THE QUEUE)" \
    --message-attributes '{"EVENT_TYPE" : { "DataType":"String", "StringValue":"ERROR"}}'


awslocal sqs get-queue-attributes \
    --queue-url http://localhost:4576/queue/test_queue \
    --attribute-names All

Glomeration

This one is more concise in my opinion. – Conant

Answer 3

The overall process in the previous answers is OK, but there is a huge issue:

I do not know why no one is mentioning this, but the whole thing doesn't work if you do not set the SQS queue policy document correctly.

The github example doesn't do it and for some reason, AWS documentation doesn't even mention it.

Refer to this question: Set SQS policy document with AWS CLI

I wasted a couple of days troubleshooting this. So hope it helps.

Iceblink