I am pretty convinced that you cannot call an SQS queue from within a VPC using Lambda using an SQS endpoint. I'd consider it a bug, but maybe the Lambda team did this for a reason. In any case, You will get a message timeout. I cooked up a simple test Lambda
import json
import boto3
import socket
def lambda_handler(event, context):
print('lambda-test SQS...')
sqsDomain='sqs.us-west-2.amazonaws.com'
addr1 = socket.gethostbyname(sqsDomain)
print('%s=%s' %(sqsDomain, addr1))
print('Creating sqs client...')
sqs = boto3.client('sqs')
print('Sending Test Message...')
response = sqs.send_message(
QueueUrl='https://sqs.us-west-2.amazonaws.com/1234567890/testq.fifo',
MessageBody='Test SQS Lambda!',
MessageGroupId='test')
print('SQS send response: %s' % response)
return {
'statusCode': 200,
'body': json.dumps(response)
}
I created a VPC, subnet, etc per - Configuring a Lambda function to access resources in a VPC. The EC2 instance in this example has no problem invoking SQS through the private endpoint from the CLI per this tutorial.
If I drop my simple Lambda above into the same VPC and subnet, with SQS publishing permissions etc. and invoke the test function it will properly resolve the IP address of the SQS endpoint within the subnet, but the call will timeout (making sure your Lambda timeout is more than 60 seconds to let boto fail). Enabling boto debug logging further confirms that the IP is resolved correctly and the HTTP request to SQS times out.
I didn't try this with a non-FIFO queue but as the HTTP call is failing on connection request this shouldn't matter. It's got to be a routing issue from the Lambda as the EC2 in the same subnet works.
I modified my simple Lambda and added an SNS endpoint and did the same test which worked. The issue issue appears to be specific to SQS best I can tell.
import json
import boto3
import socket
def testSqs():
print('lambda-test SQS...')
sqsDomain='sqs.us-west-2.amazonaws.com'
addr1 = socket.gethostbyname(sqsDomain)
print('%s=%s' %(sqsDomain, addr1))
print('Creating sqs client...')
sqs = boto3.client('sqs')
print('Sending Test Message...')
response = sqs.send_message(
QueueUrl='https://sqs.us-west-2.amazonaws.com/1234567890/testq.fifo',
MessageBody='Test SQS Lambda!',
MessageGroupId='test')
print('SQS send response: %s' % response)
return {
'statusCode': 200,
'body': json.dumps(response)
}
def testSns():
print('lambda-test SNS...')
print('Creating sns client...')
sns = boto3.client('sns')
print('Sending Test Message...')
response = sns.publish(
TopicArn='arn:aws:sns:us-west-2:1234567890:lambda-test',
Message='Test SQS Lambda!'
)
print('SNS send response: %s' % response)
return {
'statusCode': 200,
'body': json.dumps(response)
}
def lambda_handler(event, context):
#return testSqs()
return testSns()
I think your only options are NAT (per John above), bounce your calls off a local EC2 (NAT will be simpler, cheaper, and more reliable), or use a Lambda proxy outside the VPC. Which someone else suggested in a similar post. You could also subscribe an SQS queue to an SNS topic (I prototyped this and it works) and route it out that way too, but that just seems silly unless you absolutely have to have SQS for some obscure reason.
I switched to SNS. I was just hoping to get some more experience with SQS. Hopefully somebody can prove me wrong, but I call it a bug.
