AWS Lambda times out connecting to RedShift

Question

My Redshift cluster is in a private VPC. I've written the following AWS Lamba in Node.js which should connect to Redshift (dressed down for this question):

'use strict';
console.log('Loading function');

const pg = require('pg');

exports.handler = (event, context, callback) => {
var client = new pg.Client({
    user: 'myuser',
    database: 'mydatabase',
    password: 'mypassword',
    port: 5439,
    host: 'myhost.eu-west-1.redshift.amazonaws.com'
});


    // connect to our database
    console.log('Connecting...');
    client.connect(function (err) {
        if (err) throw err;

        console.log('CONNECTED!!!');

    });

};

I keep getting Task timed out after 60.00 seconds unfortunately. I see in the logs "Connecting...", but never "CONNECTED!!!".

Steps I've taken so far to get this to work:

• As per Connect Lambda to Redshift in Different Availability Zones I have the Redshift cluster and the Lamba function in the same VPC
• Also Redshift cluster and the Lamba function are on the same subnet
• The Redshift cluster and the Lamba function share the same security group
• Added an inbound rule at the security group of the Redshift cluster as per the suggestion here (https://github.com/awslabs/aws-lambda-redshift-loader/issues/86)
• The IAM role associated with the Lamba Function has the following policies: AmazonDMSRedshiftS3Role, AmazonRedshiftFullAccess, AWSLambdaBasicExecutionRole, AWSLambdaVPCAccessExecutionRole, AWSLambdaENIManagementAccess scrambled together from this source: http://docs.aws.amazon.com/lambda/latest/dg/vpc.html (I realize I have some overlap here, but figured that it shouldn't matter)
• Added Elastic IP to the Inbound rules of the Security Group as per an answer from a question listed prior (even if I don't even have a NAT gateway configured in the subnet)
• I don't have Enhanced VPC Routing enabled because I figured that I don't need it.
• Even tried it by adding the Inbound rule 0.0.0.0/0 ALL types, ALL protocols, ALL ports in the Security Group (following this question: Accessing Redshift from Lambda - Avoiding the 0.0.0.0/0 Security Group). But same issue!

So, does anyone have any suggestions as to what I should check?

*I should add that I am not a network expert, so perhaps I've made a mistake somewhere.

Knopp

As far as I can see, you did everything fine. I have very similar setup, with the difference being our redshift clusters have public ip, for reasons different than lambda handling, and was changed from private to public at later time. I also use python, but shouldn't really matter. If you have aws support in your plan, I would use it. If you don't or want to tinker your way out of this one, setup a small ec2 machine in the same vpc as your redshift and lambda and see if you can connect from there. If you can, problem is probly with your lambda, if you can't, problem is likely with your vpc/RS. – Fugere

Thanks for your comment, but I used the public hostname while I should use the private IP (see accepted answer) – Knopp

Ah crap, sorry, this tab was sitting in my browser since yesterday, didn't see the answer:) – Fugere

No worries. You were kind enough to try to help... – Knopp

Possible duplicate of Conecting AWS Lambda to Redshift - Times out after 60 seconds – Bohaty

Answer 1

The timeout is probably because your lambda in VPC cannot access Internet in order to connect to your cluster(you seem to be using the public hostname to connect). Your connection options depend on your cluster configuration. Since both your lambda function and cluster are in the same VPC, you should use the private IP of your cluster to connect to it. In your case, I think simply using the private IP should solve your problem.

Depending on whether your cluster is publicly accessible, there are some points to keep in mind.

• If your cluster is configured to NOT be publicly accessible, you can use the private IP to connect to the cluster from your lambda running in a VPC and it should work.

• If you have a publicly accessible cluster in a VPC, and you want to connect to it by using the private IP address from within the VPC, make sure the following VPC parameters to true/yes:

  • DNS resolution
  • DNS hostnames

The steps to verify/change these settings are given here.

If you do not set these parameters to true, connections from within VPC will resolve to the EIP instead of the private IP and your lambda won't be able to connect without having Internet access(which will need a NAT gateway or a NAT instance).

Also, an important note from the documentation here.

If you have an existing publicly accessible cluster in a VPC, connections from within the VPC will continue to use the EIP to connect to the cluster even with those parameters set until you resize the cluster. Any new clusters will follow the new behavior of using the private IP address when connecting to the publicly accessible cluster from within the same VPC.

Valorie

Oh no! I am using the public hostname to connect! Thank you very much. This solves it. – Knopp

For what it's worth, I had a similar issue. My problem was that I had set the lambda to have access to my public subnets only. My public subnet is routing all outbound traffic to an internet gateway, while my private subnets are routing outbound traffic via an NAT Gateway. But according to the doc "You cannot use an Internet gateway attached to your VPC, since that requires the ENI to have public IP addresses." Switching the lambda to the private subnets (and therefore using the NAT Gateway) solved the problem. – Victorvictoria

Answer 2

My issues got resolved after adding the CIDR range of the VPC to the Redshift Inbound rules.

Gyrostatics

Please provide more details for others to get what you propose as a solution. – Expansile

Answer 3

For the ones that are trying to move to redshift serverless due to it's recent release to the public... this may be a commom issue but at least for me the answer from @pcothenet worked:

For what it's worth, I had a similar issue. My problem was that I had set the lambda to have access to my public subnets only. My public subnet is routing all outbound traffic to an internet gateway, while my private subnets are routing outbound traffic via an NAT Gateway. But according to the doc "You cannot use an Internet gateway attached to your VPC, since that requires the ENI to have public IP addresses." Switching the lambda to the private subnets (and therefore using the NAT Gateway) solved the problem. – pcothenet

You must use the Endpoint to connect.

Best.

Clung

Answer 4

I had this same issue and followed the steps above and I found that in my case the issue was that the lambda was in a subnet that did not have a route to the NAT gateway. So I moved the lambda into a subnet with route to the NAT gateway.

Avestan