"Unable to execute HTTP request: Connect to <bucket-name>.s3.amazonaws.com:443 failed: connect timed out
Asked Answered
S

4

10

I am trying to write a Lambda function in java which connects to S3 and then fetch the data.

When I run it locally, using main function it works fine and returns the result. Buit when I upload it to AWS lambda and the run it, I get this error message:

"errorMessage": "Unable to execute HTTP request: Connect to bucket-name.s3.amazonaws.com:443 [bucket-name.s3.amazonaws.com/52.217.1.172] failed: connect timed out", "errorType": "com.amazonaws.SdkClientException",

I have my S3 bucket as public.

My pom.xml:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.493</version>
</dependency>
    <dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpcore</artifactId>
</dependency>
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-lambda-java-core</artifactId>
    <version>1.1.0</version>
</dependency>

My request Handler:

public class LambdaRequestHandler implements RequestHandler<String, String> {
@Autowired
public ClaimSuffixNumberService csService;
    @Override
    public String handleRequest(String input, Context context) {
        if(csService == null) {
            csService = Application.getBean(ClaimSuffixNumberService.class);
        }
        String result= csService.readAndMakeCall("claimSuffix");
        return result;
    }
}

My service

public String getObject(String fileName) {
    System.out.println("Inside Get Object");
    try {
        BasicAWSCredentials awsCreds = new BasicAWSCredentials("access-key","secret-key");
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                                .withRegion(Regions.US_EAST_1)
                                .withCredentials(new AWSStaticCredentialsProvider(awsCreds))
                                .build();
        System.out.println(s3Client);
        S3Object s3object = s3Client.getObject(new GetObjectRequest(bucket-name, object-name));
        InputStream is = s3object.getObjectContent();
        String content = StreamUtils.copyToString(is, 
 StandardCharsets.UTF_8);
        return content;
    } catch (IOException e) {
        e.printStackTrace();
    }
    return null;
}

public String readAndMakeCall(String fileName) {
    try {
        System.out.println("Reading for " + fileName);
        String content = getObject(fileName);
        ObjectMapper mapper = new ObjectMapper();
        mapper.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
        List<ClaimSuffixNumber> claimSuffixList = mapper.readValue(content, mapper.getTypeFactory().constructCollectionType(List.class, ClaimSuffixNumber.class));
        System.out.println(claimSuffixList.toString());
        for(ClaimSuffixNumber i: claimSuffixList) {
            System.out.println(i);
        }
        return claimSuffixList.toString();
    } catch (IOException e) {
        e.printStackTrace();
    }
    return " ";
}
Solidary answered 8/2, 2019 at 8:29 Comment(2)
have you configure role and bucket policy ???Knitter
what worked eventually?Oppidan
T
6

Like Mario said, your Lambda currently has no internet access. This is most likely because of your VPC configuration. I believe Mario's answer is correct. You can setup a S3 VPC endpoint. In my case, since I was working with SES and Lambda, you can't setup an endpoint for SES, so I'm adding a workaround using NAT Gateway. (NOTE: NAT Gateway does NOT have a free tier)

  • Go into the VPC console, into the Subnets menu and create a new subnet in the VPC where you have your Lambda function.
  • Next, go into the NAT Gateways menu and create a NAT Gateway. Choose a subnet that has internet access (not the new one) and is in the same VPC. You can check if a subnet has internet access by going to the subnet menu, choosing a subnet and clicking the Route Table tab. If it has internet access you should see something like:
|---------------------|------------------|
|     Destination     |      Target      |
|---------------------|------------------|
|     0.0.0.0/0       |     igw-####     |
|---------------------|------------------|
  • Next, go to the Route Tables menu and create a new route table. Make sure it's in the same VPC. When you create a new one, it comes with an internet gateway route by default so click the Routes tab and replace that route a new one pointing to your NAT Gateway. It should look like this:
|---------------------|------------------|
|     Destination     |      Target      |
|---------------------|------------------|
|     0.0.0.0/0       |     nat-####     |
|---------------------|------------------|
  • Next, go to the Subnet Associations tab and associate the subnet you just created.
  • Finally, go back to your Lambda function and change the subnet(s) to only the subnet you just created. It should now have internet access. If you need more availability you can create additional subnets with different availability zones and repeat the same process.

Just to reiterate, NAT Gateway does NOT have a free tier so keep that in mind. Here's the pricing info https://aws.amazon.com/vpc/pricing/

Tallboy answered 3/4, 2020 at 7:34 Comment(2)
this helped me, thanks. but I'm wondering, for what reason the lambda should not have an internet access all of a sudden? is that a misconfiguration, or something else? I have other lambdas using the same VPC, subnets, and security group, and they have no problems accessing whichever aws resources I specify in the policyCalcific
oh lol, my other lambdas didn't use VPC )) this lambda required specifying VPC, subnets and security group on setup, that's why I had to explicitly set them, but after that I removed VPC from lambda and it still works normally, so I don't need to use NAT, but thanks in any case (if curious, this lambda was the one created for Athena to access RDS through JDBC connector)Calcific
I
2

Based on the error i believe your lambda does’t have internet. You’re loosing internet connection when you’re configuring your lambda in a Vpc. To regain internet access you should:

  • launch lambda in the default vpc (if possible)
  • setup a s3 vpc endpoint for your custom vpc (allows applications from your custom vpc to access s3, there are other endpoints that can be configured; check pricing, this works only for some aws services)
  • setup a NAT Gateway (see pricing, connectivity not limited to aws services only)

Unrelated: there is no reason to hardcode aws credentials. All required permissions should be assigned to your execution role.

Intone answered 9/2, 2019 at 0:4 Comment(0)
K
0

Check Role assign to lambda. Lambda Role having permission to access to s3 read and write permission.

Check lambda role policy.

check for bucket policy assign to it and Cors config to bucket also.

Knitter answered 8/2, 2019 at 8:49 Comment(0)
A
0

I spent a fair bit of time to make this work. I was behind a proxy. Like Mario said, the error says no internet connection.

A couple of things needs to be checked here.

  1. bucket policy
  2. user permissions in IAM - does it have AmazonS3FullAccess for full operations
  3. Client proxy if you are using a proxy in the network and the bucket is not publicly available. like

ClientConfiguration clientConfiguration = new ClientConfiguration().withProxyHost(""). withProxyPort(3128).withProxyUsername("xxxxx").withProxyPassword("xxx");

amazonS3Client = AmazonS3ClientBuilder
    .standard()
    .withRegion(Regions.XXXX).withClientConfiguration(clientConfiguration)
    .withCredentials(new AWSStaticCredentialsProvider(
                            new BasicAWSCredentials("XXXXX", "XXXXX")))
    .build();
Annalist answered 18/1, 2022 at 0:48 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.