aws batch: submit job using lambda

Question

Context: AWS, S3, Lambda, Batch.

I have a lambda that is triggered when a file is uploaded in a S3 Bucket. I want that the lambda submit a Batch job.

(edit: Between S3 and Lambda everything works fine. The problem is between Lambda and Batch.)

Q: What is the role I have to give to the lambda in order to be able to submit the batch job?

My lambda gets an AccessDeniedException and fail to submit the job when:

const params = {
  jobDefinition: BATCH_JOB_DEFINITION,
  jobName: BATCH_JOB_NAME,
  jobQueue: BATCH_JOB_QUEUE,
};

Batch.submitJob(params).promise() .then .......

Andresandresen

Answer 1

It seems that this was the role I was looking for: batch:SubmitJob. Using this role, the lambda was able to submit the job.

iamRoleStatements:
  - Effect: Allow
    Action:
      - batch:SubmitJob
    Resource: "arn:aws:batch:*:*:*"

Andresandresen

Hi Costin. How did you figure out how to do this? Specifically, how did you realize the action you needed was SubmitJob? And how did you know the resource had to be arn:aws:batch:*:*:*? I see the resource has the same pattern as some other examples I found online (e.g. n2ws.com/blog/aws-automation/lambda-function-s3-event-triggers). But what is the difference between arn:aws:batch:*:*:* and batch:*? – Gurney

I do not remember where I've seen it, but I thing there was a lucky guess from the AWS Batch - Actions page. The arn:aws:batch:*:*:* follows the ARN syntax. Once you understand how it is built you'll be able to target any AWS resource, with closed eyes :) – Andresandresen

It might be good to restrict the resources available (e.g. don't want your testing lambda to deploy to a batch instance in production) – Hydrobomb

Answer 2

You can Create a Policy like AWS Batch Managed Policy,

The following Policy Allows Admin Access,You can modify it as per your needs:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "batch:*",
                "cloudwatch:GetMetricStatistics",
                "ec2:DescribeSubnets",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeKeyPairs",
                "ecs:DescribeClusters",
                "ecs:Describe*",
                "ecs:List*",
                "logs:Describe*",
                "logs:Get*",
                "logs:TestMetricFilter",
                "logs:FilterLogEvents",
                "iam:ListInstanceProfiles",
                "iam:ListRoles"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": ["iam:PassRole"],
            "Resource": [
                "arn:aws:iam::*:role/AWSBatchServiceRole",
                "arn:aws:iam::*:role/ecsInstanceRole",
                "arn:aws:iam::*:role/iaws-ec2-spot-fleet-role",
                "arn:aws:iam::*:role/aws-ec2-spot-fleet-role",
                "arn:aws:iam::*:role/AWSBatchJobRole*"
            ]
        }
    ]
}

Attach the policy to lambda and try it again , Refer AWS Documentation

Fishwife

Thanks Kush. There is no problem between S3 and Lambda. The problem is that Lambda can not launch the Batch (job). – Andresandresen

Sorry Kush, but this is not the answer. These are the roles for the Batch. What I was looking for is the role for the Lambda. The role which enables submitJob() from within a lambda. Thanks. – Andresandresen

But it can be achieved in that way also refer the aws documentaion – Fishwife