I'm trying to run a 100 node AWS Batch job, when I set my computing environment to use only m4.xlarge and m5.xlarge instances everything works fine and my job is picked up and runs. However, when ...

I'm trying to get my elastic file system (EFS) to be mounted in my docker container so it can be used with AWS batch. Here is what I did: Create a new AMI that is optimized for Elastic Container ...

I wanted to ask the Snakemake community if anybody has had success implementing Snakemake workflows in AWS Batch. Page 4 of a recent publication from Oct 2018 seems to suggest that Snakemake does ...

In AWS Batch, I am trying to override the image hardcoded in the job definition at the time of submitting a job. But I dont see any documentation around that. When I try to add an image field duri...

I've tried to get all subnet ids to add aws batch with terraform with following code: data "aws_subnet_ids" "test_subnet_ids" { vpc_id = "default" } data "aws_subnet" "test_subnet" { count = "${...

I want to create jobs in AWS Batch that vary on the image that is used to launch the container. I'd like to do this without creating a different Job Definition for each image. Is it possible to par...

while creating a batch job, we can provide vCPUs and Memory in Container properties. In cloudwatch, we can check memory and CPU usage per cluster under Elastic Container Service per cluster. Is t...

I have a docker image that is 1.46GB on my local machine, but when this is pushed to AWS ECR (either via my local machine or via CicleCI deployment) it is only 537.05MB. I'm pretty new to Docker an...

I have an existing AWS Steps orchestration that is executing a AWS Batch job via lambdas. However AWS have recently added the ability to directly invoke other services like AWS Batch from a step. I...

I am following this tutorial to run a simple fetch-and-run example in AWS batch. However, I'm unable to pass arguments to the script fetched through this example. The basic example will produce th...

We setup a batch compute environment, job queue, and job definition. The min CPUs for the compute environment is set to 16, so it should always have at least one EC2 instance running. It's a MANAGE...

Is there a way to set a maximum running time for AWS Batch jobs (or queues)? This is a standard setting in most batch managers, which avoids wasting resources when a job hangs for whatever reason. ...

I am looking for a way to limit the number of batch jobs that are running by holding the remaining jobs in the queue. Is it possible with aws batch?

We are looking for aws batch job and we want to submit this batch job on a certain predefined interval. For e.g.: we want to auto submit this job every 4 hours. What could be a preferred way to a...

Flow :- Dynamo DB --> Lambda --> Batch If a role arn is inserted in dynamo DB, it is retrieved from lambda event, it is then submitted to batch using submit_job API with role arn being passed as ...

Context: AWS, S3, Lambda, Batch. I have a lambda that is triggered when a file is uploaded in a S3 Bucket. I want that the lambda submit a Batch job. (edit: Between S3 and Lambda everything works...

I have written a lambda that is triggered off s3 bucket to unzip a zip file and process a text document inside. Due to the limitation of memory of lambda i need to move my process over to something...