How to pass script arguments to AWS Batch fetch-and-run

Question

I am following this tutorial to run a simple fetch-and-run example in AWS batch. However, I'm unable to pass arguments to the script fetched through this example.

The basic example will produce this execution:

export BATCH_FILE_TYPE="script"

export BATCH_FILE_S3_URL="s3://my-bucket/my-script"

fetch_and_run.sh script-from-s3 [ <script arguments> ]

where script arguments are only mentioned in:

This shows that it supports two values for BATCH_FILE_TYPE, either “script” or “zip”. When you set “script”, it causes fetch_and_run.sh to download a single file and then execute it, in addition to passing in any further arguments to the script.

I tried passing them with AWS CLI through the --parameters and --container-overrides parameters (in the latter under the command key), however they are not received from the script.

I would like not to modify either my Dockerfile ENTRYPOINT for each run or the fetch_and_run.sh script, but I cannot understand how to achieve this differently.

Charles

Answer 1

Mixing these example of job definitions I achieved it with aws batch using:

aws batch submit-job --job-name <job_name> --job-definition <job_def_name> \
  --job-queue <queue_name> \
  --container-overrides '{
    "command": ["<script-from-s3>", "Ref::param1", "Ref::param2"], \
    "environment": [ \
      {"name": "BATCH_FILE_S3_URL", "value": "<script-from-s3>"}, \
      {"name": "BATCH_FILE_TYPE", "value": "script"}]}' \
  --parameters '{"param1": "<param1>", "param2": "<param2>"}'

Charles

Epic answer, first place I've found a working solution, the exact syntax in the AWS CLI docs don't work – Butterfish

Note that it's crucial that you use the EXEC form of ENTRYPOINT ENTRYPOINT ["/usr/local/bin/fetch_and_run.sh"] not the shell form. The shell form ignores command line args – Allnight

Thanks for posting this. I'm wondering why you couldn't just add the command-line args to the "command" property directly and avoid the "Ref::param" with --parameters. – Folklore