AWS CodeBuild, buildspec.yml "bad substitution" error

Question

I'm using AWS CodeBuild, and I need to manipulate an environment variable. I originally tried using a bash pattern substitution, like this, in the buildspec.yml:

  build:
    on-failure: ABORT
    commands:
      - env="${CODEBUILD_WEBHOOK_TRIGGER/tag\//}"

CODEBUILD_WEBHOOK_TRIGGER should be something like tag/my-tag-name, and I want to remove the tag/ part of it. This command works fine from a local bash shell, but when performned in CodeBuild, this is the output:

[Container] 2021/08/02 21:29:28 Running command env="${CODEBUILD_WEBHOOK_TRIGGER/tag\//}"
/codebuild/output/tmp/script.sh: 4: Bad substitution
...
[Container] 2021/08/02 21:29:28 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: env="${CODEBUILD_WEBHOOK_TRIGGER/tag\//}". Reason: exit status 2

I ended up replacing the pattern substitution with an awk command just to get it working, but it makes for more complex code. And I don't understand why the pattern substitution doesn't work?

Here's the awk command I ended up using, which is working fine:

  build:
    on-failure: ABORT
    commands:
      - env="`echo $CODEBUILD_WEBHOOK_TRIGGER | awk -F/ '$1=="tag" {print $2;}'`"

Huskamp

Thanks @Philippe, that works! If you add as an answer I'll be happy to accept and upvote it. – Huskamp

It's probably using the dash shell to run the command. Substitution with ${var/pattern/replacement} is an extension to the standard shell syntax, and is supported by ksh, bash, and zsh, but not by more basic shells like dash. ${var#pattern}, on the other hand, is part of the POSIX standard, and will work in any POSIX-compliant shell. – Indrawn

Answer 1

CodeBuild might not be using bash. Try this :

env="${CODEBUILD_WEBHOOK_TRIGGER#tag/}"

Spellbound

Answer 2

Gordon's comment on the original post above explains why this happens - that bash extensions won't work on more basic shells (and it appears that CodeBuild's Ubuntu containers may use dash).

However, you can force CodeBuild to use bash with the following in your buildspec:

env:
  shell: bash

(documentation at https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#build-spec-ref-syntax)

This should then allow you to use any of the substitutions supported by bash, and it appears to resolve this issue.

(Alternatively, if you're calling a separate shell script from your buildspec, you can just ensure that you have a shebang at the top of it).

Sensualist