Get GitHub git branch for AWS CodeBuild

Question

I'm setup AWS CodeBuild to build automatically from GitHub. Other CI services provide an environment variable for the branch, but I can't find one for AWS CodeBuild. There is a CODEBUILD_SOURCE_VERSION that is set to either pr/7 where 7 is the pull request number or the git commit sha.

Given the commit sha, I've been trying to get the branch name, but no luck so far.

git branch --contains <commitsha> doesn't work because it is a detached head. How do I get the git branch for the commit from CodeBuild?

Mcfadden

Answer 1

You can run:

git branch -a --contains <sha>

-a means all branches. If your sha is in no branch, which could happen in some cases, you won't see anything.

Update:

You can also get this information from CodeBuild environment variables. However, it will depend on how you are triggering your build or your specific case (Read the variable docs)

Brooklime

Answer 2

You can get inspired by https://github.com/thii/aws-codebuild-extras

⚠️I don't recommend to run the curl command for security sake! If some vilain stole access to the thii/aws-codebuild-extras repo, you are screwed!

Just copy paste the script (understand it!) and add it to your docker image and then call it from your file system.

#!/bin/bash

export CI=true
export CODEBUILD=true

export CODEBUILD_GIT_BRANCH=`git symbolic-ref HEAD --short 2>/dev/null`
if [ "$CODEBUILD_GIT_BRANCH" == "" ] ; then
  CODEBUILD_GIT_BRANCH=`git branch -a --contains HEAD | sed -n 2p | awk '{ printf $1 }'`
  export CODEBUILD_GIT_BRANCH=${CODEBUILD_GIT_BRANCH#remotes/origin/}
fi

export CODEBUILD_GIT_MESSAGE=`git log -1 --pretty=%B`
export CODEBUILD_GIT_AUTHOR=`git log -1 --pretty=%an`
export CODEBUILD_GIT_AUTHOR_EMAIL=`git log -1 --pretty=%ae`
export CODEBUILD_GIT_COMMIT=`git log -1 --pretty=%H`
export CODEBUILD_GIT_TAG=`git describe --tags --abbrev=0`

export CODEBUILD_PULL_REQUEST=false
if [[ $CODEBUILD_GIT_BRANCH == pr-* ]] ; then
  export CODEBUILD_PULL_REQUEST=${CODEBUILD_GIT_BRANCH#pr-}
fi

export CODEBUILD_PROJECT=${CODEBUILD_BUILD_ID%:$CODEBUILD_LOG_PATH}
export CODEBUILD_BUILD_URL=https://$AWS_DEFAULT_REGION.console.aws.amazon.com/codebuild/home?region=$AWS_DEFAULT_REGION#/builds/$CODEBUILD_BUILD_ID/view/new

echo "==> AWS CodeBuild Extra Environment Variables:"
echo "==> CI = $CI"
echo "==> CODEBUILD = $CODEBUILD"
echo "==> CODEBUILD_GIT_AUTHOR = $CODEBUILD_GIT_AUTHOR"
echo "==> CODEBUILD_GIT_AUTHOR_EMAIL = $CODEBUILD_GIT_AUTHOR_EMAIL"
echo "==> CODEBUILD_GIT_BRANCH = $CODEBUILD_GIT_BRANCH "
echo "==> CODEBUILD_GIT_COMMIT = $CODEBUILD_GIT_COMMIT"
echo "==> CODEBUILD_GIT_MESSAGE = $CODEBUILD_GIT_MESSAGE"
echo "==> CODEBUILD_GIT_TAG = $CODEBUILD_GIT_TAG"
echo "==> CODEBUILD_PROJECT = $CODEBUILD_PROJECT"
echo "==> CODEBUILD_PULL_REQUEST = $CODEBUILD_PULL_REQUEST"

Inimitable

@joe-bowbeer Can you provide us with a lin the the doc that states that? Thanks! I then will update my answer – Inimitable

The CodeBuild spec states: "Do not set any environment variable with a name that begins with CODEBUILD_" -- docs.aws.amazon.com/codebuild/latest/userguide/… – Nevus

Codebuild is no longer happy with this. – Push

Answer 3

It's now possible to obtain this information directly from CodeBuild environmental variables:

• CODEBUILD_WEBHOOK_BASE_REF: The base reference name of the webhook event that triggers the current build. For a pull request, this is the branch reference.
• CODEBUILD_WEBHOOK_HEAD_REF: The head reference name of the webhook event that triggers the current build. It can be a branch reference or a tag reference.
• CODEBUILD_WEBHOOK_TRIGGER: Shows the webhook event that triggered the build. This variable is available only for builds triggered by a webhook. The value is parsed from the payload sent to CodeBuild by Github, Github Enterprise, or Bitbucket. The value's format depends on what type of event triggered the build.
  • For builds triggered by a pull request, it is pr/pull-request-number.
  • For builds triggered by creating a new branch or pushing a commit to a branch, it is branch/branch-name.
  • For builds triggered by a pushing a tag to a repository, it is tag/tag-name.

Booklet

Are there prerequisites to have access to those env vars? I tried logging them in a build but they're all coming back as empty imgur.com/a/pgEu8DQ – Stent

These probably only appear when the build is actually triggered by a webhook -- manually triggering, or retrying a build that you stopped won't have them. (/guessing) – Russellrusset

@Russellrusset yes, that's true – Booklet

Answer 4

Add the following command to the install or pre_build phase of your buildspec.yml:

bash -c "$(curl -fsSL https://raw.githubusercontent.com/thii/aws-codebuild-extras/master/install)"

You can get more information about the build via the following environment variables: CI, CODEBUILD, CODEBUILD_GIT_AUTHOR, CODEBUILD_GIT_AUTHOR_EMAIL, CODEBUILD_GIT_BRANCH, CODEBUILD_GIT_COMMIT, CODEBUILD_GIT_MESSAGE, CODEBUILD_GIT_TAG, CODEBUILD_PROJECT, CODEBUILD_PULL_REQUEST.

Propend

This is such a lifesaver-- thanks Thi! I was dreading building really solid Slack notifications and you just did all the heavy lifting :) – Allies

⚠️Nice! But I don't recommend to run the curl command for security sake! If some vilain stole access to the thii/aws-codebuild-extras repo, you are screwed! See https://mcmap.net/q/496439/-get-github-git-branch-for-aws-codebuild for full copy/pasted script – Inimitable

If you don't want to keep a local copy, at least pin to a specific version like this: bash -c "$(curl -fsSL https://raw.githubusercontent.com/thii/aws-codebuild-extras/b9e1d1ef105378349c2c50ee60e3af4a72d77d7b/install)" – Polarity

Answer 5

CodeBuild strips git information from the filesystem. There is no .git folder, so running a git command will be fruitless.

I have added a parameter to my CI/CD CloudFormation template:

  GitBranch:
    Description: Github branch to be deployed
    Type: String
    Default: master

And I have a Bash script that creates / updates the CI/CD stack:

readonly git_branch=$(git branch 2>/dev/null | grep "^*" | colrm 1 2)

aws cloudformation create-stack \
  --stack-name ${cicd_stack_name} \
  --parameters ParameterKey=GitBranch,ParameterValue=${git_branch}

I then export the value as an environment variable to CodeBuild machine:

CodeBuildProject:
Type: AWS::CodeBuild::Project
Properties:
  Environment:
    Type: LINUX_CONTAINER
    Image: aws/codebuild/java:openjdk-8
    EnvironmentVariables:
      - Name: GIT_BRANCH
        Value: !Ref GitBranch

Now I have access to it in my buildspec.yml:

post_build:
  commands:
    - echo [PHASE] Entered the post_build phase...
    - echo "[DEBUG] Git branch ${GIT_BRANCH}"

Cockatiel

This is great, but in the reference architecture for codebuild (reference architecture for ecs continuous deployment, anyway) you must specify a fixed branch. I want codebuild to build any branch that I push, and any PRs and tags as well. Is there a reference architecture which handles all of this? (FYI the solution you proposed worked for me, within the reference architecture which has these limitations.) It would be great to get a reference architecture for CI Docker Builds from AWS which does not push ECS or Fargate, and only builds the docker images. – Off

@Off Thanks for the kind words! I do have a design that builds the whole stack for any given branch, not sure how referential it is but it works for me ;-) – Cockatiel

I understand now after talking to coworkers who have gone deeper that my issue is a limitation of CodePipeline, not with CodeBuild. Thanks for reply – Off

Answer 6

When I tried, the Variables CODEBUILD_WEBHOOK_BASE_REF, CODEBUILD_WEBHOOK_HEAD_REF, CODEBUILD_WEBHOOK_TRIGGER were returning empty.

The solution provided by Yves M. worked, but with a few caveats:

1. In my CodePipeline, the Source's Output artifact format had to be set with Full Clone.
2. Then I had to add Clone Permissions
3. The solution doesn't work when you manually trigger a build.

Finally, since I only needed the Branch name, I simplified the solution a bit:

commands:
  - | 
    GIT_BRANCH="$(git symbolic-ref HEAD --short 2>/dev/null)"
    if [ "$GIT_BRANCH" = "" ] ; then
      GIT_BRANCH="$(git rev-parse HEAD | xargs git name-rev | cut -d' ' -f2 | sed 's/remotes\/origin\///g')";
    fi
   - echo GIT_BRANCH - $GIT_BRANCH

So for a branch named staging output will be:

GIT_BRANCH - staging

Varied

Answer 7

If you couple AWS CodeBuild with AWS CodePipeline, you can reliably get the branch name through the #{SourceVariables.BranchName} variable. You can expose this as an environment variable to your CodeBuild project and use it as needed.

No custom scripting is needed

Empiricism

Answer 8

Just expanding on the answer by @zdenek-f. If you are using just CodeBuild with a Source setup (not CodePipeline) then you can add the following in your buildspec file to get the branch name

This will set the branch name if you manually start the build or if you have a Git Webhook setup with GitHub.

export BRANCH_NAME=$(echo ${CODEBUILD_WEBHOOK_TRIGGER:-$CODEBUILD_SOURCE_VERSION} | cut -d'/' -f2)

N.B Only tested with GitHub.

Coloration

Answer 9

The Full clone that CodePipeline does leaves the local repository .git in a detached HEAD state, meaning that in order to get the branch name you will have to either get it with the help of CodeBuild environment variables to retrieve it from CodePipeline, or to execute the following command (either in your buildspec.yml or later in Elastic Beanstalk, if that's what you're using after deployment):

git branch -a --contains HEAD | sed -n 2p | awk '{ printf $1 }'

If you want to go with the CodeBuild environment variables option, then navigate to:

CodePipeline -> Edit: Build -> Environment variables - optional

and add BranchName: #{SourceVariables.BranchName} as Plaintext

Then, in your buildspec.yml code you can do the following:

version: 0.2

phases:
build:
  commands:
    - echo Branch - $BranchName
    - echo Commit - $CommitId
    - echo Checking out branch - $BranchName
    - git checkout $BranchName
artifacts:
files:
  - '**/*'

The above will print:

Branch - master
Commit - f4f78b319c308600eab015a5d6529add21660dc1 
Checking out branch - master

PS 1: The Environment Variables must be set from CodePipeline -> Edit: Build -> Environment variables - optional. If you set these variables in CodeBuild -> Edit -> Environment -> Additional configuration -> Environment variables it WON'T WORK! This is so dumb, but I spent too much time on this myself.
PS 2: For a bigger list of Environment variables during CodeBuild, see Variables List, Action Variables, and CodeBuild variables.

Doyenne