Trigger specific AWS Codepipeline source stage when change is made to a specific directory in repo

Question

I have a number of services in a single GitHub repository, each service has its own CodePipeline on AWS managed through Terraform. Instead of triggering all of the pipelines on commit, I'd like to know how I can trigger each service's pipeline if its directory had any changes on commit, without having to split the services each into its own repository.

Polycythemia

Answer 1

It ended up being simpler than I had anticipated, there are github actions that do exactly what I needed.

This action checks whether a path had a change, and this action triggers a specific pipeline.

Polycythemia

Answer 2

I don't think that there's a conditional source stage support per folder at code pipeline as we speak. Just finished checking this documentation about sources in CodePipeline. It does not seem to contain a folder-level filtering.

You could try this CDK-based template solution which showcases a mono-repository, which is composed of multiple services, have different CI/CD pipelines for each service. The solution detects which top level directory the modification happened and triggers the AWS CodePipeline configured to that directory.

This is sad but they might add it in the future. I've also wanted Quality gates, images from readme files in code-commit but these features seem too hard to implement haha.

Otila

Perhaps not CDK itself, but the lambda idea is good enough, I can have a GitHub webhook trigger the lambda, and then the lambda triggers the relevant pipeline – Polycythemia

Glad it helped man! Terrraform or other ways of provisioning are not a bad idea to be honest! – Otila

Answer 3

Create GitHub workflow under actions with the following yaml file:

name: Conditional Deploy

on:
  push:
    branches: [ "master" ]
    paths:
      - 'ENTER_YOUR_DIRECTORY/**'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Trigger AWS CodePipeline
        uses: zulhfreelancer/[email protected]
        with:
          aws-region: ${{ secrets.AWS_REGION }}
          aws-access-key: ${{ secrets.AWS_PIPELINE_ACCESS_KEY }}
          aws-secret-key: ${{ secrets.AWS_PIPELINE_SECRET_KEY }}
          pipeline-name: ${{ secrets.AWS_PIPELINE_NAME }}

Create 4 secrets (AWS_REGION, AWS_PIPELINE_ACCESS_KEY, AWS_PIPELINE_SECRET_KEY, AWS_PIPELINE_NAME) in Github. You can check here on how to create secrets. Change ENTER_YOUR_DIRECTORY to your directory name. On change in your directory, it will trigger the aws code pipeline.

I spend more than a couple of hours to figure this out.

Hope this helps and saves some time for you.

Fleuron

Answer 4

This approach will help monorepo-based application in AWS CodeCommit. With this approach each app/microservice in your monorepo-based application can have a dedicated CI/CD pipeline which will detect changes in relevant app directory!

https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/automatically-detect-changes-and-initiate-different-codepipeline-pipelines-for-a-monorepo-in-codecommit.html

AWS Sample code - https://github.com/aws-samples/monorepo-multi-pipeline-trigger

Enliven

Answer 5

In CDK you should be able to do this now. Looks like this is only supported for code star connections but that is the new preferred way of connecting anyway.

new codepipeline.Pipeline(this, "pipeline", {
      triggers: [
        {
          providerType: codepipeline.ProviderType.CODE_STAR_SOURCE_CONNECTION,
          gitConfiguration: {
            sourceAction,
            pullRequestFilter: [
              { 
                events: [...], // events to apply filters on, default is all
                filePathsExcludes: ["..."], // paths you want to exclude
                filesPathsIncludes: ["..."], // paths you want to include
              },
            ],
          },
        },
      ],
    });

Toulouselautrec