Use two sources in an AWS-CodePipeline pipeline

Question

I have a specific case which I'm not sure if it's possible with AWS CodePipeline. I'm not able to find any information about such scenario in the documentation and google.

So I would like to know if I can set two sources in a pipeline (it could be in the same stage or different stages).

Here is my use case:

• I would like my pipeline to start when a file (a specific object) is modified in my s3 bucket
• When this file changes and the pipeline is triggered, I would like to clone a codecommit repository and then process the build and other stages.
• On the other hand, when there is a commit on the master branch of my codecommit repository, I would like the pipeline to start and build my sources.
• So The pipeline should be triggered either when the change comes from s3 or codecommit
• I don't want to version the s3 file in my codecommit repository because it should be encrypted and used by others teams than dev team working with the git repository
• And any time my pipeline starts either if it's from the s3 bucket change or the codecommit push, I should source the commit from the repository for build purposes.

Is it possible to use two source actions in a pipeline as described above? If yes, how to achieve this?

Campo

Answer 1

Yes. It is possible to have two sources for an AWS CodePipeline. Or many for that matter. The two sources have to be in your first stage.

Then in your build phase properties, you need to tell it that you are expecting two sources.

Then tell the build project which is your primary source. This is going to be the one that you want your build project to execute the codebuild.

From your buildspec or from any scripts you call, you can then access the source directories by referencing:

• $CODEBUILD_SRC_DIR_SourceOutput1

• $CODEBUILD_SRC_DIR_SourceOutput2

Just replace SourceOutputX above with what you call your output from the source stage.

I found the following link with more information: https://docs.aws.amazon.com/codebuild/latest/userguide/sample-multi-in-out.html

Sande

If you aren't seeing CodePipeline trigger on the second GitHub repository, check to see that a GitHub webhook was created in the repository. I found this out the hard way. The GitHub user you use to authenticate with AWS must have admin privileges on the second repository you add when you set up the pipeline. – Aube

Answer 2

Yes, CodePipeline allows multiple source actions in a single pipeline. A change in either source will trigger a pipeline execution. The thing to know is that every pipeline execution will pull the latest source for both actions (not just the one with a change that triggered the pipeline execution).

Stratiform

can I proceed like this ?: - Create my first pipeline stage : the source - In that source stage : create one source action type with my first source (lets say s3 bucket) - Create a second source action type for this same source stage with my second source (codecommit repository) - Then create the build stage, which will use the last output artifact - And follow the standard workflow for the rest of the pipeline execution Would that work? In which order? – Campo

Yes, you can incrementally add actions and stages to your pipeline subject to two constraints: stages can't be empty and CodePipeline requires two or more stages. – Stratiform

i cant get two sources working in codepipeline with CDK.... build only sees content of the primary source. – Omentum

Can one of the sources NOT trigger the pipeline? – Myel

@Myel Yes, you have to uncheck "Start the pipeline on source code change" on your source action – Greiner