Jenkins: Trigger Multi-branch pipeline on upstream change

Question

I am currently testing the pipeline approach of Jenkins 2.0 to see if it works for the build environment I am using.

First about the environment itself. It currently consists of multiple SCM repositories. Each repository contains multiple branches, for the different stages of the development and each branch is build with multiple configurations. Not all configurations apply to every repository.

Currently every repository/branch is setup as a Matrix Project for the different configurations. Each project exposes it's building results as a artifact and these artifacts are used in the downstream projects.

The different repositories depend on each other, so a successful build on a upstream job triggers some specific down stream jobs. Currently all that works, but the amount of work required to setup a new branch or to tweak the building process is a lot, since many different projects need to be altered by hand.

Now I wanted to give the new pipelines a try. My idea was to create multi-branch pipeline projects and place a Jenkinsfile inside the repository containing the instructions for the build.

The main problem is getting the builds to trigger each other, because basically a build in a specific upstream branch, needs to trigger a downstream branch. How ever the information what downstream branches need to be triggered is not known to the upstream project. Each downstream project fetches the artifacts from some upstream branches and the ideal solution would be if the downstream build would be triggered in case the upstream build that is the source for the artifact finishes it's build.

The problem is only the downstream projects really know what artifacts they require. The branch names are unlikely to match in most cases and that makes triggering the builds from the upstream project very difficult.

Currently this is solved using the ReverseBuildTrigger. But this thing stops working as soon as it gets anywhere close to a pipeline.

I am really at a loss how to get this working. Is there any way to get something like the ReverseBuildTrigger working inside pipeline scripts?

Also triggering the entire downstream build for all branches in case a single branch upstream is changed is not a option. This would create far too many equal builds.

Harts

What SCM type do you use? – Frodeen

@Frodeen Shouldn't be relevant. But it is SVN. – Harts

Answer 1

I'm currently trying to get this to work for our deployment. The closest I've got is adding the following to the downstream Jenkinsfile;

properties([
    pipelineTriggers([
        triggers: [
            [
                $class: 'jenkins.triggers.ReverseBuildTrigger',
                upstreamProjects: "some_project", threshold: hudson.model.Result.SUCCESS
            ]
        ]
    ]),
])

That at least gets Jenkins to acknowledge that it should be triggering when 'some_project' get's built i.e it appears in the "View Configuration" page.

However so far builds of 'some_project' still don't trigger the downstream project as expected.

That being said maybe you'll have more luck. Let me know if it works for you.

(Someone else has asked a similar question Jenkins multi-branch pipeline and specifying upstream projects )

Morpheme

Absolutely correct. The new version of the job plugin, starting at version 2.4 of the Pipeline: Job plugin, this use case it now supported. I am already using it. – Harts

Works great for me, thank you very much for posting this! On a side note, I had to go and enable the SUCCESS staticField in the "In-process Script Approval" menu in "Manage Jenkins", but after doing that it worked flawlessly. Thanks! – Entoil

Answer 2

If you're using a declarative multi-branch pipeline, you can use:

triggers {
  upstream(upstreamProjects: "some_project/some_branch", threshold: hudson.model.Result.SUCCESS)
}

If you wish for branch matching to occur across dependencies you can use:

triggers {
  upstream(upstreamProjects: "some_project/" + env.BRANCH_NAME.replaceAll("/", "%2F"), threshold: hudson.model.Result.SUCCESS)
}

Note: if you use "organizations" in github, then you must add your organization name as a prefix before the project name so it looks like this:

triggers {
    upstream(upstreamProjects: 'some_organization/some_project/some_branch', threshold: hudson.model.Result.SUCCESS)
}

Note on 2 useless logs: after adding "triggers{upstream(...)" section, in case your "upstreamProjects" is not good (for example you forgot to add organization or did typo in the upstream project name) you will see Jenkins successfully rebuilding your 1 project, but you will not see anything trigger-related in either of 2 logs:

• Jenkins system log: [jenkins-fqdn]/manage/log/all
• Job-related pipeline log: blue ocean -> job -> Artifacts -> pipeline.org available by URL: [jenkins-fqdn]/blue/rest/organizations/jenkins/pipelines/some_organization/pipelines/some_project/branches/master/runs/4/log/?start=0

Only after your "triggers" section is good/working you will see at the end of you upstream project pipeline.log line like this:

[Pipeline] End of Pipeline
Triggering a new build of some_organization » some_project » master #4

GitHub has been notified of this commit’s build result

Finished: SUCCESS

And the corresponding downstream project will have pipeline.log 1st line:

Started by upstream project "some_organization/some_project/some_branch" build number 4

Instead of what you saw before when the build was triggered by github webhooks:

Push event to branch some_branch

Hannon

the syntax for multiple upstreamProjects looks like this (upstreamProjects: "some_project/some_branch, other_project/other_branch", ...) just the projects combined in one string separated within the string with comma according to jenkins.io/doc/book/pipeline/syntax/#triggers – Inanimate

Answer 3

Pipeline Job configuration still natively supports Build Triggers, including reverse build trigger, Build after other projects are built. You can even specify a branch from Pipeline Multi-branch project.

Unfortunately reverse triggering is not available Pipeline Multi-branch jobs. The closest you can get to reverse triggering is by using Promoted Builds Plugin. But it still doesn't let you configure per branch setup.

Additionally Snippet Generator clarifies:

The following variables are currently unavailable inside a Pipeline script:

NODE_LABELS WORKSPACE SCM-specific variables such as SVN_REVISION

ps. Maybe the only way would be to prompt from upstream to downstream.

Frodeen

That is correct. How ever I am looking for a solution for Multi-Branch Pipeline Jobs and in the best case for one that can be configured using the pipeline script. – Harts

@Nitram, did you end up getting upstream triggers to work for your multibranch pipeline projects? I have same job-chaining needs as you, and the triggers { upstream() } are not working for me, using Jenkins 2.249.2, Pipeline: Multibranch v2.22, Pipeline v2.6, Pipeline: Job v2.4. – Seiber

@Seiber I am still using this case, and the upstream() trigger is working fine by now. The only thing you need to look out for, is that you need to replace / with %2F if you want to reference a branch containing slashes. And / outside of the ones in the branch name, must not be replaced. – Harts