Jenkins Parallel Trigger and Wait

Question

I have 4 jobs which needs to be executed in the following sequence

JOB A 
 |------> JOB B
 |------> JOB C 
            |------> JOB D

In the above

1. A should trigger B & C parallely and C inturn triggers D.
2. A should hold the job as running till all 3 of them completed.

I tried the following plugins and couldn't achieve what I am looking for

• Join Plugin
• Multijob Plugin
• Multi-Configuration Project
• Paramterized Trigger Plugin

Is there any plugin which I haven't tried would help me in resolving this. Or is this can be achieved in a different way. Please advise.

Tippler

Answer 1

Use DSL Script with Build Flow plugin.

try this Example for your execution:

   build("job A")

   parallel
   (
      {build("job B")}
      {build("job C")}
   )

   build("job D")

Cown

Answer 2

Try the Locks and Latches plugin.

Serve

Its soon to be deprecated. Anyhow also I couldn't find any documentation on how to make it work? Do you have any? – Tippler

Saw this but even couldn't figure out how to release the lock, etc. – Tippler

Answer 3

This may not be optimal way, but it should work. Use the Parameterized Trigger Plugin. To Job A, add a build step (NOT a Post Build Action) to start both Jobs B and C in the same build step AND block until they finish. In Job C, add a build step (NOT a Post Build Action) that starts Job D AND blocks until it is finished. That should keep Job A running for the full duration.

This isn't really optimal though: Job A is held open waiting for B and C to finish. Then C is held open until D is finished.

Is there some reason that Job A needs to remain running for the duration? Another possibility is to have Job A terminate after B and C are started, but have a Promotion on Job A that will execute your final actions after jobs B, C and D are successful.

Embryectomy

Thanks for your suggestion @Jason. I suppose I already tried this approach. But the problem here is I need to pass 2 different set of parameters to B & C. For B it is on what Node to run (using NodeLabel plugin) and for C set of properties. There is no option in Parameterized Trigger Plugin to trigger 2 jobs at same time with 2 different sets of parameter. :( – Tippler

Answer 4

I am trying to build a same system. I am building a certification pipeline where I need to run packager/build/deploy jobs and and corresponding test jobs. When all of them are successful, I want to aggregate the test results and trigger the release job that can do an automated maven release.

I selected Build pipeline plugin for visualization of the system. Initially tried with Parameterized trigger Plugin with blocking builds. I could not setup archiving the artifacts/fingerprinting and downstream build relationship this way since archiving the artifacts works only in postbuild. Then I put the Parameterized trigger in Post build activity. This way I was able to setup downstream builds, fingerprinting, aggregate test results but the build failures were not bubbling to upstream job chain and upstream jobs were non blocking

I was finally able to achieve this using these plugins-

• Build Pipeline
• MultiJob Plugin
• FingerPrint Plugin
• Copy Artifacts Plugin
• Join Plugin

I'm using Jenkins 1.514

System looks like this

Trigger Job --> build (and deploy) Job (1..n) ---> Test Job (1..n)

Trigger Job -

• Create as MultiJob and create a fingerprint file in shell exec

  echo date +%s > fingerprint.txt

Trick is that file needs to be archived during the build, to do that execute this script-

ARCHIVEDIR=$JENKINS_HOME/jobs/$JOB_NAME/builds/$BUILD_ID/archive
mkdir $ARCHIVEDIR
cp fingerprint.txt $ARCHIVEDIR

• Create MultiJob Phase consisting of build/deploy job.
• Build/deploy job is itself a multijob
• follow the same steps for creating build/deploy job as above relative to fingerprinting.
• Copy the fingerprint.txt artifact from upstream job
• Setup MultiJob phase in deploy job that triggers the test job
• create a new fingerprint file and force archive it similar to above step
• Collect Junit results in the final test job.

In the trigger Job, use Join Plugin to execute the Release Job by choosing 'Run Post Build Actions at join' and execute the release project only on stable build of Trigger Job. This way all the steps are showing up in Build Pipeline view and Trigger job is blocking for all downstream builds to finish and sets its status as the worst downstream build to give a decision point for release job.

Momus

Sounds like an interesting solution for a Continuous Delivery pipeline. Wondering if there isn't a simple way. – Ramie

Answer 5

Multijob Plugin

If you'd like to stop the mess with downstream / upstream jobs chains definitions. Or when you want to add a full hierarchy of Jenkins jobs that will be executed in sequence or in parallel. Add context to your buildflow implementing parameter inheritance from the MultiJob to all its Phases and Jobs. Phases are sequential while jobs inside each Phase are parallel.

https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin

Trappings

This Plugin also can be useful for managing parallel/serial tasks: [wiki.jenkins-ci.org/display/JENKINS/… – Trappings

This can also give some idea :[#4511140 – Trappings