Jenkins - abort running build if new one is started

Question

I use Jenkins and Multibranch Pipeline. I have a job for each active git branch. New build is triggered by push in git repository. What I want is to abort running builds in current branch if new one appears in same branch.

For example: I commit and push to branch feature1. Then BUILD_1 started in Jenkins. I make another commit and push to branch feature1 while BUILD_1 is still running. I want BUILD_1 to be aborted and to start BUILD_2.

I tried to use stage concurrency=x option, and stage-lock-milestone feature, but didn't manage to solve my problem.

Also I've read this thread Stopping Jenkins job in case newer one is started, but there is no solution for my problem.

Do you know any solution to this?

Sharl

We let the current job finish, and them we have some cases where we let the jobs in queue be cleaned-up if we have never ones (as suggested in the referenced question.) Don't like the idea of aborting already started jobs. – Katrinka

@Katrinka For situations such as automated testing of git branches, there is frequently little benefit to completing a test on a branch if the branch has been updated as the updates will need to be tested as well. The obvious solution is to abort the earlier test. Cleanup may still need to be done, but resources aren't wasted completing an unnecessary test. – Tend

Answer 1

With Jenkins script security many of the solutions here become difficult since they are using non-whitelisted methods.

With these milestone steps at the start of the Jenkinsfile, this is working for me:

def buildNumber = env.BUILD_NUMBER as int
if (buildNumber > 1) milestone(buildNumber - 1)
milestone(buildNumber)

The result here would be:

• Build 1 runs and creates milestone 1
• While build 1 is running, build 2 fires. It has milestone 1 and milestone 2. It passes milestone 1, which causes build #1 to abort.

Brainstorm

Milestones are definitely the way to go with a multibranch declarative pipeline project. – Stretchy

JENKINS-43353 proposes making this official. – Turncoat

are milestones branch specific? – Calamitous

@Calamitous I cannot give you any documentation on this, but speaking from testing & experience - yes they are branch specific (the are not cancelling one another across branches at least not on my setup) – Mease

@LucasCarnevalli that is true -- make sure the above milestone code is one of the first things defined in your Jenkinsfile. It does not require a 'node', so in theory you should be able to run this code before anything else runs. If your job is failing this early in the job due to a failed import or something like that you probably have bigger problems to sort out :) – Brainstorm

@brandonsquizzato - where does this go in the Jenkinsfile? – Carmelcarmela

Super cool feature. Thanks for this. Also works with scripted pipeline. – Peddler

What is the reason for the line if(buildNum>1) milestone(buildNum - 1)? Wouldn't jus the milestone(buildNum) suffice ? – Autoxidation

Per doc, Once a build passes the milestone, it will never be aborted by a newer build that didn't pass the milestone yet., which means, build :1 which already has milestone:1 will not be aborted by build:2 as build:2 has not passed milestone:1 ahead of build:1, which contradicts this explanation. Am I reading the doc wrong? – Dominga

Also, per my understanding, if another build has already crossed a given milestone, it is the call to milestone that results in an AbortException be thrown, but once the build goes past the milestone call (i.e., it already successfully obtained the milestone), the milestone step no longer has an opportunity to raise that exception and cause the build to be aborted. Is my understanding wrong? – Dominga

I experimented this solution and it worked as expected. Then I reread the doc and figured that the right explanation for this solution comes from this point: When a build passes a milestone, any older build that passed the previous milestone but not this one is aborted.. Since build:2 passed milestone:2 but build:1 didn't, it will be aborted, and the way this happens is by causing whatever the step that the build:1 is executing to terminate with a FlowInterruptedException consisting of a cause of type org.jenkinsci.plugins.pipeline.milestone.CancelledCause. – Dominga

Answer 2

enable job parallel run for your project with Execute concurrent builds if necessary

use execute system groovy script as a first build step:

import hudson.model.Result
import jenkins.model.CauseOfInterruption

//iterate through current project runs
build.getProject()._getRuns().iterator().each{ run ->
  def exec = run.getExecutor()
  //if the run is not a current build and it has executor (running) then stop it
  if( run!=build && exec!=null ){
    //prepare the cause of interruption
    def cause = { "interrupted by build #${build.getId()}" as String } as CauseOfInterruption 
    exec.interrupt(Result.ABORTED, cause)
  }
}

and in the interrupted job(s) there will be a log:

Build was aborted
interrupted by build #12
Finished: ABORTED 

Waybill

Sounds very good ! Currently looking for a way to port it to a pipeline file scm commited – Unhurried

The system groovy script runs inside the Jenkins master's JVM, that's why it can access everything in jenkins. But pipeline runs in a forked JVM, on the slave where the build is run - I did not find it in docs, but quite sure it is. – Waybill

i've found this: #33532368 / i have no pipeline to try now but you can try to get current build in pipeline using this expression: def build = currentBuild.rawBuild – Waybill

Got the code to work, but curiously, _getRuns only ever list the current running build :/ – Unhurried

what is the value of currentBuild.rawBuild.getClass() ? – Waybill

class org.jenkinsci.plugins.workflow.job.WorkflowRun – Unhurried

Thanks to that question I looked at the correct javadoc and got a working solution ^^ – Unhurried

For anybody who, like me, got to this answer and has trouble making the code run - remove the id from the closure. basically change line: build.getProject()._getRuns().each{id,run-> into build.getProject()._getRuns().each{ run -> – Rhamnaceous

It gives me this error even after defining build. groovy.lang.MissingPropertyException: No such property: build for class: groovy.lang.Binding at groovy.lang.Binding.getVariable(Binding.java:63) at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:242) – Merchantman

it will not work in sandbox. execute system groovy script – Waybill

Also this approach would seem to be risky in regards to master branch. If you work in a team, you merged to master, your team mate merged 1 min later. your job would be canceled and his would run. – Sladen

system groovy seems to be disabled in recent jenkins releases..... i dont think this is still possible – Accustom

@Waybill What is meant with 'will not work in sandbox'? – Mihrab

I was not able to run it with "script sandbox security": wiki.jenkins.io/display/JENKINS/Script+Security+Plugin – Waybill

I'm not sure this will work unless the build is being run on an executor. In my case, I need this to trigger when the build is QUEUED. Is that possible? This is because we have so many parallel PRs that are rebuilt upon a merge into Develop, that many are queued and stay queued because stale branches are being built – Peruse

remove queue and make builds run in parallel, so last build will stop the previous. – Waybill

Will that cancel downstream projects as well? – Kirimia

For anyone else, we had to previousBuild.getListener().getLogger().println("abort reason") rather than specify custom cause to prevent builds from resuming on a restart – Am‚lie

Answer 3

From Jenkins workflow-job plugin version 2.42 you can simply do

// as a step in a scripted pipeline    
properties([disableConcurrentBuilds(abortPrevious: true)]) 
// as a directive in a declarative pipeline
options { disableConcurrentBuilds abortPrevious: true } 

Found solution in comments here https://issues.jenkins.io/browse/JENKINS-43353

Ledezma

Sorry, but is that release even out yet? I find the latest to be 2.327... – Abecedary

Got it, you're talking about the plugin plugins.jenkins.io/workflow-job/#releases – Abecedary

42 < 327 this trips me up all the time. It should be 042. – Suki

Does this work to allow concurrent builds across branches/PRs but terminate old ongoing build jobs for a given branch/PR? – Spectacles

The answer is perfectly working. But in my case, the old build takes some time to abort, So the new build automatically takes a second workplace i.e "xxxxxx@2". since I am hardcoding the workspace path so the new build is getting failed. is there any workaround for this? – Wilmerwilmette

works like a charm... simple and straight forward :) – Pebble

Answer 4

If anybody needs it in Jenkins Pipeline Multibranch, it can be done in Jenkinsfile like this:

def abortPreviousRunningBuilds() {
  def hi = Hudson.instance
  def pname = env.JOB_NAME.split('/')[0]

  hi.getItem(pname).getItem(env.JOB_BASE_NAME).getBuilds().each{ build ->
    def exec = build.getExecutor()

    if (build.number != currentBuild.number && exec != null) {
      exec.interrupt(
        Result.ABORTED,
        new CauseOfInterruption.UserInterruption(
          "Aborted by #${currentBuild.number}"
        )
      )
      println("Aborted previous running build #${build.number}")
    } else {
      println("Build is not running or is current build, not aborting - #${build.number}")
    }
  }
}

Galibi

Maybe it is worth checking that the build number is lower than the current. Otherwise, you might kill even newer builds. – Epirus

Answer 5

Based on the idea by @C4stor I have made this improved version... I find it more readable from @daggett 's version

import hudson.model.Result
import hudson.model.Run
import jenkins.model.CauseOfInterruption.UserInterruption

def abortPreviousBuilds() {
    Run previousBuild = currentBuild.rawBuild.getPreviousBuildInProgress()

    while (previousBuild != null) {
        if (previousBuild.isInProgress()) {
            def executor = previousBuild.getExecutor()
            if (executor != null) {
                echo ">> Aborting older build #${previousBuild.number}"
                executor.interrupt(Result.ABORTED, new UserInterruption(
                    "Aborted by newer build #${currentBuild.number}"
                ))
            }
        }

        previousBuild = previousBuild.getPreviousBuildInProgress()
    }
}

Scarab

This solved the problem in my pipeline script. The "Aborting older build" message is being displayed, but the "Aborted by newer build" isn't. Maybe it is because my older build was waiting for a input action. – Subauricular

@Subauricular Could be. Also just in case it's not obvious: the "Aborted by newer build" message is displayed on the other (older) build. – Scarab

This approach is using static methods. So I'm geting this error: Scripts not permitted to use staticMethod hudson.model.Hudson getInstance – Jankell

@DmitryKuzmenko Maybe you're running the script inside the sandbox? It wouldn't work there. Also, this is from 2018, maybe there are differences in newer versions. – Scarab

Where does "currentBuild" come from? We have a lot of custom code I'm struggling to parse in our jenkins setup and I'm not sure what type that object is. – Taciturnity

Answer 6

Got it to work by having the following script in the Global Shared Library :

import hudson.model.Result
import jenkins.model.CauseOfInterruption.UserInterruption

def killOldBuilds() {
  while(currentBuild.rawBuild.getPreviousBuildInProgress() != null) {
    currentBuild.rawBuild.getPreviousBuildInProgress().doKill()
  }
}

And calling it in my pipeline :

@Library('librayName')
def pipeline = new killOldBuilds()
[...] 
stage 'purge'
pipeline.killOldBuilds()

Edit : Depending on how strongly you want to kill the oldBuild, you can use doStop(), doTerm() or doKill() !

Unhurried

Is there any way to send a message to the terminated build?It sends this hard kill signal but no log on who killed it. – Merchantman

I wouldn't know, we're living with this full gray lines for the moment, good enough for us ^^' – Unhurried

The order from graceful to destructive goes doStop() -> doTerm() -> doKill() – Degression

How did this ever work for you? it's wrong :) But thanks for the idea... I have a working version... see my answer – Scarab

Well, it's working in our production stack right now, so I don't think it's wrong. The fact you were unable to use the code as it can come from a lot factors, including jenkins version, java version, os used, file permissions in use.... – Unhurried

@TahaTariq the solution https://mcmap.net/q/258478/-jenkins-abort-running-build-if-new-one-is-started prints a message in older jobs. Unfortunately it looks it doesn't work if your other running job is waiting for a input. – Subauricular

Answer 7

Adding to Brandon Squizzato's answer. If builds are sometimes skipped the milestone mechanism as mentioned will fail. Setting older milestones in a for-loop solves this.

Also make sure you don't have disableConcurrentBuilds in your options. Otherwise the pipeline doesn't get to the milestone step and this won't work.

def buildNumber = env.BUILD_NUMBER as int
for (int i = 1; i < buildNumber; i++)
{
    milestone(i)
}
milestone(buildNumber)

Infeasible

The potential problem with this is that when you have a large number of builds, creating that many milestones can take up a considerable amount of time. I don't know exactly at what point things changed -- creating many milestones used to go for quickly for me. Then more recently, creating a milestone was taking about half a second each -- obviously not ideal if you are on build #900. So I created my solution that does not use a for loop. – Brainstorm

Answer 8

Based on @daggett method. If you want to abort running build when the new push is coming and before fetch updates.
1. Enable Execute concurrent builds if necessary
2. Enable Prepare an environment for the run
3. Running bellow code in Groovy Script or Evaluated Groovy script

import hudson.model.Result
import hudson.model.Run
import jenkins.model.CauseOfInterruption

//def abortPreviousBuilds() {
    Run previousBuild = currentBuild.getPreviousBuildInProgress()

    while (previousBuild != null) {
        if (previousBuild.isInProgress()) {
            def executor = previousBuild.getExecutor()
            if (executor != null) {
                println ">> Aborting older build #${previousBuild.number}"
                def cause = { "interrupted by build #${currentBuild.getId()}" as String } as CauseOfInterruption 
                executor.interrupt(Result.ABORTED, cause)
            }
        }
        previousBuild = previousBuild.getPreviousBuildInProgress()
    }
//}

Madigan

Answer 9

Before each build task, first determine whether all the tasks currently under construction are the same as the branch of this build. If they are the same, keep the latest task.

    stage('Setup') {
        steps {
            script {
                JOB_NAME = env.JOB_NAME
                branch_name = "${env.gitlabTargetBranch}"
                def job = Jenkins.instance.getItemByFullName( JOB_NAME )
                def builds = job.getBuilds()
                for( build in job.getBuilds()) {
                    if (build.isBuilding()) {
                      String parameters = build?.actions.find{ it instanceof ParametersAction }?.parameters?.collectEntries {
                            [ it.name, it.value ]
                            }.collect { k, v -> "${v}" }.join('\n')
                            
                             if (branch_name == "${parameters}") {
                                 if (env.BUILD_NUMBER > "${build.getId()}") {
                                     build.doKill()  
                                 }
                             }
                    }
                }
                
                
            }
           ......

Estabrook

Answer 10

I also compiled a version from the previously given ones with a few minor tweaks:

• the while() loop generated multiple outputs for each build
• the UserInterruption currently expects a userId instead of a reasoning string, and will not show a reasoning string anywhere. Therefore this just provides the userId

def killOldBuilds(userAborting) {
    def killedBuilds = []
    while(currentBuild.rawBuild.getPreviousBuildInProgress() != null) {
        def build = currentBuild.rawBuild.getPreviousBuildInProgress()
        def exec = build.getExecutor()

        if (build.number != currentBuild.number && exec != null && !killedBuilds.contains(build.number)) {
            exec.interrupt(
                    Result.ABORTED,
                    // The line below actually requires a userId, and doesn't output this text anywhere
                    new CauseOfInterruption.UserInterruption(
                            "${userAborting}"
                    )
            )
            println("Aborted previous running build #${build.number}")
            killedBuilds.add(build.number)
        }
    }
}

Gona