How to schedules jobs with specific parameters in a Jenkins multibranch pipeline
Asked Answered
M

4

12

We were having 2 FreeStyle projects on Jenkins:

One to generate builds(daily builds+manual builds), one other to execute tests.

We are moving to a Multibranch pipeline on jenkins, so my understanding is that we have one project per repository, and that we should use options to have different behavior.

So I can create parameters, to indicate if we want to run the tests, if we want to build the setups, that part I'm ok with it.

My issue is that I need that by default, the tests are NOT executed(because they take a lot of time to generate, and I don't want that developers can by mistake just let the "Execute tests" option checked.

And I need that this option is checked when executing the daily build in the night.

So 2 questions:

  1. How to schedule?
  2. How to provide the parameters value used for this schedule?
Mingrelian answered 15/6, 2018 at 6:52 Comment(2)
By "jenkins descriptive configuration file" do you mean Declarative Pipeline?Twain
@VitaliiVitrenko: Okay great, thank you!Mingrelian
T
10

You can create a separate multi-branch job that will run by schedule and trigger your main job overriding all necessary parameters. It will look something like this

pipeline {
    agent any
    triggers {
        pollSCM('0 0 * * *')
    }
    stages {
        stage('Triggering the main job') {
            steps {
                build job: "main/${BRANCH_NAME.replace('/', '%2F')}", 
                      parameters: [string(name: 'RUN_TESTS', value: 'true')]
            }
        }
    }
}

You should put this file along with your main Jenkinsfile in the repository and configure a separate multi-branch pipeline job to use this file.

Twain answered 27/6, 2018 at 21:52 Comment(8)
What kind of job is this?Mingrelian
@J4N, I updated the answer. It's declarative job now.Twain
Great, I've just one question: It's a multibranch pipeline, how will it know which branch to execute? It will execute for each branch?Mingrelian
@Mingrelian you have to specify which branch you want to build periodically i.e. job: 'main/master'. Do you need to build all branches at once?Twain
Currently, it's scheduled to poll every night the SCM to build all the branches that have changes. It's working with the pollSCM trigger.Mingrelian
@Mingrelian then you can use pollSCM in the scheduled job as well and trigger your main job like this "main/${BRANCH_NAME}". See updated answer.Twain
main being? The name of the second job? Will this work with branch like bugfix/my-bug-branch(with additional /)Mingrelian
@Mingrelian yes, main is your main job that executes build, tests, etc. In order to make it working for that kind of branches you have to replace slashes with %2F. Like this main/${BRANCH_NAME.replace('/', '%2F')}Twain
B
5

To keep this in the same job is going to require a little bit of groovy coding. Since your using MultiBranch pipeline this can all live in your Jenkinsfile

First, Set your cron as Vitalii mentions, this will kick of the job on schedule.

properties([
    pipelineTriggers([cron('0 0 * * *')])
])

Next, when this job is triggered by the schedule, we want to adjust the params its running with. So first we need to check what caused the build. This will probably require Security script approval.

List causes = currentBuild.rawBuild.getCauses().collect { it.getClass().getCanonicalName().tokenize('.').last() }

If this contains 'TimerTriggerCause' then we want to update the parameter.

if (causes.contains('TimerTriggerCause') { 
    setBooleanParam("EXECUTE_TESTS", true)
}

We wrote a function in a shared lib for this, you can put it in the same Jenkinsfile if you wish (At the bottom outside the pipeline logic):

/**
 * Change boolean param value during build
 *
 * @param paramName new or existing param name
 * @param paramValue param value
 * @return nothing
 */
Void setBooleanParam(String paramName, Boolean paramValue) {
    List<ParameterValue> newParams = new ArrayList<>();
    newParams.add(new BooleanParameterValue(paramName, paramValue))
    try {
        $build().addOrReplaceAction($build().getAction(ParametersAction.class).createUpdated(newParams))
    } catch (err) {
        $build().addOrReplaceAction(new ParametersAction(newParams))
    }
}

And let the job continue as normal. When it gets to evaluating params.EXECUTE_TESTS, this will now be true (instead of the default false).

Note: May need to import model for the value

import hudson.model.BooleanParameterValue

Putting that all together (Just cobbled the bits together quickly for an overall picture), Your jenkinsfile would end up something like this

#!groovy
import hudson.model.BooleanParameterValue


List paramsList = [
    choice(name: 'ACCOUNT_NAME', choices: ['account1', 'account2'].join('\n'), description: 'A choice param'),
    string(name: 'PARAM', defaultValue: 'something', description: 'A string param'),
    booleanParam(defaultValue: false, name: 'EXECUTE_TESTS', description: 'Checkbox'),
]

properties([
    buildDiscarder(logRotator(numToKeepStr: '20')),
    pipelineTriggers([cron('0 18 * * *')]), // 4am AEST/5am AEDT
    disableConcurrentBuilds(),
    parameters(paramList)
])


ansiColor {
    timestamps {
        node {
            try {
                causes = currentBuild.rawBuild.getCauses().collect { it.getClass().getCanonicalName().tokenize('.').last() }
                if (causes.contains('TimerTriggerCause') { 
                    setBooleanParam("EXECUTE_TESTS", true)
                }
                stage('Do the thing') {
                    // Normal do the things like build
                }
                stage('Execute tests if selected') {
                    if (params.EXECUTE_TESTS == true) {
                        // execute tests
                    } else {
                        echo('Tests not executed (Option was not selected/False)')
                    }
                }
            } catch (err) {
                throw err
            } finally {
                deleteDir()
            }
        }
    }
}

/**
 * Change boolean param value during build
 *
 * @param paramName new or existing param name
 * @param paramValue param value
 * @return nothing
 */
Void setBooleanParam(String paramName, Boolean paramValue) {
    List<ParameterValue> newParams = new ArrayList<>();
    newParams.add(new BooleanParameterValue(paramName, paramValue))
    try {
        $build().addOrReplaceAction($build().getAction(ParametersAction.class).createUpdated(newParams))
    } catch (err) {
        $build().addOrReplaceAction(new ParametersAction(newParams))
    }
}
Boak answered 28/6, 2018 at 3:17 Comment(3)
I'm using Descriptive Jenkinsfile, not scripted ones.Mingrelian
Ah sorry, didn't pick that up. This may be too advanced for declarative to handle. Its still possible but you would need to nest what I have done in script{} blocks.Boak
This answer really looks good. Knowing that you can embed the scripted parts in script {} blocks, it should be rather easy to work out the solution.Homothermal
U
0

Allow me to suggest a simpler approach. This is for declarative pipelines.

I would recommend keeping the tests (treated as first class code along with the application code) and application source code in same repository.

When Jenkins checks out from your SCM keeping them together in the one repository will allow you to apply a tag (label) when the test suite passes. Labels are your best friend and should be applied on success whenever possible. At time of writing multi-SCM checkouts don't seem to support application of labels, unfortunately.

I currently have a team of 5 developers and am currently using a multibranch pipeline to handle all the feature branches they are generating. We also have a 'master' and 'integration' branch. Master is for clean releases. Integration is my key branch.

Scheduling in a declarative pipeline is as simple as:

triggers { 
    cron('0 22 * * *') 
}

pollSCM doesn't seem to work as reliably for more complex crontabs.

One way you might like to consider introducing conditionality into your declarative pipeline is via branchname.

success {
    script {
        if( "${env.BRANCH_NAME}" == "integration" ) {
           //Create Package
           //Upload to Artifactory
           //Apply tag to Git
        }
    }
}

Feature branches in the above example are only executing the unit tests and providing feedback to the developers. It is only the integration branch that additionally on success generates an artifact (for later testing phases) and tags the repository.

If you don't want bifurcate behavior based on branch, I would suggest you have 2 different jobs (pipelines) defined in Jenkins: one to run every commit from the developers during the day and one that is scheduled to run nightly that executes your long running tests.

This is what I do with Unit tests and systems test.

Unit Test job is a multibranch pipeline that runs for every branch in the Enterprise GitHub repository. Polls every minute for changes with only the Integration branch creating the artifact and tagging. Unit tests take 10 minutes to run.

The System Test job is a simple pipeline that is scheduled to run nightly and it takes an hour or so to execute.

Unconscionable answered 29/6, 2018 at 2:45 Comment(3)
Not sure to understand, I'm willing to run my tests for pull request before their integration, so that would not work. Also, my developer are also willing to build setup from the integration branchMingrelian
@J4N, Trying to make one pipeline do everything makes things far too complex. This approach is simpler and works perfectly well. I am using it right now for a large corporate client.Unconscionable
Good for you, but I'm telling you, We need to be able to run tests for all branches, not only for integration branch. We have pull request and we want to have them validated by the tests.Mingrelian
B
0

This doesn't map so well into multibranch pipeline. I've seen stages dependent on a branch, but not stages dependent on the parameter - it would break the stage view daily as well.

What I would recommend is to write two separate Jenkinsfiles, for example one called Jenkinsfile and second maybe Jenkinsnightlyfile. Then you can create two multibranch pipeline projects just using different jenkinsfile names on the same repository.

Put your regular stages in the first one, and all the tests in the other (you can divide the work here into multiple stages as well for clarity), be sure to have the other one use the appropriate triggers like for jenkins-pipeline:

properties([
    pipelineTriggers([cron('0 0 * * *')])
])

or for declarative pipeline:

triggers { 
    cron('0 0 * * *') 
}
Batts answered 2/7, 2018 at 13:21 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.