Jenkins: Dynamic Job creation throws "Pipeline CPS Method Mismatches " error

Question

I am trying to create multiple dynamic jobs from within one of the pipeline job that should run in parallel. I want my jenkins pipeline script to download and install my software binary based on user input. Below are my sample stages:

1. stage 1: will download the build
2. stage 2: take parameter and install my "cloud" portion of my software
3. stage 3: will take user input "how many edges need to deploy" (wherein edges are my 2nd component of the build) and create dynamic jobs based on number of edges. If edges count is 2 then 2 jobs shall be created at runtime. If edges is 3 then 3 jobs shall be created at runtime

Below are my sample script that I have managed to create

def SETUP_DIR = ""
Map<String, ?> myMap = [:]
Map<String, List<String>> minionMap = [:]

pipeline {
    agent any

    parameters {
        string(name: 'RELEASE_VERSION', defaultValue: '1.0.0', description: 'Define build to use')
        string(name: 'Cloud_IP', defaultValue: 'xxx.xx.x.xx', description: 'Central IP')
        choice(name: 'No_Of_Edges', choices: ['1', '2', '3', '4', '5'], description: 'Total Edges to Deploy')
 
    }

    stages {
        stage('Identify Installation Parameters') {
            steps {
                script {
                    // Retrieve the parameter values
                    def RELEASE_VERSION = params.RELEASE_VERSION
                    def Cloud_IP = params.Cloud_IP
                    def No_Of_Edges = params.No_Of_Edges
                   
                }
            }
        }

        stage('Download Build') {
            steps {
                script {
                    def dirname = sh(returnStdout: true, script: 'date +"%d-%m-%Y_%H:%M:%S"').trim()
                    sh """
                    sudo mkdir -p /root/Automation/${dirname}
                    echo $dirname
                    SETUP_DIR=$dirname
                    cd /root/Automation/${SETUP_DIR}/
                    VER=${RELEASE_VERSION}
                    echo \${VER}
                    sudo curl -u cicd:cicd -X GET url\${VER}.tar.gz --output installer.tar.gz
                    sleep 60
                    sudo tar -vxf installer.tar.gz
                    sleep 30
                    sudo rm -rf installer.tar.gz
                    sleep 5
                    sudo chmod 777 installer
                    cd installer
                    <some-tasks>
                    """
                }
            }
        }

        stage('Install Cloud') {
            steps {
                script {
                    echo "Installing IEAP Central"
                    printf "\n \n Updating all files \n \n"
                    sh """
                    <some-tasks>

                    printf "\n \n Deployed cloud successfully \n \n"
                    """
                }

            }
        }

        stage('Deploy Edge Cluster') {
            steps {
                script {
                    echo "Installing Edge Control"
        
                    def loopCount = No_Of_Edges.toInteger()
        
                    for (int i = 1; i <= loopCount; i++) {
                        def paramInput = input(
                            id: "paramInput-${i}",
                            message: "Enter Edge${i} Value",
                            parameters: [
                                string(name: "Edge_Control_IP_${i}", defaultValue: '10.139.9.178', description: "Edge Control IP"),
                               
                            ]
                        )
                        def jobName = "Dynamic-Job-${i}"
                        def jenkinsfileContent = generateJenkinsfileContent(jobName, paramInput)
        
                        // Create dynamic pipeline job
                        createPipelineJob(jobName, jenkinsfileContent)
                    }
                }
            }
        }
    }
}

def generateJenkinsfileContent(String jobName, Map<String, String> params) {
    def edgeControlIp = params["Edge_Control_IP"]
    return """
        pipeline {
            agent any

            stages {
                stage('Dynamic Job') {
                    steps {
                        script {
                            echo "This is the dynamic job ${jobName}"
                            echo "Edge Control IP: ${edgeControlIp}"
                           
                        }
                    }
                }
            }
        }
    """
}

@NonCPS
def createPipelineJob(String jobName, String jenkinsfileContent) {
    def jobFolder = "Installation" // Folder to store dynamic jobs
    def jobFullName = "${jobFolder}/${jobName}" // Full name of the dynamic job

    // Create or update the Jenkinsfile for the dynamic job
    writeFile file: "${jobFullName}/Jenkinsfile", text: jenkinsfileContent

    // Trigger the dynamic job by loading the Jenkinsfile
    build job: "${jobFullName}"
}

Output received:

expected to call WorkflowScript.createPipelineJob but wound up catching writeFile; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/

Ludwick

Any suggestion is appreciated – Ludwick

Where is this method? WorkflowScript.createPipelineJob(jobFullName) – External

Updated the last line. Please check now @External – Ludwick

I do not see a reason of using @NonCPS for createPipelineJob. You are just using variables and regular steps inside. What is the output when you run without @NonCPS ? – External

In that case it does not try to create new job. It throws error that particular job does not exists----- No item named Installation/Dynamic-Job-1 found – Ludwick

Ok, so we just solved your CPS error problem. Now there is issue with job creation. I think it is just the way you try to create job. You should use jenkinscli instead. In your jenkins installation navigate to: your-jenkins-installation/cli/command/create-job to see the example. Other resource: jenkins.io/doc/book/managing/cli In general it is about downloading jar file in your pipeline and running it to achieve some goal - create job in your case. – External

Thanks @External Do you have any sample reference to it? I tried the certain approach but not able to create dynamic job. – Ludwick

You just need to follow the documentation. Which approach did you try? You gave no examples. Here is the simple example of copying existing job using jenkinsCli: def response = sh(script: "curl.exe -O http://localhost:8080/jnlpJars/jenkins-cli.jar", returnStdout: true) sh "java -jar jenkins-cli.jar -s http://localhost:8080/ -auth user:pass get-job existingJob > existingJob_downloaded" sh "java -jar jenkins-cli.jar -s http://localhost:8080/ -auth user:pass -webSocket create-job GeneratedJob<existingJob_downloaded" – External

Answer 1

I put the example here instead of the comment for better clarity. This is how can one use jenkins cli in the pipeline: downloading cli, downloading existing job and sending specification back to Jenkins to create the same job with different name. This is good starting point to modify specification xml in order to create desired job or do other needed stuff.

def response = sh(script: "curl.exe -O http://localhost:8080/jnlpJars/jenkins-cli.jar", returnStdout: true) 
sh "java -jar jenkins-cli.jar -s http://localhost:8080/ -auth user:pass get-job existingJob > existingJob_downloaded" 
sh "java -jar jenkins-cli.jar -s http://localhost:8080/ -auth user:pass -webSocket create-job GeneratedJob<existingJob_downloaded"

External