Get step id in Jenkins Pipeline for linking to BlueOcean or Pipeline Steps view (flowGraphTable)

Question

Given a Jenkins pipeline that runs a series of steps, some within parallel blocks, is there any way to obtain within the pipeline the Flow id of a given step or the most recent step?

What's a Flow ID? If you look at a Run of your Pipeline job you can see a "Pipeline Steps" link that points to flowGraphTable/. There you have links to specific job steps like execution/node/113/. These seem to represent a FlowNode.

Is there any way to get these IDs from within the pipeline, for generating links etc?

In particular I want to get a link to the sub-Flow for my parallel branches so I can link to the BlueOcean views of them. (The builtin Jenkins view is useless because it doesn't show a subtree).

I can see that the BlueOcean links correspond to the /execution/ links, they have the same id value. If my pipeline branch is myjob/9/execution/node/78/ then on blueocean it'll be jobname/9/pipeline/78.

But how do I get that ID if I want to use the build summary plugin or similar to generate the links and add them to the build results page?

Retinoscope

Answer 1

In particular I want to get a link to the sub-Flow for my parallel branches so I can link to the BlueOcean views of them.

You can get the head flow node (most recent step) of the current thread (aka branch) using CpsThread.current().head.get(). Then you can use FlowNode.iterateEnclosingBlocks() to find the parent branch by checking whether a block start node contains an instance of ThreadNameAction.

Complete pipeline example:

import org.jenkinsci.plugins.workflow.cps.CpsThread
import org.jenkinsci.plugins.workflow.graph.FlowNode
import org.jenkinsci.plugins.workflow.actions.LabelAction
import org.jenkinsci.plugins.workflow.actions.ThreadNameAction

pipeline {
    agent any
    
    stages {
        stage("parallel start"){
            parallel {
                stage("A"){
                    steps{
                        showCurrentBranchUrls()
                        echo "Another step"
                    }
                }
                stage("B"){
                    steps{
                        echo "Some stuff"
                        showCurrentBranchUrls()
                        echo "More stuff"
                    }
                }
            }
        }
    }
}

void showCurrentBranchUrls() {
    // Get the most recent FlowNode of current branch
    FlowNode headNode = CpsThread.current().head.get()
    
    // Find the nearest parent branch
    FlowNode branchNode = getFlowNodeOfParentBranch( headNode )
    if( branchNode ) {
        // Print some useful URLs based on branchNode
        echo "Blue Ocean branch view: ${JENKINS_URL}blue/organizations/jenkins/${JOB_NAME}/detail/${JOB_BASE_NAME}/${BUILD_ID}/pipeline/${branchNode.id}"
        echo "Blue Ocean branch log: ${JENKINS_URL}blue/rest/organizations/jenkins/pipelines/${JOB_NAME}/runs/${BUILD_ID}/nodes/${branchNode.id}/log"
        echo "Pipeline steps: ${JENKINS_URL}${branchNode.url}"
    }
}

// Get FlowNode of parent branch
@NonCPS
FlowNode getFlowNodeOfParentBranch( FlowNode node ) {
    node.iterateEnclosingBlocks().find{ enclosing ->
        enclosing != null && 
        enclosing.getAction( LabelAction.class ) != null && 
        enclosing.getAction( ThreadNameAction.class ) != null
    }
}

In a sandboxed pipeline script the code may trigger some security errors, so I recommend to put it into a shared library which doesn't have such restrictions.

---

In case you want to find the node ID of another branch instead of the current branch, the class PipelineNodeGraphVisitor comes in handy. There are already many examples on SO, e. g. this one of mine.

---

For further reading, here is a good overview about working with the Jenkins Flow Graph.

Deiform

Answer 2

I was struggling with the a similar use case and managed to find a solution that is working for me. This https://issues.jenkins-ci.org/browse/JENKINS-28119 might be an interesting read on the issue. It eventually pointed me in a good direction.

Besides the Pipeline and Pipeline Stage View Plugin, I had to install the HTTP Request Plugin (https://wiki.jenkins.io/display/JENKINS/HTTP+Request+Plugin) and the Pipeline Utility Steps Plugin (for parsing JSON, https://wiki.jenkins.io/display/JENKINS/Pipeline+Utility+Steps+Plugin) on our Jenkins server. I am not sure which other plugins might be required.

Here is my working example, only missing the stages being evaluated:

#!groovy

pipeline {
    agent any

    stages {
        stage('Test') {
            steps {
                script {
                    def responseRun = httpRequest(
                        //consoleLogResponseBody: true,
                        contentType: 'APPLICATION_JSON',
                        httpMode: 'GET',
                        url: BUILD_URL + 'wfapi',
                        validResponseCodes: '200'
                    )
                    def runJson = readJSON text: responseRun.getContent()
                    def headNodeUrl = ''
                    runJson.stages.each {
                        if (it.name.toString() == 'Stage node label') {
                            // Found head node: it.id
                            headNodeUrl = BUILD_URL + 'execution/node/' + it.id.toString() + '/'
                        }
                    }
                    def responseNode = httpRequest(
                        contentType: 'APPLICATION_JSON',
                        httpMode: 'GET',
                        url: headNodeUrl + 'wfapi',
                        validResponseCodes: '200'
                    )
                    def nodeJson = readJSON text: responseNode.getContent()
                    def execNodeUrl = ''
                    nodeJson.stageFlowNodes.each {
                        if (it.name.toString() == 'Execution node label') {
                            // Found execution node: it.id
                            execNodeUrl = BUILD_URL + 'execution/node/' + it.id.toString() + '/log/'
                        }
                    }
                    echo execNodeUrl
                }
            }
        }
    }
}

BUILD_URL is a global environment variable, supplied by Jenkins, I assume. In my full script I have a stage('Stage node label') { ... } containing a statement bat label: 'Execution node label', script: ... whose log URL will be constructed and printed with echo.

The result is a URL like http://myjenkinsserver.org:8080/job/some_folder/job/my_job_name/181/execution/node/50/log/

I think the use of each in my example might not be ideal, since I cannot abort it after the first match. Also I didn't manage to encapsulate the httpRequest and readJSON into a class method or something because I couldn't figure out the return type of readJSON. Any hints are appreciated.

I hope this helps.

Cheers

Carabao

Answer 3

This gets the workspace link for the enclosing node step. You can get the Id by using .getId() instead of .url This works for my case when using node { .... }, but may require some polishing for using in declarative or inside parallel steps.

def getNodeWsUrl(flowNode = null) {
    if(!flowNode) {
        flowNode = getContext(org.jenkinsci.plugins.workflow.graph.FlowNode)
    }
    if(flowNode instanceof org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode && flowNode.typeFunctionName == 'node') {
        // Could also check flowNode.typeDisplayFunction == 'Allocate node : Start'
        return "/${flowNode.url}ws/"
    }

    return flowNode.parents.findResult { getNodeWsUrl(it) }
}

Davie

wow, that thing is a deep structure, it seems it needs to recurse a couple of dozen times before it hits a usable node – Rosado

In fact, I hit recursion limits, so this needs to be done right after a node instruction I believe. – Rosado

Answer 4

I use some hack. Add to sh step for which I would like get Flow ID some output. Then find in log this string and get Flow ID from log. Where ${e} is index for cycle for make output unique.

sh "echo "Run TESTENV=TestEnv_${e}"

Then at the end of pipeline, for generate URL, get the Flow ID from logs. Initial log Index first once.

node('master'){
   logIndex=getLogMap()

Then in cycle get URL, where e is index in cycle:

   stepURL=getURL(logIndex, "Run TESTENV=TestEnv_${e} ")
}

Functions which you can put in shared lib:

import jenkins.branch.NameMangler
import groovy.transform.Field

@Field def keyString = 'Run TESTENV=TestEnv'

def getLogMap() {
  def tokens = "${env.JOB_NAME}".tokenize('/')
  def repo = tokens[tokens.size()-2]
  try {
    def i
    def result=[:]
    exec = """
      set +x
      LOG_FILE="\$JENKINS_HOME/jobs/${repo}/branches/${NameMangler.apply(env.BRANCH_NAME)}/builds/\$BUILD_ID/log"
      LOG_INDEX_FILE="\$JENKINS_HOME/jobs/${repo}/branches/${NameMangler.apply(env.BRANCH_NAME)}/builds/\$BUILD_ID/log-index"
      LOG_LINES=\$(grep --byte-offset --text "${keyString}" "\$LOG_FILE"| sed "s/[^[:print:]\t]//g; s/\\(^[0-9]*:\\).*=\\(.*\\)/\\1 \\2/g; s/'\$//g")
      LOG_INDEX=\$(grep '.* .*' "\$LOG_INDEX_FILE")
      while read -r line ; do
          offset=\$(echo \$line | cut -d ":" -f1)
          str=\$(echo \$line | cut -d " " -f2)
          if [[ "X\$offset" == "X" ]]; then
            echo "Offset if empty in line=\$line"
            continue
          fi
          index=\$(echo "\$LOG_INDEX" | awk '\$1 > '\$offset' { print  prev; exit; } { prev = \$2 }')
          echo "\$str \$index"
      done <<< "\$LOG_LINES" | uniq
      """
    return sh(script: exec, returnStdout: true).trim().tokenize('\n')
  } catch (error) {
    throw (error)
  }
}

def getURL(logIndex, findString) {
  findString=findString.replaceAll(".*=", "")
  resultStr=logIndex.findAll { it.contains(findString) }
  def result=''
  for (s in resultStr){
    i=s.tokenize()
    result = result + "[log|${env.BUILD_URL}execution/node/" + i[1] + '/log/] '
  }
  return result.trim()
}

Freefloating

Answer 5

Instead of getting Step Id for parallel job you can echo or specify the node name that can work.

Lith

Can that then be used to construct URLs referring to nodes in BlueOcean display of the job, or to the flow graph table / pipeline steps view though? if so, how? – Retinoscope