Access files on a node slave from Jenkins master using Groovy

Question

I am using the Jenkins Build Flow plugin to achieve parallelization. The Groovy DSL does certain file operations. Even though the option Restrict where this project can be run is set to run the job on a specific slave, the DSL runs on master. This is not intended.

Could someone tell me how I can restrict the DSL to run on the specified slave? Even if there is a way we can access the slave file system via the DSL, that should work.

In general, how can we access files on a node slave from Jenkins master using Groovy?

def fp = new hudson.FilePath(build.workspace.channel, "/srv/jenkins/workspace/myworkspace_on_slave_node")
assert fp.exists()      // returns true :)

def ant = new AntBuilder()

if (fp != null) {
  def scanner = ant.fileScanner {    // fails here :(, says /srv/jenkins/workspace/myworkspace_on_slave_node not found
    // grab ALL files requested to be run
    fileset(dir: "$fp", includes: "**/*.java")
  }

  // now lets iterate over - print - and count test files
  int numFiles = 0
  for (f in scanner) {
    println("Found file $f")    
    numFiles++
  }
  println("Total files $numFiles")
}

The workspace is there on the slave node, but the above code is failing when I am trying to open the FileSet to the remote FilePath.

Darr

Answer 1

The Groovy DSL is always executed on master (in tomcats directory). Even if you install Node Label Parameter plugin and set build job to be executed on some specific slave. If you want to get access from Groovy DSL to job workspace on slave you can use channel. There's my example of creating a file in build flow job workspace:

if(build.workspace.isRemote()){
channel = build.workspace.channel
}
String fp = build.workspace.toString() + "\\" + "newfile.txt"
newFile = new hudson.FilePath(channel, fp)
newFile.write("xyz", null)

An easier way is executing file operations in downstream jobs in Execute Groovy script (not in build flow job) running on specific slave. You must have node plugin installed and pass slave name as a parameter in DSL script: build("jobA", paramNode: "nodename")

Meneses

Thanks @jussuper. I have updated the main posting with updates made from you recommendation, and I am still seeing failure. – Darr

Also, since the project type is build flow, I believe I cannot use the Groovy Script plugin there. It requires the Jenkins project to be free style. – Darr

In fact I got mine working extending on this one. Thanks @jussuper. – Darr

This method worked for me as well. I wanted to fetch some content from a file generated in the build and parse it. – Piotr

Using snippet mentioned says 'No such property: build for class'. How shall I make this build variable known? – Orelee

Answer 2

This is an almost drop in replacement for reading file contents from slave in Groovy:

def workspace = build.getEnvVars()["WORKSPACE"];
def nodeChannel = build.workspace.channel;
String svnVersion = new hudson.FilePath(nodeChannel, workspace + '/relative/path/svn-revision').readToString();

Where relative/path/svn-revision is a file created remotely (on the slave).

For local files an equivalent would be: svnVersion = new File(workspace, 'artifacts/svn-revision').text;. So a bit shorter, but not much.

As noted by others -- Groovy scripts always run on master, not on a slave. Hence you have to use hudson.FilePath and use commands that understand FilePath API.

There is also act(FileCallable) which seems to run File commands on a slave, but haven't tried that.

See: https://javadoc.jenkins.io/hudson/FilePath.html

Sigman

Answer 3

The Workflow Plugin "Originally inspired by the Build Flow Plugin" has the following section in its tutorial:

Using slaves

The parameter may be a slave name, or a single label, or even a label expression such as:

  node('unix && 64bit') {
      // as before
  }

The following question in the Build Flow Plugin's comments is unanswered since Jan 27, 2014:

Alexander Uvizhev says:

Is there a way to specify where particular build should run? By providing a label or a node name.

Eccrinology

Does it mean that it cannot be done using build flow? There isn't anything in the Jenkins API that might help? – Darr

@naspras Since you write that Restrict where this project can be run isn't taken into account it seems it cannot be done. (I didn't try it myself to double-check. I don't have slaves here in my configuration.) – Eccrinology

Answer 4

Install NodeLabel parameter plugin. It Provides parameter option Label.

Then you can use this parameter in your DSL script to pass node name or value.

Unstudied

Hi @Mahi, can you please provide an example how it can be done? Say, the parameter is defined as "nodeParam". How do I use it in the DSL? – Darr