Restart Jenkins slave from master

Question

I use jenkins master-slave configuration for capturing Performance metrics of a product. We have observed that jenkins-slave tends to accumulate memory and thus influences the Performance metrics being captured.

To ensure consistency of the metrics being captured, we are thinking of restarting jenkins slave every day from the master, when there are no jobs running on the slave. Is this feasible?

How can we accomplish it?

Note: Using jenkins-slave as a service is not an option because we are having other security access issues with it.

Tanguy

Answer 1

I know this answer is coming in a bit late :

This is how I did the same for the same reasons, not sure if this is the best way to achieve this, but it solved many of our problems :

For Windows Machines :

1. Create a job that simply runs "shutdown -r -f" on windows machines. It will restart the machines.
2. Now bringing it back online part. For similar reasons as yours, I didn't use "jenkins-slave as a service". Instead I configured the nodes to connect via JNLP client, and then added the slave.jar command for each node in Window's task scheduler (to run on startup)
3. Now the job restarts the machine and the Windows machine bring itself online on Jenkins itself right after restart.

For Mac Machines :

1. The process is comparatively easier on mac. First, make a job to run "shutdown -r now" on Mac node

2. The node should simply be setup to get connected via ssh. That will take care of bringing it up online on Jenkins.

This was the "execute shell" part of my script to restart all the machines used for our automation :

distro=`uname`
if [ "$distro" = "Windows_NT" ] || [ "$distro" = "WindowsNT" ] ;then
echo "Restarting Windows Machine...."
shutdown -r -f
else
echo "Restarting Mac Machine...."
sudo shutdown -r now
fi

PS:

It's not exactly related to the question, but may be useful for the situation that you specified. It may be a good idea to add a batch script to clean temp files on startup of Windows machines. Add following to a batch script (Say, cleanTemp.bat) in the startup folder of your Windows machine. (For Windows 10, C:\Users\\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup)

rmdir %temp% /s /q

md %temp%

Disproportion

How did you manage to run the shutdown command on all the slaves simultaneously? Or did you just run the job multiple times until every slave was hit? I have a similar setup (many windows, mac and centos slaves), but I can't figure out how to restart them all at once in an automated way. – Hag

Answer 2

If you still need an answer: https://wiki.apache.org/general/Jenkins#How_do_I_restart_a_Jenkins_Unix_Slave.3F

Although, I just did a disconnect and then I saw that the processes died in the slave. I did not have to kill them manually. Then launch the slave again and that's it.

This is good from web UI. I have not searched for CLI for this yet.

Vincents

I can confirm that this also worked for me. I changed some of the agent's settings and needed to restart it. Thanks for this! – Marlite

The link is dead – Puberty

Answer 3

1. Create a job e.g. "Reboot-Slave", and set it with shell "shutdown -r -t 0", and take the target slave name as a parameter. (in this way, the restart command will be executed directly on the target slave that you want to restart.)

2. Create another job e.g. "Reboot-Check-Slave-Online", in this job, you should call the 1st job and pass the target slave name as parameter, plus, you'd better write some logic to determine whether your slave finished the restarting and connected to Jenkins server again, you can implement it by adding an "Execute system groovy script" step in your job and write below code:

   import hudson.model.*
   
   def target_slave_param = "target_slave"
   def resolver = build.buildVariableResolver
   def target_slave = resolver.resolve(target_slave_param)
   
   println "target_slave is: ${target_slave}"
   
   def status = 0;
   
   //do{
   println "Searching for ${target_slave}";
   slave = Hudson.instance.slaves.find({it.name == target_slave});
   
   if (slave != null)
   {
     computer = slave.getComputer();
     if (computer.isOffline())
    {
       println "Error! $target_slave is offline.";
       status = 1;
     }
     else 
     {
       println "OK: $target_slave is online";
     }
   }
   else 
   {
     println "Slave $target_slave not found!";
     status = 1;
   }
   //}
   

Skirret

Answer 4

Steps:

1. Install Node and Label parameter plugin
2. Check This project is parameterized option:
3. Use following command in Execute shell field:

(sudo bash -c "(sleep 30 && sudo shutdown -r now) &") &

Jenkins job is detached correctly and shows success execution.

Bertolde