How do I make sure that all my background processes have finished execution before I exit my script (TCL/Bash).
I was thinking of writing all my background process pids to a pidfile. And then at the end pgrep the pidfile to see if any processes are still running before I exit.
Is there some simpler way to do this? And is there a TCL specific way to do this?
If you want to wait for jobs to finish, use wait. This will make the shell wait until all background jobs complete. However, if any of your jobs daemonize themselves, they are no longer children of the shell and wait will have no effect (as far as the shell is concerned, the child is already done. Indeed, when a process daemonizes itself, it does so by terminating and spawning a new process that inherits its role).
#!/bin/sh
{ sleep 5; echo waking up after 5 seconds; } &
{ sleep 1; echo waking up after 1 second; } &
wait
echo all jobs are done!
: command not found while running this simple script: sleep 10 & wait Btw, apart the annoying warning, the wait command works as intended. Any idea about what is causing such warning? - NB: there is a carriage return between sleep 10 & and wait but the forum is stripping them –
Jailbreak & is followed by the next command, and the interpreter will try to look up a program whose name is a carriage return. –
Fermi You can use kill -0 for checking whether a particular pid is running or not.
Assuming, you have list of pid numbers in a file called pid in pwd
while true;
do
if [ -s pid ] ; then
for pid in `cat pid`
do
echo "Checking the $pid"
kill -0 "$pid" 2>/dev/null || sed -i "/^$pid$/d" pid
done
else
echo "All your process completed" ## Do what you want here... here all your pids are in finished stated
break
fi
done
WARNING: Long script ahead.
A while ago, I faced a similar problem: from a Tcl script, launch a number of processes, then wait for all of them to finish. Here is a demo script I wrote to solve this problem.
#!/usr/bin/env tclsh
# Launches many processes and wait for them to finish.
# This script will works on systems that has the ps command such as
# BSD, Linux, and OS X
package require Tclx; # For process-management utilities
proc updatePidList {stat} {
global pidList
global allFinished
# Parse the process ID of the just-finished process
lassign $stat processId howProcessEnded exitCode
# Remove this process ID from the list of process IDs
set pidList [lindex [intersect3 $pidList $processId] 0]
set processCount [llength $pidList]
# Occasionally, a child process quits but the signal was lost. This
# block of code will go through the list of remaining process IDs
# and remove those that has finished
set updatedPidList {}
foreach pid $pidList {
if {![catch {exec ps $pid} errmsg]} {
lappend updatedPidList $pid
}
}
set pidList $updatedPidList
# Show the remaining processes
if {$processCount > 0} {
puts "Waiting for [llength $pidList] processes"
} else {
set allFinished 1
puts "All finished"
}
}
# A signal handler that gets called when a child process finished.
# This handler needs to exit quickly, so it delegates the real works to
# the proc updatePidList
proc childTerminated {} {
# Restart the handler
signal -restart trap SIGCHLD childTerminated
# Update the list of process IDs
while {![catch {wait -nohang} stat] && $stat ne {}} {
after idle [list updatePidList $stat]
}
}
#
# Main starts here
#
puts "Main begins"
set NUMBER_OF_PROCESSES_TO_LAUNCH 10
set pidList {}
set allFinished 0
# When a child process exits, call proc childTerminated
signal -restart trap SIGCHLD childTerminated
# Spawn many processes
for {set i 0} {$i < $NUMBER_OF_PROCESSES_TO_LAUNCH} {incr i} {
set childId [exec tclsh child.tcl $i &]
puts "child #$i, pid=$childId"
lappend pidList $childId
after 1000
}
# Do some processing
puts "list of processes: $pidList"
puts "Waiting for child processes to finish"
# Do some more processing if required
# After all done, wait for all to finish before exiting
vwait allFinished
puts "Main ends"
#!/usr/bin/env tclsh
# child script: simulate some lengthy operations
proc randomInteger {min max} {
return [expr int(rand() * ($max - $min + 1) * 1000 + $min)]
}
set duration [randomInteger 10 30]
puts " child #$argv runs for $duration miliseconds"
after $duration
puts " child #$argv ends"
Main begins
child #0, pid=64525
child #0 runs for 17466 miliseconds
child #1, pid=64526
child #1 runs for 14181 miliseconds
child #2, pid=64527
child #2 runs for 10856 miliseconds
child #3, pid=64528
child #3 runs for 7464 miliseconds
child #4, pid=64529
child #4 runs for 4034 miliseconds
child #5, pid=64531
child #5 runs for 1068 miliseconds
child #6, pid=64532
child #6 runs for 18571 miliseconds
child #5 ends
child #7, pid=64534
child #7 runs for 15374 miliseconds
child #8, pid=64535
child #8 runs for 11996 miliseconds
child #4 ends
child #9, pid=64536
child #9 runs for 8694 miliseconds
list of processes: 64525 64526 64527 64528 64529 64531 64532 64534 64535 64536
Waiting for child processes to finish
Waiting for 8 processes
Waiting for 8 processes
child #3 ends
Waiting for 7 processes
child #2 ends
Waiting for 6 processes
child #1 ends
Waiting for 5 processes
child #0 ends
Waiting for 4 processes
child #9 ends
Waiting for 3 processes
child #8 ends
Waiting for 2 processes
child #7 ends
Waiting for 1 processes
child #6 ends
All finished
Main ends
allFinished variable, instead of using the vwait command. That seems to be re-inventing the wheels, though. –
Featherweight GNU parallel and xargs
These two tools that can make scripts simpler, and also control the maximum number of threads (thread pool). E.g.:
seq 10 | xargs -P4 -I'{}' echo '{}'
or:
seq 10 | parallel -j4 echo '{}'
See also: how to write a process-pool bash shell
Even if you do not have the pid, you can trigger 'wait;' after triggering all background processes. For. eg. in commandfile.sh-
bteq < input_file1.sql > output_file1.sql &
bteq < input_file2.sql > output_file2.sql &
bteq < input_file3.sql > output_file3.sql &
wait
Then when this is triggered, as -
subprocess.call(['sh', 'commandfile.sh'])
print('all background processes done.')
This will be printed only after all the background processes are done.
© 2022 - 2024 — McMap. All rights reserved.
/procmounted, checking in there is probably the fastest way to look for a PID from pure Tcl. – Garratt