I'm testing out the parLapplyLB()
function to understand what it does to balance a load. But I'm not seeing any balancing happening. For example,
cl <- parallel::makeCluster(2)
system.time(
parallel::parLapplyLB(cl, 1:4, function(y) {
if (y == 1) {
Sys.sleep(3)
} else {
Sys.sleep(0.5)
}}))
## user system elapsed
## 0.004 0.009 3.511
parallel::stopCluster(cl)
If it was truly balancing the load, the first job (job 1) that sleeps for 3 seconds would be on the first node and the other three jobs (jobs 2:4) would sleep for a total of 1.5 seconds on the other node. In total, the system time should only be 3 seconds.
Instead, I think that jobs 1 and 2 are given to node 1 and jobs 3 and 4 are given to node 2. This results in the total time being 3 + 0.5 = 3.5 seconds. If we run the same code above with parLapply()
instead of parLapplyLB()
, we get the same system time of about 3.5 seconds.
What am I not understanding or doing wrong?