Can I use `std::this_thread::sleep_for()` with an MPI process?

Question

MPI runs my program with multiple processes.

I'd one of these processes to sleep for a while so that it's using minimal CPU.

std::this_thread::sleep_for() looks like what I want, but that thread bit looks a little sketchy in this context.

Is it okay to do this?

Banjermasin

Most MPI implementations start each rank in its own separate process. Each process has at least one thread. The use of this_thread is therefore perfectly fine, but mind that MPI often spawns additional threads for internal use and you should not mess with them (which is actually hard to impossible using the C++ interface anyway). – Coney

@HristoIliev: I would upvote this as an answer. – Banjermasin

Answer 1

This is perfectly OK to do - nothing should crash or hang as a result of it.

However, your "so that it's using minimal CPU" is a little worrying. Are you running more MPI processes than you have hardware threads available to execute them? That sort of oversubscription is generally terrible for performance, and should be avoided. Best performance is often seen with one fewer process per hardware node than the number of hardware threads it offers, to allow system processes someplace to run without pre-empting the application.

The case where this could be well-justified (on which I just published a paper) is if you have a section of your program where you have less parallelism than you have processes. If you're running on Intel CPUs with Turbo Boost, then having the idle processes actually sleep can allow the core(s) running the working process(es) to run at a higher clock speed.

Sawtelle

Thanks. I am limiting the number of active MPI processes to equal hardware threads, but it is highly convenient to have (many) more processes which are not doing work but able to hold on to intermediate results of computation. I have set the inactive processes to loop over an MPI_Iprobe + sleep_for(50ms) pair. – Banjermasin

@Richard, why don't you simply use the blocking MPI_Probe instead and tell the MPI library to not use busy loops while polling the network interfaces? Or are you doing some additional work in that MPI_Iprobe + sleep loop? – Coney

@HristoIliev: OpenMPI did not seem to recognise the mca option I sent it. It's unclear whether Intel MPI responded to an environment variable. I have asked a new and glorious question about this here. – Banjermasin

@Richard, which MCA parameter in particular? – Coney

@HristoIliev: that's probably best discussed on the new question. – Banjermasin