What is the difference between isend and issend?

Question

Need clarification to my understanding of isend and issend as given in Send Types

My understanding is that isend will return once the send buffer is free, i.e. when all the data has been released. Issend on the other hand returns only when it receives an ack from the receive of getting/not getting the entire data. Is this all there is to it?

Teutonic

Just to clarify your question title, MPI_ISEND/MPI_ISSEND/everything else is part of MPI (the Standard or API), not Open MPI (an implementation of the standard). You can think of it the same way as the WiFi standard. There's one standard that every device maker adheres to so everything can interoperate. It's the same kind of thing with MPI. If you write code that you compile with Open MPI, you would also be able to compile it with MPICH (or Intel MPI, MS-MPI, etc.) – Foliated

Answer 1

Both MPI_Isend() and MPI_Issend() return immediately, but in both cases you can't use the send buffer immediately.

Think of the difference that there is between MPI_Send() and MPI_Ssend():

• MPI_Send() can be buffered or it can be synchronous if the buffer is too large to be buffered locally, and in this case it waits to complete sending the data to the corresponding receive operation.

• MPI_Ssend() is always synchronous: it always waits to complete sending the data to the corresponding receive operation.

The inner working of the corresponding "I"-operations is very similar, except for the fact that they both don't block (return immediately): the difference is only when the MPI library signals to the user program that you can use the send-buffer (that is: MPI_Wait() returns or MPI_Test() returns true - the so called send-complete operation of the non-blocking send):

• with MPI_Isend() this can happen either when the data has been copied locally in a buffer owned by the MPI library, if below the "synchronous threshold", or when the data has been actually moved to the sibling task: the send-complete operation can be local, in case the underlying send operation is buffered.

• With MPI_Issend() MPI doesn't ever buffer data locally and the "buffer-free condition" is returned only after the data has been actually transferred (and probably ack'ed, at low level): the send-complete operation is non-local.

The MPI standard document is quite pedantic on these aspects. See section 3.7 Nonblocking Communication.

Dukey

1 clarification: MPI defines that synch sends - regardless of whether they are blocking or not - will complete when the receiver has started to receive the msg. Specifically: completion of a synch does does not mean that the receiver has completely received the msg. Indeed, it is common for MPI implementations - Open MPI included, at least in many cases - will allow a sync send to complete when a) it has locally sent all of the msg, and b) when it has received some form of an ACK back from the receiver indicating that the receiver has matched and/or started to receive the msg. – Cummine

Answer 2

Correct. Obviously both of those will only be true when the request that you get back from the call to MPI_ISEND or MPI_ISSEND is completed via a MPI_WAIT* or MPI_TEST* function.

Foliated