How does the MPI_SENDRECV work?

Question

I have a question about MPI_SENDRECV. here is an example:

 PROGRAM sendrecv
  IMPLICIT NONE
  INCLUDE "mpif.h"
  INTEGER a,b,myrank,nprocs,ierr
  integer istat(MPI_STATUS_SIZE)
  CALL MPI_INIT(ierr)
  CALL MPI_COMM_SIZE(MPI_COMM_WORLD, nprocs, ierr)
  CALL MPI_COMM_RANK(MPI_COMM_WORLD, myrank, ierr)
  if (myrank.eq.0) then
     a=1;b=3
  else
     a=2;b=4
  endif

  if (myrank == 0) then
     call MPI_SENDRECV(b,1,MPI_REAL,1,0,
 .                     a,1,MPI_REAL,1,0,
 .                     MPI_COMM_WORLD,istat,ierr)
  elseif (myrank == 1) then
     call MPI_SENDRECV(b,1,MPI_REAL,0,0,
 .                     a,1,MPI_REAL,0,0,
 .                     MPI_COMM_WORLD,istat,ierr)
  end if
  if (myrank.eq.0) then
     write(*,*) b,a
  else
     write(*,*) a,b
  endif
  CALL MPI_FINALIZE(ierr)
  END

After this we get 3 4 and 3 4. My question is that is we replace the MPI_SENDRECV(if we assume that MPI_SENDRECV is send first and then receive)

if (myrank == 0) then
   call MPI_SEND(b,1,MPI_REAL,1,0,MPI_COMM_WORLD,ierr)
   call MPI_RECV(a,1,MPI_REAL,0,0,MPI_COMM_WORLD,istat,ierr)                  
elseif (myrank == 1) then
   call MPI_SEND(b,1,MPI_REAL,0,0,MPI_COMM_WORLD,ierr)
   call MPI_RECV(a,1,MPI_REAL,1,0,MPI_COMM_WORLD,istat,ierr)                      
end if

Then this will be deadlock So this means that MPI_SENDRECV it not first sends and then receives, but sends ans receives simultaneously,right?

Baum

Answer 1

You are right, MPI_Sendrecv is not the same a send followed by a receive. Think of it as an MPI_Isend, MPI_Irecv, and a pair of MPI_Waits. So in effect, the send and receive proceed in parallel.

Incidentally, this is how it is commonly implemented in the MPI libraries.

If you wanted to fix the deadlock in your second example, the processes would have to issue the sends and receives in a different order. So rank 0 would issue a send followed by receive, and rank 1 - a receive followed by a send.

Tatiania

Thanks a lot, I know it now ! – Baum

How does a MPI_Sendrecv_replace() work? Does it use blocking Send and Recv? – Ganger

MPI_Sendrecv_replace probably deserves a separate question (it is a newly-added operation). I can imagine a couple of different implementations (and I'm sure the research groups working on optimizing MPI will benchmark these against one another). The implementation used by OpenMPI copies the data to be sent into a temporary buffer and then calls the regular Sendrecv (github.com/open-mpi/ompi/blob/…) – Tatiania

Answer 2

Even though the message is routed to receive process B , process B still has to acknowledge that it wants to receive A's data. Once it does this, the data has been transmitted. Process A is acknowledged that the data has been transmitted and may go back to work.

So your second code can't satisfy the condition, which seems like that you don't answer the call by others. It should be like the follows:

if (myrank == 0) then
   call MPI_SEND(b,1,MPI_REAL,1,0,MPI_COMM_WORLD,ierr)
   call MPI_RECV(a,1,MPI_REAL,1,0,MPI_COMM_WORLD,istat,ierr)
elseif (myrank == 1) then
   call MPI_SEND(b,1,MPI_REAL,0,0,MPI_COMM_WORLD,ierr)
   call MPI_RECV(a,1,MPI_REAL,0,0,MPI_COMM_WORLD,istat,ierr)
end if

Catbird