Is it possible to attach a callback to be executed on a request completion?

#include "mpi.h" #include <stdio.h> void mycallback(void* data){ (int*)data += 1; // add one to the received data } int main(int argc, char *argv[]){ int myid, numprocs, left, right; int buffer[10], buffer2[10]; MPI_Request request; MPI_Status status; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD, &numprocs); MPI_Comm_rank(MPI_COMM_WORLD, &myid); right = (myid + 1) % numprocs; left = myid - 1; if (left < 0) left = numprocs - 1; MPI_Irecv(buffer, 10, MPI_INT, left, 123, MPI_COMM_WORLD, &request); // Attach_Callback(request, &mycallback); //somewhere after this point recv is completed an f is executed MPI_Send(buffer2, 10, MPI_INT, right, 123, MPI_COMM_WORLD); MPI_Wait(&request, &status); //the recv and the callback must have been called at this point MPI_Finalize(); return 0; }

In the standard there is not such a thing.

As said by@AhmedMasud , you can find a way around using generalized requests: http://mpi-forum.org/docs/mpi-3.1/mpi31-report/node297.htm#Node297

As you can read in it, the standard will probably never include callback for the Irecv, due to some reason I agree with (split of the work between MPI threads and your program threads).

What you attempt to do is not trivial and is linked to quite a lot of portability issue. The question you should ask yourself: will I really get any benefit from having the callback executed in the MPI thread? I suspect that this benefit in your mind is efficency, but for efficiency reasons one should avoid Irecv and Isend, non-blocking is a nice feature but should be used only and only if you have no other choices (for example a output server, where you definitely do not want to wast time of the compute clients (but even it that case, buffered send is usually better and leads you to a larger bandwidth and a smaller latency)).

What is the real structure of the comm you need? If it is 0->1, 1->2 ... n-1->n, n->0 this code works well (and will be faster than your solution) then you can define a callback easily with your favorite way of doing it (time to solution will be much smaller, debugging infinitely easier :-)):

template<class Type> 
void Parallel::sendUp(Type& bufferSend,
                      Type& bufferRec, 
                      long len)
{
    if(this->rank()%2==0)
    {
        if(this->rank()!=this->size()-1)
        {
            this->send(bufferSend,len,this->rank());
        }
        if(this->rank()!= 0)
        {
            this->receive(bufferRec,len,this->rank()-1);
        }
        else if(this->size()%2==0)
        {
            this->receive(bufferRec,len,this->size()-1);
        }
    }
    else
    {
        this->receive( bufferRec, len , this->rank()-1);
        if(this->grid_rank()!=this->grid_size()-1)
        {
            this->send(bufferSend,len,this->rank()+1);
        }
        else
        {
            this->send( bufferSend, len , 0);
        }
    }

    if(this->size()%2!=0)
    {
        if(this->rank()==this->size()-1)
        {
            this->send( bufferSend, len , 0);
        }
        if(this->grid()==0)
        {
            this->receive(bufferRec, len , this->size()-1);
        }
    }
}

In that code, the parallel object is "just" a wrapper to some MPI calls, just to simplify the calls:

parallel.rank() = rank in the comm
parallel.size() = size of the comm
parallel.send/rec() is defined as follow

template<class Type> 
void Parallel::send(Type* array, int len, int to)
{
    MPI_Send(array, len*sizeof(Type), MPI_BYTE, to, 0,comm_);
}

template<class Type> 
void Parallel::rec(Type* array, int len, int to)
{
    MPI_Send(array, len*sizeof(Type), MPI_BYTE, to, 0,comm_);
}

template<class Type>
MPI_Status Parallel2d::receive(Type& array, int from, int len)
{
    MPI_Status  status;
    MPI_Recv( &array, len*sizeof(Type), MPI_BYTE, from, 0,comm_,&status);
    return status;
}

Hope it helps.

Recommended topics

Hot tags