how to compile MPI and non-MPI version of the same program with automake?

Question

I have a C++ code that can be compiled with MPI support depending on a certain preprocessor flag; missing the appropriate flag, the sources compile to a non-parallel version.

I would like to setup the Makefile.am so that it compiles both the MPI-parallel and the sequential version, if an option to ./configure is given.

Here's the catch: MPI has its own C++ compiler wrapper, and insists that sources are compiled and linked using it rather than the standard C++ compiler. If I were to write the Makefile myself, I would have to do something like this:

myprog.seq: myprog.cxx
    $(CXX) ... myprog.cxx

myprog.mpi: myprog.cxx
    $(MPICXX) -DWITH_MPI ... myprog.cxx

Is there a way to tell automake that it has to use $(MPICXX) instead of $(CXX) when compiling the MPI-enabled version of the program?

Pastel

Do you care if the serial program is linked against the MPI libraries in the "build both" case? If not, you can just use mpicxx for both -- it's not like it will hurt anything. And if you aren't building the mpi version, you can set use g++ for everything. Empirically, this is a what a lot of packages that have both parallel and serial versions seem to do -- hdf5 comes to mind. – Talion

In general, no, and I used to compile this way. I develop some PMPI tools, though, and in general you do not want to link an interceptor library to MPI, which you are forced to do if you compile with mpicc. – Dentation

Answer 1

I have the same problem, and I've found that there's no really good way to get autotools to conditionally use MPI compilers for particular targets. Autotools is good at figuring out which compiler to use based on what language your source is written in (CC, CXX, FC, F77, etc.), but it really isn't good at figuring out whether or not to use MPI compiler for a particular target. You can set MPICC, MPICXX, etc., but you essentially have to rewrite all your Makefile rules for your target (as you've done above) if you use the compiler this way. If you do that, what's the point of writing an automake file?

Someone else suggested using MPI like an external library, and this is the approach I'd advocate, but you shouldn't do it by hand, because different MPI installations have different sets of flags they pass to the compiler, and they can depend on the language you're compiling.

The good thing is that all the currently shipping MPI compilers that I know of support introspection arguments, like -show, -show-compile or -show-link. You can automatically extract the arguments from the scripts.

So, what I did to deal with this was to make an m4 script that extracts the defines, includes, library paths, libs, and linker flags from the MPI compilers, then assigns them to variables you can use in your Makefile.am. Here's the script:

lx_find_mpi.m4

This makes MPI work the way automake expects it to. Incidentally, this is the approach CMake uses in their FindMPI module, and I find it works quite well there. It makes the build much more convenient because you can just do something like this for your targets:

bin_PROGRAMS = mpi_exe seq_exe

# This is all you need for a sequential program
seq_exe_SOURCES = seq_exe.C

# For an MPI program you need special LDFLAGS and INCLUDES
mpi_exe_SOURCES = mpi_exe.C
mpi_exe_LDFLAGS = $(MPI_CXXLDFLAGS)

INCLUDES = $(MPI_CXXFLAGS)

There are similar flags for the other languages since, like I said, the particular flags and libraries can vary depending on which language's MPI compiler you use.

lx_find_mpi.m4 also sets some shell variables so that you can test in your configure.ac file whether MPI was found. e.g., if you are looking for MPI C++ support, you can test $have_CXX_mpi to see if the macro found it.

I've tested this macro with mvapich and OpenMPI, as well as the custom MPICH2 implementation on BlueGene machines (though it does not address all the cross-compiling issues you'll see there). Let me know if something doesn't work. I'd like to keep the macro as robust as possible.

Dentation

You should add it to LDADD instead of LDFLAGS otherwise some linker will throws errors. And the script actual has a bug: with sed, to remove trailing space you should use sed/ +/ /g instead of sed/ */ /g. – Mont

Your macro runs fine for me. However, I have to run it twice to get both the values from MPICC and MPICXX. In the second call I wrap the macro between AC_LANG_PUSH([C++]) and AC_LANG_POP([C++]). Thank for the SW. – Cristalcristate

Answer 2

I am sorry that having automake use MPI is so difficult. I have been struggling with this for many months trying to find a good solution. I have a source tree that have one library and then many programs in sub-folders that use the library. Some of the folders are mpi programs, but when I try to replace CXX with the MPI compiler using in Makefile.am.

if USE_MPI
  MPIDIR = $(MPICOMPILE)
  MPILIB = $(MPILINK)
  CXX=@MPICXX@
  F77=@MPIF77@
  MPILIBS=$(MPILINK)
endif

I get

CXX was already defined in condition TRUE, which includes condition USE_MPI ...
configure.ac:12: ... `CXX' previously defined here

I don't have a rule that specifies the compiler, so maybe there is a way to do that.

SUBDIRS = .
bin_PROGRAMS = check.cmr
check_ccmr_SOURCES = check_gen.cpp
check_ccmr_CXXFLAGS = -I$(INCLUDEDIR) $(MPIDIR)
check_ccmr_LDADD = -L$(LIBDIR)
check_ccmr_LDFLAGS = $(MPILIB)

Shlomo

Answer 3

If you have disabled the subdir-objects option to automake, something like this might work:

configure.ac:

AC_ARG_ENABLE([seq], ...)
AC_ARG_ENABLE([mpi], ...)
AM_CONDITIONAL([ENABLE_SEQ], [test $enable_seq = yes])
AM_CONDITIONAL([ENABLE_MPI], [test $enable_mpi = yes])
AC_CONFIG_FILES([Makefile seq/Makefile mpi/Makefile])

Makefile.am:

SUBDIRS =
if ENABLE_SEQ
SUBDIRS += seq
endif
if ENABLE_MPI
SUBDIRS += mpi
endif

sources.am:

ALL_SOURCES = src/foo.c src/bar.cc src/baz.cpp

seq/Makefile.am:

include $(top_srcdir)/sources.am

bin_PROGRAMS = seq
seq_SOURCES = $(ALL_SOURCES)

mpi/Makefile.am:

include $(top_srcdir)/sources.am

CXX = $(MPICXX)
AM_CPPFLAGS = -DWITH_MPI

bin_PROGRAMS = mpi
mpi_SOURCES = $(ALL_SOURCES)

The only thing stopping you from doing both of these in the same directory is the override of $(CXX). You could, for instance, set mpi_CPPFLAGS and automake would handle that gracefully, but the compiler switch makes it a no-go here.

Treehopper

Downvote sans comment. Great job. I'll grant that it's a bit of a dirty hack, but do you have a better idea? – Treehopper

Answer 4

A possible workaround for not using different sources could be:

myprog.seq: myprog.cxx
    $(CXX) ... myprog.cxx

myprog-mpi.cxx: myprog.cxx
    @cp myprog.cxx myprog-mpi.cxx

myprog.mpi: myprog-mpi.cxx
    $(MPICXX) -DWITH_MPI ... myprog-mpi.cxx
    @rm -f myprog-mpi.cxx

for Automake:

myprog-bin_PROGRAMS = myprog-seq myprog-mpi

myprog_seq_SOURCES = myprog.c

myprog-mpi.c: myprog.c
    @cp myprog.c myprog-mpi.c

myprog_mpi_SOURCES = myprog-mpi.c
myprog_mpi_LDFLAGS = $(MPI_CXXLDFLAGS)

INCLUDES = $(MPI_CXXFLAGS)
BUILT_SOURCES = myprog-mpi.c
CLEANFILES = myprog-mpi.c

Daff

Clever workaround, thanks! I think there's a typo: the second myprog..._SOURCES line should read myprog_mpi_SOURCES = myprog-mpi.c and then mpyorog_mpi_LDFLAGS = ... – Pastel

You additionally have to list myprog-mpi.c as autogenerated using BUILT_SOURCES = myprog-mpi.c. – Starlike

Answer 5

Here is the solution that I came up with for building a two static libraries - one with MPI (libmylib_mpi.a) and one without (libmylib.a). The advantage of this method is that there is no need for duplicate source files, a single Makefile.am for both variants, and capability to use subdirs. You should be able to modify this as needed to produce a binary instead of a library. I build the non-MPI library as normal, then for the MPI variant, I leave _SOURCES empty and use _LIBADD instead, specifying an extension of .mpi.o for the object files. I then specify a rule to generate the MPI object files using the MPI compiler.

Overall file / directory structure is something like

configure.ac
Makefile.am
src
    mylib1.cpp
    mylib2.cpp
    ...
include
    mylib.h
    ...

configure.ac:

AC_INIT()
AC_PROG_RANLIB
AC_LANG(C++)
AC_PROG_CXX
# test for MPI, define MPICXX, etc. variables, and define HAVE_MPI as a condition that will evaluate to true if MPI is available and false otherwise.
AX_MPI([AM_CONDITIONAL([HAVE_MPI], [test "1" = "1"])],[AM_CONDITIONAL([HAVE_MPI], [test "1" = "2"])]) #MPI optional for xio
AC_CONFIG_FILES([Makefile])
AC_OUTPUT

There is probably a more efficient way to do the conditional check than I have listed here (I'm welcome to suggestions).

Makefile.am:

AUTOMAKE_OPTIONS = subdir-objects
lib_LIBRARIES = libmylib.a
libmylib_a_SOURCES = src/mylib_1.cpp src/mylib_2.cpp ...

#conditionally generate libmylib_mpi.a if MPI is available
if HAVE_MPI
    lib_LIBRARIES += libmylib_mpi.a
    libmylib_mpi_a_SOURCES = #no sources listed here
    #use LIBADD to specify objects to add - use the basic filename with a .mpi.o extension
    libmylib_mpi_a_LIBADD = src/mylib_1.mpi.o src/mylib_2.mpi.o ...
endif
AM_CPPFLAGS = -I${srcdir}/include

include_HEADERS = include/mylib.h

# define a rule to compile the .mpi.o objects from the .cpp files with the same name
src/%.mpi.o: ${srcdir}/src/%.cpp ${srcdir}/include/mylib.h
    $(MPICXX)  $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -DWITH_MPI=1 -c $(patsubst %.mpi.o,$(srcdir)/%.cpp,$@) -o $@

#define a rule to clean the .mpi.o files
clean-local:
    -rm -f src/*.mpi.o

Inconvenient

Answer 6

MPI installations do (usually) ship with compiler wrappers, but there is no requirement that you use them -- MPI does not insist on it. If you want to go your own way you can write your own makefile to ensure that the C++ compiler gets the right libraries (etc). To figure out what the right libraries (etc) are, inspect the compiler wrapper which is, on all the systems I've used, a shell script.

At first sight the compiler wrappers which ship with products such as the Intel compilers are a little daunting but stop and think about what is going on -- you are simply compiling a program which makes use of an external library or two. Writing a makefile to use the MPI libraries is no more difficult than writing a makefile to use any other library.

Godevil

Thanks for your answer. Although it's not required that I use the MPI compiler wrapper, there are many MPI implementations out there and each one uses its own library names, etc. -- I'd rather re-write a Makefile stanza to use $(MPICXX) than maintaining several lines of automake/autoconf code to provide CPPFLAGS/LDFLAGS/LIBS for each MPI version out there... – Pastel

It's no more difficult if all you care about is one platform. However, I run on 3 or 4 different cluster architectures, and while they all share an mpicc compiler, the particular flags and libraries for the different platform are, well, different.You can't do this portably unless you introspect the wrappers in a script, which is what I recommend doing. See my answer. – Dentation