What is -lnuma and what program uses it for compilation?

Question

I am compiling a message passing program using openmpi with mpicxx on a Linux desktop. My makefile does the following:

mpicxx -c readinp.cpp
mpicxx -o exp_fit driver.cpp readinp.o 

at which point i get the following error:

/usr/lib64/gcc/x86_64-suse-linux/4.5/../../../../x86_64-suse-linux/bin/ld: cannot find -lnuma

My questions are:

what is -lnuma? what is using it? how should i go about linking to it?

Shithead

Answer 1

The build script can't find the numa library - NUMA (Non Uniform Memory Access). The -l option tells the linker to link the library, but your system ether doesn't have the right one installed or your search path for the linker is incomplete/wrong.

Try querying your package-manager (apt or rpm) for a package libnuma.

Sewoll

i know of others who have compiled using OpenMPI on this system. is it possible that my user account simply doesn't have access? do i need to add the path to where the library is located? if so, how do i find said path? – Shithead

It may depend on your distro where you'll find the stuff. Most distros seprate the binaries and headers in to packages like 'libnuma' and 'libnuma-devel'. Try searching for both. If the are installesd, you should be able to query the paths (e.g. rpm -ql libnuma or dpkg -L libnuma). – Sewoll

Answer 2

Thanks Jonathan Dursi!

On Ubuntu, the package name is libnuma-dev.
apt-get install libnuma-dev

Immoralist

Answer 3

OpenMPI, and I think mpich2, uses libnuma (`a simple programming interface to the NUMA (Non Uniform Memory Access) policy supported by the Linux kernel') for memory affinity -- to ensure that the memory for a particular MPI task stays close to the core that the task is running on, as vs. being kept in cache on another socket entirely. This is important for performance on multicore nodes.

You may need to use YaST to install libnuma-devel if your linker can't find the library.

Eastward

Answer 4

I got the same error working on a remote server, which had the NUMA library installed. In particular, the file /usr/lib64/libnuma.so.1 existed. It appears that the linker only looked for the file under the name libnuma.so. Creating the symlink

ln -s /usr/lib64/libnuma.so.1 /usr/lib64/libnuma.so

as described here might have worked, but in my case I did not have permission to create files in /usr/lib64. I got around this by creating the symlink in some other location of which I have write permission:

ln -s /usr/lib64/libnuma.so.1 /some/path/libnuma.so

and then add this path to the compilation flags. In your case this would be

mpicxx -L/some/path -o exp_fit driver.cpp readinp.o

In my case of a larger build process (compiling fftw), I added the path to the LDFLAGS environment variable,

export LDFLAGS="${LDFLAGS} -L/some/path"

which fixed the issue.

Minute

the right fix is to have your sysadmin install the devel package (libnuma-dev on ubuntu, numactl-devel in RHEL, ...). libnuma is used to build Open MPI, but most MPI apps are unlikely to use it directly (which means the libnuma.so.1 should be enough at runtime) – Dingle