Cross-Compiling Armadillo Linear Algebra Library

Question

I enjoy using the Armadillo Linear Algebra Library. It becomes extremely nice when porting octave .m files over to C++, especially when you have to use the eigen methods.

However I ran into issues when I had to take my program from my native vanilla G++ and dump it onto my ARM processor. Since I spent a few hours muddling my way though it I wanted to share so others might avoid some frustration.

If anyone else could add anything else I would love it. This was the process I used to tackle this and surely isn't the only or best approach.

Euphroe

Answer 1

First off I use Code Sourcery as my cross-compiler. I know there others out there, but I haven't gotten around to rebuilding for another compiler yet, regardless this should be applicable to any compiler.

Information:

The Armadillo library requires LAPACK and BLAS, but Code Sourcery has no Fortran compiler. This led me to a f2c'ed version of LAPACK and BLAS.

1. Get the sources:

First off go grab the sources.

• Armadillo
• LAPACK & BLAS

2. Cross-Compile CLAPACK

Once we've got the sources the next thing to do is cross-compile them for our end hardware. In my case an ARM7 using CodeSourcery. Its a REALLY good idea to read the READMEs here, you really can do all of this by just taking the time and reading them.

1. First thing to do is change the make.inc file to look at our cross-compiler instead of the normal GCC. Normally you would $export, but I found it easier to keep track by modifying the makefiles.

   Edit clapack-3.2.1-CMAKE/make.inc from:

   CC = GCC
   LOADER = GCC
   

   to:

   CC = [CROSS-COMPILER-GCC location]
   LOADER = [CROSS-COMPILER-GCC location]
   

   Edit clapack-3.2.1-CMAKE/F2CLIBS/libf2c/Makefile from:

   ld -r -x -o $*.xxx $*.o
   

   to:

   [CROSS-COMPILER-LD location] -r -x -o $*.xxx $*.o
   

2. Compile the f2c libraries:

   $make f2clib
   

   When I make the f2c libraries I get an error at the very end:

   ./a.out > arith.h
   /bin/sh: ./a.out: cannot execute binary file
   make[1]: *** [arith.h] Error 126
   make[1]: Leaving directory `/home/matt/clapack-3.2.1-CMAKE/F2CLIBS/libf2c'
   make: *** [f2clib] Error 2
   

   No actual problem here. Of course it will have trouble executing, it was cross-compiled!

3. Compile BLAS:

   $make blaslib
   

   Once this is done you will notice you have a new "blas_XXXXX.a". This is your cross-compiled BLAS library.

4. Compile LAPACK:

   The make.inc will point you to use $make lapacklib, but this will lead it to attempt more execution of cross-compiled items. Instead $cd into the SRC directory and:

   $make
   

   That should generate your new "lapack_XXXXX.a". Now that we have our F2C, LAPACK and BLAS I recommend placing them somewhere that makes sense so you can find them later. In my case I placed them where I keep my Code Sourcery compiler /CodeSourcery/arm-none-linux-gnueabi/usr/lib. Remember to rename these files:

   $cp libf2c.a [CROSS-COMPILE LIBRARY PATH]/libf2c.a
   $cp blas_XXXXX.a [CROSS-COMPILE LIBRARY PATH]/libblas.a
   $cp lapack_XXXXX.a [CROSS-COMPILE LIBRARY PATH]/liblapack.a
   

   Remember they have to have the "lib" to be recognized later. Again go ahead and store these in your cross-compiled library location. I have it setup with my tool-chain to make it easier to separate from normal gcc/g++.

3. Cross-Compile ARMADILLO

First off read the README, always the best place to start.
Go ahead and run:

    $cmake .

This will get everything ready and generate everything cmake will need for creating our shared armadillo library. The way I proceeded here is not how I think you are supposed to, but as I am no wizard with makefiles in general I thought it would be helpful to show what I did to get it to cross-compile. I modified the generated CMakeCache.txt lines with the following:

    //CXX compiler.
    CMAKE_CXX_COMPILER:FILEPATH=[CROSS-COMPILER-G++ location]

I know there is somewhere in that CMakeCache.txt file where you can specify the path to the location of our BLAS and LAPACK, but I struggled to figure it out. Instead of bashing my head against this issue I just modified the "CMakeFiles/armadillo.dir/link.txt" and manually added "-L [Cross-Compiled BLAS/LAPACK directory]. Someone more familiar with how to do this could specify in the comments? Next since we want to manually link the BLAS and LAPACK libraries when we compile our program later (like the README says) modify "include/armadillo_bits/config.hpp" and make sure the line defining the use of the arma wrapper is commented out:

    //  #define ARMA_USE_WRAPPER

The only thing left to do is $cd back to the root of the armadillo directory and

    $make

Once the make completes you should be able to use Armadillo in your programs.

4. Using ARMADILLO in your program

To use Armadillo in your program add the include #include <armadillo> and the namespace using namespace arma;. Now you should be able to use all of the vec and mat you feel like. Normally when using arma all you need to do at compile time is to link libarmadillo.so library, but as I stated earlier we will need to link BLAS and LAPACK directly instead. So here is my GCC C++ Compiler synatx:

    [CROSS-COMPILER-G++] -I [CROSS-COMPILED ARMADILLO DIRECTORY]/include ...

and my linker:

    [CROSS-COMPILER-G++] -L [CROSS-COMPILED LIBRARY] -o ... -llapack -lf2c -lblas

Also note that the order in which you link libraries does matter! lapack must come first, then f2c, then blas.

Really all you need to make sure happens is that the cross-compiled armadillo directory is included when you compile and your linking is setup correctly as above.

Again more information is better, please feel free to add more comments. What worked for you different that what I did, what I did wrong, what could be done to improve.

Thanks.

Euphroe

In point 3, instead of $make . (with a dot at the end) did you mean cmake . ? – Wheels

You are correct, edited to change $make . to $cmake .. Thanks! – Euphroe

@Euphroe the correct way to cross compile with CMake is to use toolchain file. If you were using Buildroot, you would have toolchain file already setup. – Whereupon

As another data point, I've managed to jump through a bunch of hoops to build libopenblas using a Fortran cross compiler, then linked Armadillo to it and built it using the toolchain file from github.com/taka-no-me/android-cmake . It's a little bit less hacky IMO and hopefully I'll be able to clean up and post instructions some day soon. But at the least using the toolchain file should be easier than changing the compiler path manually. – Quest

Answer 2

My specific setup is OSX (IDE Eclipse) cross compiled to a Beaglebone Black. However, these instructions should work for similar setups.

Optional:

For compiling I used the Mac OS X ARM GNU Linux G++ Lite 2013.11-33 Toolchain. Specifically, the ARM GNU/Linux G++ Lite 2013.11-33 Advanced Binary.

1. Download:

Just as Matt posted, the GCC cross compiler doesn't support Fortran, so if you want to compile LAPACK and BLAS, use a modified version found here. I'm using is clapack-3.2.1-CMAKE.tgz

2. Make a cross compile cmake file:

You can use a toolchain builder or just write one. I wrote one.

Example:

# http://www.cmake.org/Wiki/CMake_Cross_Compiling#The_toolchain_file

# REQUIRED
SET(CMAKE_SYSTEM_NAME Linux)
SET(CMAKE_SYSTEM_VERSION 1)
SET(CMAKE_SYSTEM_PROCESSOR arm)

# Added for the beaglebone
SET(FLOAT_ABI_SUFFIX "")

# specify the cross compiler
SET(CMAKE_C_COMPILER   /usr/local/arm-2013.11/bin/arm-none-linux-gnueabi-gcc)
SET(CMAKE_CXX_COMPILER /usr/local/arm-2013.11/bin/arm-none-linux-gnueabi-c++)

# where is the target environment 
SET(CMAKE_FIND_ROOT_PATH  /usr/local/arm-2013.11)

# search for programs in the build host directories
SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# for libraries and headers in the target directories
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

Note: This is my first time making a cmake file. It isn't guaranteed to be correct.

You'll want to replace /usr/local/arm-2013.11/bin/arm-none-linux-gnueabi-gcc with the path to your selected compiler, as well as /usr/local/arm-2013.11/bin/arm-none-linux-gnueabi-c++ and /usr/local/arm-2013.11

I chose to save this cmake file as beaglebone.cmake

3. Compile:

Extract clapack-3.2.1-CMAKE.tgz and cd

compile: cmake -DCMAKE_TOOLCHAIN_FILE=~/Dropbox/workspaces/beaglebone/beaglebone.cmake
where ~/Dropbox/workspaces/beaglebone/beaglebone.cmake is the path to your cmake file.

make

For some sort of reason I get:

ds-mac-pro:clapack-3.2.1-CMAKE bunny$ make
Scanning dependencies of target arithchk
[  0%] Building C object F2CLIBS/libf2c/CMakeFiles/arithchk.dir/arithchk.c.o
Linking C executable arithchk
[  0%] Built target arithchk
[  0%] Generating arith.h
/bin/sh: arithchk: command not found
make[2]: *** [F2CLIBS/libf2c/arith.h] Error 127
make[1]: *** [F2CLIBS/libf2c/CMakeFiles/f2c.dir/all] Error 2
make: *** [all] Error 2

Oddly, running make again compiles fine:

ds-mac-pro:clapack-3.2.1-CMAKE bunny$ make
[  0%] Built target arithchk
Scanning dependencies of target f2c
[  0%] Building C object F2CLIBS/libf2c/CMakeFiles/f2c.dir/f77vers.c.o
[  0%] Building C object F2CLIBS/libf2c/CMakeFiles/f2c.dir/i77vers.c.o
[  0%] Building C object F2CLIBS/libf2c/CMakeFiles/f2c.dir/main.c.o
[  0%] Building C object F2CLIBS/libf2c/CMakeFiles/f2c.dir/s_rnge.c.o

...

[100%] Building C object TESTING/EIG/CMakeFiles/xeigtstz.dir/xerbla.c.o
[100%] Building C object TESTING/EIG/CMakeFiles/xeigtstz.dir/xlaenv.c.o
[100%] Building C object TESTING/EIG/CMakeFiles/xeigtstz.dir/chkxer.c.o
[100%] Building C object TESTING/EIG/CMakeFiles/xeigtstz.dir/__/__/INSTALL/dsecnd.c.o
Linking C executable xeigtstz
[100%] Built target xeigtstz

4. Copy (Install):

find . | grep \.a$

returns

./BLAS/SRC/libblas.a
./F2CLIBS/libf2c/libf2c.a
./SRC/liblapack.a
./TESTING/MATGEN/libtmglib.a

cp libblas.a libf2c.a and liblapack.a to your favorite library folder. I made /usr/local/armadillo-4.300.3/lib

Required:

1. Copy Includes:

Armadillo doesn't need to be compiled. I haven't ran any benchmarks to verify, but it appears that using LAPACK and BLAS works fine with an uncompiled copy of Armadillo by using #define in code. Code examples later.

extract armadillo-4.300.3.tar.gz

(optional) cp armadillo-4.300.3/include/ /usr/local/armadillo-4.300.3/include
and of course, replace /usr/local/armadillo-4.300.3/include with your path of choice

2. Setup GCC to use Armadillo:

I'm using Eclipse with the C/C++ Cross Compiler Support addon (Help Menu -> Install New Software...), but the instructions are easy to convert to cli or other IDEs.

In the project properties window: C/C++ Build -> Settings

Cross G++ Compiler -> Includes -> Include paths (-I)

Click the + to add an include. My include path is: /usr/local/armadillo-4.300.3/include

3. OPTIONAL - Setup GCC to use the compiled libraries:

Cross G++ Linker -> Libraries ->

Libraries (-l)

lapack
f2c
blas

Library Search Path (-L)

Click the + to add a path. My path is: /usr/local/armadillo-4.300.3/lib

3. Code Example:

In a cpp file try:

#include <armadillo>
using namespace arma;

mat A = randu<mat>(4,5);
mat B = randu<mat>(4,5);

std::cout << A*B.t() << std::endl;

Success! :D

Some functionality Armadillo does not directly support and will only work with compiled libraries. You can test that the libraries are compiled and working correctly by running a simple test like:

#define ARMA_DONT_USE_WRAPPER
#define ARMA_USE_LAPACK
#define ARMA_USE_BLAS
#include <armadillo>
using namespace arma;

mat    A = randu<mat>(5,5);
double x = det(A);

std::cout << x << std::endl;

Resistless