Linking static libraries to other static libraries
Asked Answered
D

6

177

I have a small piece of code that depends on many static libraries (a_1-a_n). I'd like to package up that code in a static library and make it available to other people.

My static library, lets call it X, compiles fine.

I've created a simple sample program that uses a function from X, but when I try to link it to X, I get many errors about missing symbols from libraries a_1 - a_n.

Is there a way that I can create a new static library, Y that contains X and all the functionality needed by X (selected bits from a_1 - a_n), so that I can distribute just Y for people to link their programs to?


UPDATE:

I've looked at just dumping everything with ar and making one mega-lib, however, that ends up including a lot of symbols that are not needed (all the .o files are about 700 MB, however, a statically linked executable is 7 MB). Is there a nice way to include only what is actually needed?


This looks closely related to How to combine several C/C++ libraries into one?.

Descendent answered 28/1, 2010 at 20:14 Comment(0)
A
104

Static libraries do not link with other static libraries. The only way to do this is to use your librarian/archiver tool (for example ar on Linux) to create a single new static library by concatenating the multiple libraries.

Edit: In response to your update, the only way I know to select only the symbols that are required is to manually create the library from the subset of the .o files that contain them. This is difficult, time consuming and error prone. I'm not aware of any tools to help do this (not to say they don't exist), but it would make quite an interesting project to produce one.

Aboutship answered 28/1, 2010 at 20:25 Comment(4)
Hi Neil, I've updated the question -- do you know any way of including only the .o files that are necessary?Descendent
Update -- If you find yourself wanting to do this, take a step back, and pursue something else (unless, as John Knoeller points out, you are using Visual Studio). I sunk a lot of time into this approach and didn't get anything useful.Descendent
The GNU ld tool provides an option -r with that the output can be used as an input for ld. If you link once you should get a relocatable the can be surrogate your libraries. (did tried it thogh).Otherworldly
how would i go about this using cmake on windows/visual studio?Trifacial
A
54

If you are using Visual Studio then yes, you can do this.

The library builder tool that comes with Visual Studio allows you to join libraries together on the command line. I don't know of any way to do this in the visual editor though.

lib.exe /OUT:compositelib.lib  lib1.lib lib2.lib
Ambassadoratlarge answered 28/1, 2010 at 20:31 Comment(3)
In VS2008, in the project properties of compositelib under Librarian/General, if you check [x] Link Library Dependencies, it will do this for you if lib1 and lib2 are dependencies of compositelib. It seems slightly buggy, I would set the checkbox separately in each build configuration, not once in 'All Configurations'.Mclendon
VS2015 IDE - don't you use "Additional Dependencies" under Librarian/General to get additional libraries linked directly into the library your project is building?Pentarchy
@Pentarchy I am trying to figure this out in the last couple of days and apparently this option is obsolete and does not do anything?Spindlelegs
M
27

On Linux or MingW, with GNU toolchain:

ar -M <<EOM
    CREATE libab.a
    ADDLIB liba.a
    ADDLIB libb.a
    SAVE
    END
EOM
ranlib libab.a

Of if you do not delete liba.a and libb.a, you can make a "thin archive":

ar crsT libab.a liba.a libb.a

On Windows, with MSVC toolchain:

lib.exe /OUT:libab.lib liba.lib libb.lib
Mcconaghy answered 28/12, 2014 at 11:33 Comment(3)
Nice to know, but doesn't really solve the OP's problem, since you're including everything from libb.a into the joint lib, which can become very large if you need only a few modules from libb.Billingsley
@ElmarZander But you can use ar crsT, which does something like "symlinking" instead of copying, then you need only to ship one copy of data.Mcconaghy
Thin archives are notably used on the Linux kernel v4.19: unix.stackexchange.com/questions/5518/…Bunnybunow
D
17

A static library is just an archive of .o object files. Extract them with ar (assuming Unix) and pack them back into one big library.

Delenadeleon answered 28/1, 2010 at 20:28 Comment(0)
B
11

Note before you read the rest: The shell script shown here is certainly not safe to use and well tested. Use at your own risk!

I wrote a bash script to accomplish that task. Suppose your library is lib1 and the one you need to include some symbols from is lib2. The script now runs in a loop, where it first checks which undefined symbols from lib1 can be found in lib2. It then extracts the corresponding object files from lib2 with ar, renames them a bit, and puts them into lib1. Now there may be more missing symbols, because the stuff you included from lib2 needs other stuff from lib2, which we haven't included yet, so the loop needs to run again. If after some passes of the loop there are no changes anymore, i.e. no object files from lib2 added to lib1, the loop can stop.

Note, that the included symbols are still reported as undefined by nm, so I'm keeping track of the object files, that were added to lib1, themselves, in order to determine whether the loop can be stopped.

#! /bin/bash

lib1="$1"
lib2="$2"

if [ ! -e $lib1.backup ]; then
    echo backing up
    cp $lib1 $lib1.backup
fi

remove_later=""

new_tmp_file() {
    file=$(mktemp)
    remove_later="$remove_later $file"
    eval $1=$file
}
remove_tmp_files() {
    rm $remove_later
}
trap remove_tmp_files EXIT

find_symbols() {
    nm $1 $2 | cut -c20- | sort | uniq 
}

new_tmp_file lib2symbols
new_tmp_file currsymbols

nm $lib2 -s --defined-only > $lib2symbols

prefix="xyz_import_"
pass=0
while true; do
    ((pass++))
    echo "Starting pass #$pass"
    curr=$lib1
    find_symbols $curr "--undefined-only" > $currsymbols
    changed=0
    for sym in $(cat $currsymbols); do
        for obj in $(egrep "^$sym in .*\.o" $lib2symbols | cut -d" " -f3); do
            echo "  Found $sym in $obj."
            if [ -e "$prefix$obj" ]; then continue; fi
            echo "    -> Adding $obj to $lib1"
            ar x $lib2 $obj
            mv $obj "$prefix$obj"
            ar -r -s $lib1 "$prefix$obj"
            remove_later="$remove_later $prefix$obj"
            ((changed=changed+1))
        done
    done
    echo "Found $changed changes in pass #$pass"

    if [[ $changed == 0 ]]; then break; fi
done

I named that script libcomp, so you can call it then e.g. with

./libcomp libmylib.a libwhatever.a

where libwhatever is where you want to include symbols from. However, I think it's safest to copy everything into a separate directory first. I wouldn't trust my script so much (however, it worked for me; I could include libgsl.a into my numerics library with that and leave out that -lgsl compiler switch).

Billingsley answered 15/1, 2015 at 23:48 Comment(1)
This is excellent. For a big project (>500k symbols in >40k objects in the original library, of which I needed ~1000 symbols), this took close to an hour, whereas gcc can do essentially the same thing with dynamic linking in a few seconds. Is there some fundamental reason why this isn't easier to do and exposed by compiler tools?Mefford
S
7

Alternatively to Link Library Dependencies in project properties there is another way to link libraries in Visual Studio.

  1. Open the project of the library (X) that you want to be combined with other libraries.
  2. Add the other libraries you want combined with X (Right Click, Add Existing Item...).
  3. Go to their properties and make sure Item Type is Library

This will include the other libraries in X as if you ran

lib /out:X.lib X.lib other1.lib other2.lib
Selfdevotion answered 13/8, 2013 at 4:23 Comment(5)
Hi, I'm using Intel IPP and I built my own functions which I want all to be wrapped into one (Static) lib. Yet when I create the lib and then send the project to other computer which I want to be able to compile the project only using the lib I created I get an error saying an .h file of the Intel IPP Library is required. Any idea?Actinal
lib file is usually not enough. If you want to use it, you will need to include header files too. But it's off topic here as this thread talks about linking and your problem is at the compilation stage.Selfdevotion
ok. To help you I will need more information. Look at the build output or log and see what's complaining on the missing .h file. I think it is cl.exe. If that's the case, it will give you the name of the compiled .cpp/.cc/.c file that uses the header. What's the name of that .cpp file and which project it belongs to?Selfdevotion
I am so confused. Before I read this, it didn't seem to have ever worked (this is how my projects are already configured). But now that I've read this and reset my projects, it is apparently working. Go figure! :(Barefoot
@Barefoot this haiku below describes your experience perfectly: Yesterday it worked Today it is not working Windows is like thatSelfdevotion

© 2022 - 2024 — McMap. All rights reserved.