Combining static libraries

Question

Suppose I have three C static libraries say libColor.a which depends on *libRGB.*a which in turn depends on libPixel.a . The library libColor.a is said to depend on library libRGB.a since there are some references in libColor.a to some of symbols defined in libRGB.a. How do I combine all the above libraries to a new libNewColor.a which is independent?

Independent means the new library should have all symbols defined. So while linking I just need to give -lNewColor. The size of the new library should be minimal i.e it should not contain any symbols in libRGB.a which is not used by libColor.a etc. I tried my luck using various options in ar command (used to create and update static libraries/archives).

Wohlen

You can alternatively look at https://mcmap.net/q/169103/-combine-static-libraries-on-apple/… and use libtool – Kilowatthour

Note that the minimization step really isn't necessary. With static libraries, the linker will only grab the object files it needs, unlike shared libraries which include everything. There are reasons for and advantages to each approach; they're just different. And you really don't need to worry while you're working with static libraries. – Petitioner

Does this answer your question? How to merge two "ar" static libraries into one? – Antechamber

Answer 1

1/ Extract ALL of the object files from each library (using ar) and try to compile your code without the libraries or any of the object files. You'll probably get an absolute bucket-load of undefined symbols. If you get no undefined symbols, go to step 5.

2/ Grab the first one and find out which object file satisfies that symbol (using nm).

3/ Write down that object file then compile your code, including the new object file. You'll get a new list of undefined symbols or, if there's none, go to step 5.

4/ Go to step 2.

5/ Combine all the object files in your list (if any) into a single library (again with ar).

Bang! There you have it. Try to link your code without any of the objects but with the new library.

This whole thing could be relatively easily automated with a shell script.

Pillow

Unnecessarily complicated, you could just extract all the .o files into a big .a, as when it's finally linked into an executable, the linker will discard any unused ones anyway – Pinkney

@MarkR, @Wohlen wanted the size of the library to be minimal, not the size of the executable. – Pillow

Accepting it,though I would love a oneliner which does that. Like passing some options to ld to link with only the symbols needed etc... – Wohlen

@CID, the functionality is not thought necessary. When you link with a static library, it doesn't pull in everything from the library to make the executable. It brings in only the object files within the library to satisfy the undefined symbols. – Pillow

Hence the size of the executable is minimized to only what it needs. The link editor (ld) will only bring in the symbols needed. Your original question was about minimizing library size - if you really meant executable size, that's already being done for you. – Pillow

@Pax I meant library size only.. I was looking for some ar options similar to what a linker does for minimizing the executable size. I want a very thin combined library...with all non essentials stripped off. – Wohlen

Answer 2

A little used feature of the GNU archiver is the archive script, it is a simple but powerful interface, and it can do exactly what you want, for example if the following script is called script.ar:

CREATE libNewColor.a
ADDLIB libColor.a
ADDLIB libRGB.a
ADDLIB libPixel.a
SAVE
END

Then you could invoke ar as follows:

ar -M < script.ar

and you would get libNewColor.a that contains all of the .o files from libColor.a libRGB.a and libPixel.a.

Additionally you can also add regular .o files as well with the ADDMOD command:

CREATE libNewColor.a
ADDLIB libColor.a
ADDLIB libRGB.a
ADDLIB libPixel.a
ADDMOD someRandomCompiledFile.o
SAVE
END

Furthermore it is super easy to generate these scripts in Makefiles, so I typically create a somewhat generic makefile rule for creating archives which actually generates the script and invokes ar on the script. Something like this:

$(OUTARC): $(OBJECTS)
    $(SILENT)echo "CREATE $@" > $(ODIR)/$(ARSCRIPT)
    $(SILENT)for a in $(ARCHIVES); do (echo "ADDLIB $$a" >> $(ODIR)/$(ARSCRIPT)); done
    $(SILENT)echo "ADDMOD $(OBJECTS)" >> $(ODIR)/$(ARSCRIPT)
    $(SILENT)echo "SAVE" >> $(ODIR)/$(ARSCRIPT)
    $(SILENT)echo "END" >> $(ODIR)/$(ARSCRIPT)
    $(SILENT)$(AR) -M < $(ODIR)/$(ARSCRIPT)

Though now that I look at it I guess it doesn't work if $(OBJECTS) is empty (i.e. if you just want to combine archives without adding extra object files) but I will leave it as an exercise for the reader to fix that issue if needed... :D

Here are the docs for this feature:

https://sourceware.org/binutils/docs/binutils/ar-scripts.html#ar-scripts

Phosgene

This is great! Except that any paths/libs that have a + in the name break the script. For example openssl has libncurses++.a – Diurnal

Neat! This is way, way easier than unpacking and repacking object files yourself. – Histopathology

Note that for some reason -D does not work in combination with -M. So if you want to get deterministic results, you still need the extract-then-archive route. – Extraterritoriality

Answer 3

A static library is not much more than an archive of some object files (.o). What you can do is extract all the objects in the two libraries (using "ar x") and then use "ar" to link them together in a new library.

Lifework