How to apply -fvisibility option to symbols in static libraries?

Question

I have a shared library project that is built from 4 static libraries (.a) and one object (.o) file. I am trying to add the -fvisibility=hidden option to restrict symbols in the output to only those that I mark in the source with an __attribute__.

I've added the -fvisibility=hidden option to the compile options for the .so project (which covers the .o file) and for the .a projects.

The symbols in the object file are removed as expected from the final .so. However the symbols from the .a projects are all still in the final .so file. Adding the -fvisibility=hidden option to the .so link command has no effect.

What am I doing wrong?

My purpose here is to remove from the .so all symbols except the interface functions to the library.

EDIT: I actually used a version map to solve this for now. However it requires continued maintenance of the version script as external symbols change. Accepted answer has a better idea.

Intermediate

Platform not mentioned, but for a solution for doing this under iOS/OS X see https://mcmap.net/q/323516/-symbol-hiding-in-static-libraries-built-with-xcode – Firehouse

Answer 1

Basically, visibility is handled during linking, and the linker doesn't seem impose it on static archives. A related question (though not a duplicate) was asked on SO here.

What I would advise you to do is to replace your linking stage: gcc -shared -o mylib.so foo.o libbar.a into a two stages process where you get back the object files:

• ar x libbar.a (possibly into a suitable, empty directory)
• gcc -fvisibility=hidden -shared -o mylib.so foo.o tempdir/*.o

Laniary

Nice idea. Fits with what I learned by experimentation. May try this later. – Intermediate

I just would like to clarify where is the correct place to add -fvisibility=hidden. According to the GCC Visibility Wiki under the section How to use the new C++ visibility support it is stated that alter your make system to pass -fvisibility=hidden to each call of GCC compiling a source file.. So I was wondering if you really need to set the visibility flag when building the so object. – Valency

What will be difference between -fvisibility=hidden and -s? – Reamonn

Would it be possible to reconstruct the archive without the exports? Basically getting back a static library ".a" from "*.o" that will not pull in exports when linking in a shared library? – Claiborne

Answer 2

Simply pass -Wl,--exclude-libs,ALL to gcc

This will tell the linker to transform all the symbols in the static libraries to hidden.

--exclude-libs also accepts a list of archives (i.e. static library names) for finer granularity on which libraries to hide symbols from.

Note: this will only work in systems using GNU binutils (e.g. Linux) or with a linker supporting --exclude-libs (e.g. it won't work with OSX's ld64)

Husein

Thanks you very much! I use this together with -fvisibility=hidden and selectively export only the public API (IMO only the whitelisting approach is sane specially when it comes to static libraries external to the project). Maybe if you can add some tags future Googler's will be thankful. – Nonunion

I believe it's -Wl,--exclude-libs=ALL (replace the second comma with an =) – Ludwog

Unfortunately this doesn't work on Mac OS since ld doesn't know this option under Mac OS. – Sefton

Also, --exclude-libs is only applicable on static libs not on dynamic libraries. – Tree

@ManishShukla The question is specifically about static libraries – Husein

What I observed is that this will not reliably exclude symbols if compiling with link-time optimization. – Annisannissa

This helped me so much, I spent lot of time trying to hide unnecessary symbols also dealing with compiler bugs which don't set visibility correctly in some cases. But this together with visibility=hidden just fixes things and makes it work as expected. I'm not totally sure why this is not default behavior for shared libraries. – Patman

This answer is very good but the fact the switch is currently not available in MacOSX is a big limitation. Sometimes 3rdparty static dependencies pull in tons of unwanted symbols and often compiling them without exports requires source code modifications. Would it be possible to just strip exported symbols from a static library archive (.a)? If so, how? – Claiborne

@Husein I am trying this method, it made a noticeable improvement. However, not all symbols from the static library have gone away. I wonder if header files associated with the static library must also be compiled with -fvisibility=hidden – Jerry

@AlexeyB. same here. In my case I didn't have any LTO enabled (that I know of), but I'm still seeing some of the symbols from the static lib being exported through the dynamic lib. I had to use the linker version script in order to hide them completely. – Subglacial

Answer 3

This is an answer to the problem for OS X.

The Mac ld doesn't support --exclude-libs, but it does support -exported_symbol sym and it does apply this to object files in static libraries. And when you're filtering to a public API, the whitelist is small enough to spell it out.

I ended up with the following in my Makefile to generate a -Wl,-exported_symbol,_api_func_1 flag for each exported symbol:

SYMBOLS   = api_func_1 api_func_2 api_func_3 api_func_4
SYMBOLS   += api_func_5 # add more as necessary
COMMA     = ,
LDFLAGS   += $(addprefix -Wl$(COMMA)-exported_symbol$(COMMA)_,$(SYMBOLS))

# ...

libmyapi.so: # ...
    $(CC) -shared -o $@ ... $(LDFLAGS)

Then you can if-gate between this version of the flags and the GNU ld version after detecting which linker the system has.

Amide

@Mecki What's the difference then between the normal -l and -hidden-l? – Gasbag

@Gasbag -l exports symbols based on their visibility. OP said he made all symbols hidden by default with -fvisibility=hidden, so -l will not export them by default. If you want to export them, you need to use -reexport-l, which will always export symbols, regardless of visibility. The counterpart is -hidden-l, which will never export symbols, regardless of visibility. I admit my comment is poorly phrased, sorry for that. – Wolff

So on macOS, to avoid re-exporting a static library's default-visibility symbols from the executable linking to the library, -hidden-l should be used? (assuming no manual -exported_symbols and the like are specified) – Gasbag

@Danra: -hidden-l is a step in the right direction, but does not look like it supports ALL like --exclude-libs? – Decapod