Symbol hiding in static libraries built with Xcode

Question

I'm trying to figure out whether I can build a static library that hides all of its internal objects and functions, etc, except for the interfaces I want to export. I'm experimenting with Xcode (gcc 4.2).

I've used the __attribute__((visibility("hidden"))) attribute on some C++ classes per this documentation. I've also defined little helper C functions as being file-local (static), etc.

However, when I run strings on the resulting .a library file, even when compiled in Release configuration, I still see the names of my ostensibly-hidden classes, with their method names, and even the names of file-local functions strewn around in there as well.

I've added the -fvisibility=hidden and even -fno-rtti to the gcc flags. While this reduces some of the strings, the class names, method names, and static functions names are all still in there in plain or mangled-but-readable form.

Is there a reliable way to get the compiler to build this stuff without having the string names of all the internal stuff emitted into the binary? It shouldn't be necessary to have for any external clients linking in.

(To clarify: I'm asking about obfuscation of internal naming, versus literal export binding needs. I'm disconcerted that all the internal workings are visible via the strings command, regardless of whether these symbols are formally exported or not.)

Thanks.

Mymya

If RTTI is enabled, all class names are going to be embedded no matter what, in order to make type_info::name() work. You could try recompiling with -fno-rtti and see if it makes a difference. Of course, if you are using dynamic_cast anywhere, this won't work. – Coffman

Thanks for the hint. Tried this, but all the names are still in there in mangled form. – Mymya

You should really think about re-awarding the answer as the one by bleater requires no trickery. Not only did it remove most of my names, but it greatly reduced the size of the library I need to share. – Naturalist

@DavidH: Done, thanks for the heads-up. Hadn't revisited this question in a while. – Mymya

Answer 1

Hiding internal names requires a few simple Xcode build settings, and it is not generally necessary to modify source or change the type of the built product.

1. Eliminate any internal symbols required between modules by performing a single-object prelink. Set the Xcode build setting named "Perform Single-Object Prelink" to Yes (GENERATE_MASTER_OBJECT_FILE=YES). This causes ld to be run with the -r flag.
2. Make sure that the setting "Strip Style" is set to "Non-global symbols" (STRIP_STYLE=non-global), this passes -x to ld.
3. Stripping is only actually performed on static libraries if post-processing is enabled (and this is not the default). Set Xcode build setting "Deployment Postprocessing" to yes. (DEPLOYMENT_POSTPROCESSING=YES). Also make sure that "Use separate strip" is set to Yes (not always the default) (SEPARATE_STRIP=YES).
4. If, in addition to local symbols, if you need to remove some of the global symbols you can supply the -unexported_symbols_list file command (where file is the path to a text file of symbols to remove, one per line) or the -unexported_symbol symbol command (where symbol is a single symbol name to remove) to the single-object prelink step using the Xcode build setting "Single-Object Prelink Flags" (i.e. PRELINK_FLAGS). Don't forget to use the decorated symbol name; for example the symbol for a C function will be the function name prefixed with the _ character.
   (Alternately, this can also be done using additional options to the strip command, under the Xcode build setting "Additional strip flags", e.g. -R file option which has the same effect as ld's unexported_symbols_list option).

Cordwainer

Excellent! I just did as you suggested with Xcode 5.1 Beta, and it worked perfectly. Just what I needed. – Naturalist

I would suggest to add 5. to the above list: Set "SymbolsHiddenByDefult" to YES. (It is in the Code Generation section). It is equivalent to -fvisibility=hidden. The step 1..4 helped me to remove strings but step 5. hides also the non public methods, static variables and buffers. – Frohne

@Frohne The OP's question stated that he had already controlled symbol visibility, but yes, your suggestion is a good idea for other cases where explicit control has not yet been put in place. It does pay to test that the final product can be correctly linked (into a dynamic section or executable) however, since there is the possibility that callers might be inadvertently linking to private symbols. – Cordwainer

"Use separate strip" is no longer an option -- deprecated by Apple with the claim that stripping is always done separately. Despite that, I still got some mileage out of this post by doing the rest -- appeared to work. – Songster

Answer 2

The main trick in hiding symbols within static libraries is to generate a Relocatable Object file (as opposed to a static library archive that simply consists of a collection of individual .o files). To build a relocatable object file, you need to choose your target in XCode as Bundle (as opposed to "Cocoa Touch Static Library"). The Bundle target appears under the OS X templates, and you can set its target to iOS in the Build settings if you are building for iOS.

Once you have set up your target correctly, the following steps will help get the symbol hiding correct:

1. Set the "Symbols hidden by default" option to Yes in the build settings. This makes sure all the symbols compiled in the files are marked as private.

2. As this is a library, you do need to keep some symbols public. You should put code for the functions you want to keep publicly visible in separate files, and compile those files with the -fvisibility=default flag (you can set this flag for individual files "Build Phases > Compile Sources > -- Compiler Flags" in Xcode). Alternately, you can prefix the name of the function/class that you wish to be visible with the __attribute__((visibility("default"))) directive.

3. Under the linking settings in the X-code project, set the Mach-O type to "Relocatable Object File". What this means is that all the .o files will be relinked to generate a single object file. It is this step that helps mark all the symbols as private when the .o files are linked together into one file. If you build a static library (i.e. a .a file) this relinking step doesn't happen so symbols never get hidden. So choosing a Relocatable object file as your target is critical.

4. Even after marking the symbols as private they still show up in the .o file. You need to enable stripping to get rid of the private symbols. This can be done by setting the "Stripped Linked Product" setting to Yes in the build settings. Setting this option runs the strip -x command on the object file that removes the private symbols from the object file.

5. Double check all the internal symbols are gone by running the nm command on the final relocatable object file generated by the build process.

The above steps will help you get rid of symbol names from the nm command. You'll still see some function names and file names if you run the strings command on your object file (due to some strings and object names being compiled in via exceptions). One of my colleagues has a script that renames some of these symbols by looking into the binary sections and renaming those strings. I've posted it up here for you to use: https://gist.github.com/varungulshan/6198167. You can add this script as an extra build step in Xcode.

Lydie

Thanks for the answer, we did it but now we cannot run it on a simulator, it doesn't link. it only works on the device. The error we are getting is: ld: Assertion failed: (atom->fixupCount() == 1), function targetClassName, file /SourceCache/ld64/ld64-136/src/ld/parsers/macho_relocatable_file.cpp, line 4973. clang: error: linker command failed with exit code 1 (use -v to see invocation). Did you encounter this issue? Thanks – Dennie

Hi Yoav, Sorry for the late reply. No, I never tried this on the simulator, so it is possible it breaks linking there. One of our engineers has come up with an even better method (he wrote to script to remove all locally visible symbols by scanning the binary sections). I'll post it soon. – Lydie

@VarunGulshan would take time to my question? Thank you #56045795 – Nowt

Answer 3

It is a bit unclear for me how to hide the symbols in static libraries from the linux command line environment based on the previous answers so I'll just post my solution here for posterity (given this is one of the top results on google for that question).

Let's say you have these two .c files:

// f1.c
const char *get_english_greeting(void)
{
  return "hello";
}

__attribute__((visibility("default")))
const char *get_greeting(void)
{
  return get_english_greeting();
}

and

// f2.c
#include <stdio.h>
const char *get_english_greeting(void);

__attribute__((visibility("default")))
void print_greeting(void)
{
  puts(get_english_greeting());
}

You want to convert these two files into a static library exporting both get_greeting and print_greeting but not get_english_greeting which you don't want to make static as you would like to use it throughout your library.

Here are the steps to achieve that:

gcc -fvisibility=hidden -c f1.c f2.c
ld -r f1.o f2.o -o libf.o
objcopy --localize-hidden libf.o
ar rcs libf.a libf.o

Now this works:

// gcc -L. main.c -lf
void get_greeting(void);
void print_greeting(void);
int main(void)
{
  get_greeting();
  print_greeting();
  return 0;
}

And this doesn't:

// gcc -L. main.c -lf
const char *get_english_greeting(void);
int main(void)
{
  get_english_greeting();
  return 0;
}

For the latter you get this error:

/tmp/ccmfg54F.o: In function `main':
main.c:(.text+0x8): undefined reference to `get_english_greeting'
collect2: error: ld returned 1 exit status

Which is what we want.

Note that the hidden symbol names are still visible in the static library but the linker will refuse to link with them outside said static library. To completely remove the symbol names you'll need to strip and obfuscate.

Munch

how do I remove the symbol name( get_english_greeting) away from the static library ? I tried "strip -x libf.a" "objcopy -N get_english_greeting libf.a" but none of them worked. – Exhaustion

@prgbenz, I'm not sure how to do that. I tried running strip -N get_english_greeting libf.a but I got strip: not stripping symbol 'get_english_greeting' because it is named in a relocation back. I think this suggests that you need to find a way to pre-link libf.o to have it pre-resolve the relocations. I haven't found any way to do that. Besides, what would be the benefit of doing this? – Munch