Question about overriding C standard library functions and how to link everything together

Question

I made my own implementation of _init , malloc , free ( and others ).

Inside these functions I use the dlfcn.h (dlopen , dlsym etc) library to call the actual standard versions. I put then in a single file and compile them as a shared library ( memory.so ). When I wish to run an executable and make it call my versions of these functions I simply set LD_PRELOAD=memory.so .

The problem is that I have a number of other modules which memory.c depends on. These include a file containing functions to scan elf files ( symbols.c ) and my own implementation of a hash table ( hashtable.c ) which I use to keep track of memory leaks among others.

My question is if there is a way to separately compile hashtable.c & symbols.c so any malloc references are resolved with the standard library and not with the ones included on memory.c. I could of course use the dlfcn.h libraries on everything that memory.c depends on but I would prefer it if there was a way to avoid that.

I still haven't completely figured out how linking works so any help would be appreciated.

Thank you

Tutankhamen

Why did you write your own version of malloc and free? What's wrong with the ones in stdlib.h? – Chavez

This is an assigment, I need my own versions to catch memory leaks , check parameters passed to memcpy etc for overlapping memory regions. I also have to be able to dynamically link it against any executable. – Tutankhamen

Answer 1

If you are working with glibc you can use alternative non-overriden function names:

[max@truth ~]$ nm --defined-only --dynamic /lib64/libc.so.6 | egrep "malloc\b"
0000003e56079540 T __libc_malloc
0000003e56079540 T malloc

Note the same function address in the above. In other words, malloc() function is given two names, so that the original malloc() version is available under __libc_malloc() name in case malloc() has been interposed.

A quick grep on glibc sources reveals the only caller of __libc_malloc() is mcheck. These function aliases are a glibc implementation detail and there is no header for them. malloc/mcheck.c declares the internal functions as below:

extern __typeof (malloc) __libc_malloc;
extern __typeof (free) __libc_free;
extern __typeof (realloc) __libc_realloc;

Other C libraries may have differently named aliases, so using dlsym() to get the original malloc() address is more portable.

Concordat

Hello and thank you, could you please explain this a bit more. I see there are 2 different functions with the same function address. Does this simply means that when I write my own malloc , I am actually overriding malloc instead of __libc_malloc ? – Tutankhamen

Answer 2

First it is important to note there is no need to do what you want to do for memory debuggers in Linux as glibc provides specific hook functions for memory functions (see: http://www.gnu.org/s/libc/manual/html_node/Allocation-Debugging.html)

But disregarding this, the general solution is to NOT use dlopen() to get a reference to glibc for dlsym() but rather use the magical handle RTLD_NEXT. Here is the relevant part from the dlopen() man page:

"There are two special pseudo-handles, RTLD_DEFAULT and RTLD_NEXT. The former will find the first occurrence of the desired symbol using the default library search order. The latter will find the next occurrence of a function in the search order after the current library. This allows one to provide a wrapper around a function in another shared library."

See for example: http://developers.sun.com/solaris/articles/lib_interposers_code.html

Anticipant

Hello and thanks, so If I used malloc hooks and indirectly called malloc using a reference returned by dlsym() would the malloc hook still be used ? Thanks – Tutankhamen

Yes, but you can use a static flag variable to see that you are reentrant and do the right thing (return from the malloc hook in the inner call) – Anticipant

Answer 3

You could take a look at electric fence. It overrides a lot of standard functions to do various memory debugging.

Kakapo