I'm using 'backtrce()' and 'backtrace_symbols_fd()' functions in a signal handler to generate a backtrace for debugging (GDB not available).
They work fine on x86 desktop (Ubuntu), but on the target device (ARM based) the backtrace on Abort signal (due to double-free error) shows only three frames: the signal handler and two from within libc, which is not useful for debugging our code! Backtrace on SEGV (e.g. using a bad pointer) DOES produce a good backtrace.
Why can't I get a useful backtrace on ABRT signal on ARM?
[Question edited for clarity]
Here's a simple test program which demonstrates the problem:
#include <execinfo.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
// Signal hangler to catch seg fault:
void handler_segv(int sig) {
// get void*'s for all entries on the stack
void *array[10];
size_t size;
size = backtrace(array, 10);
fprintf(stderr, "Error: Signal %d; %d frames found:\n", sig, size);
// print out all the frames to stderr
backtrace_symbols_fd(array, size, STDERR_FILENO);
exit(1);
}
void crashme()
{
// Deliberate Error: Abort (double free):
char *test_ptr = malloc(1);
free(test_ptr);
free(test_ptr);
// Deliberate Error #2: Seg fault:
//char * p = NULL;
//*p = 0;
}
void foo()
{
fprintf(stdout, "---->About to crash...\n");
crashme();
fprintf(stdout, "---->Crashed (shouldn't get to here)...\n");
}
// Main entry point:
int main(int argc, char *argv[])
{
fprintf(stdout, "Application start...\n");
// Install signal handlers:
fprintf(stdout, "-->Adding handler for SIGSEGV and SIGABRT\n");
signal(SIGSEGV, handler_segv);
signal(SIGABRT, handler_segv);
fprintf(stdout, "-->OK. Causing Error...\n");
foo();
fprintf(stdout, "-->Test finished (shouldn't get to here!)\n");
return 0;
}
This was compiled for x86 as follows:
gcc -o test test-backtrace-simple.c -g -rdynamic
And for ARM:
arm-none-linux-gnueabi-gcc -o test-arm test-backtrace-simple.c -g -rdynamic -O0 -mapcs-frame -funwind-tables -fasynchronous-unwind-tables
I've used various compiler options for ARM as described in other posts related to generating backtraces on ARM.
When run on the x86 desktop, it generates the expected output with plenty of debug, ending in:
Error: Signal 6; 10 frames found:
./test(handler_segv+0x19)[0x80487dd]
[0xb7745404]
[0xb7745428]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb75b0e0f]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb75b4455]
/lib/i386-linux-gnu/libc.so.6(+0x6a43a)[0xb75ed43a]
/lib/i386-linux-gnu/libc.so.6(+0x74f82)[0xb75f7f82]
./test(crashme+0x2b)[0x8048855]
./test(foo+0x33)[0x804888a]
./test(main+0xae)[0x8048962]
(i.e. the back trace generated by my handler, with my function calls at the bottom).
However, when run on the ARM platform, I get:
Application start...
-->Adding handler for SIGSEGV and SIGABRT
-->OK. Causing Error...
---->About to crash...
*** Error in `/opt/bin/test-arm': double free or corruption (fasttop): 0x015b6008 ***
Error: Signal 6; 3 frames found:
/opt/bin/test-arm(handler_segv+0x24)[0x8868]
/lib/libc.so.6(__default_sa_restorer_v2+0x0)[0xb6e6c150]
/lib/libc.so.6(gsignal+0x34)[0xb6e6af48]
The backtrace() finds only 3 frames, and they are only the signal handler and something in libc (not useful)!
I found a mailing list post which said:
If you link with the debugging C library, -lc_g, you'll get debugging info back past abort().
This might be relevant, but -lc_g doesn't work on my compiler (ld: cannot find -lg_c).
The backtrace works fine on ARM if I generate a seg fault instead (e.g. change crashme() function to use "char *p = NULL; *p = 0;" instead of the double free.
Any ideas or suggestions for other ways to get a back trace?
[--EDIT--]
I tried some MALLOC_CHECK_ options as suggested in the comments, but the only effect was to change whether the abort was generated. Here is the output from three runs on the ARM:
# MALLOC_CHECK_=0 /opt/bin/test-arm
Application start...
-->Adding handler for SIGSEGV and SIGABRT
-->OK. Causing Error...
---->About to crash...
---->Crashed (shouldn't get to here)...
-->Test finished (shouldn't get to here!)
# MALLOC_CHECK_=1 /opt/bin/test-arm
Application start...
-->Adding handler for SIGSEGV and SIGABRT
-->OK. Causing Error...
---->About to crash...
*** Error in `/opt/bin/test-arm': free(): invalid pointer: 0x015b2008 ***
---->Crashed (shouldn't get to here)...
-->Test finished (shouldn't get to here!)
# MALLOC_CHECK_=2 /opt/bin/test-arm
Application start...
-->Adding handler for SIGSEGV and SIGABRT
-->OK. Causing Error...
---->About to crash...
Error: Signal 6; 3 frames found:
/opt/bin/test-arm(handler_segv+0x24)[0x8868]
/lib/libc.so.6(__default_sa_restorer_v2+0x0)[0xb6e24150]
/lib/libc.so.6(gsignal+0x34)[0xb6e22f48]
#
MALLOC_CHECK_=0: No error message (double free is ignored!)
MALLOC_CHECK_=1: Error message, but program continues
MALLOC_CHECK_=2: Error message and ABRT signal; useless backtrace generated (this is the default behaviour!)
My cross compiler reports: gcc version 4.6.1 (Sourcery CodeBench Lite 2011.09-70) Target device has linux kernel version 3.8.8
fprintf(stderr, "Error: Signal ...: you know thatprinf()and friends are not signal-safe ? – Terzas