Wrong line numbers from addr2line

Question

I try to find the exact line of a call in the backtrace in C++ program. Right now I am using these lines (from the man page of backtrace) to get the trace:

  void *bt_buffer[1000];
  char **bt_strings;
  int bt_nptrs = backtrace(bt_buffer, 1000);
  bt_strings = backtrace_symbols(bt_buffer, bt_nptrs);

In bt_strings I find lines of the form

./prog() [0x402e42]

Now I take the address (the hex string) and feed it to addr2line. That sometimes results in apparently wrong line numbers. Internet search led me to this post, in which it is shown that

readelf -wl ./prog

indicates where the line really is, or rather how many lines the symbol has moved to the current line.

edit: This happens when I compile with -g -O0, i.e. explicetly without optimisations. The compiler is gcc 4.6.3 Is there another compiler flag that I miss?

My problem is the following: I need to automate this. I need my program to create a backtrace (done), extract the file (done) and the line number (fail).

I could of course call readelf and parse the output, but that's not really suitable, as the output differs from symbol to symbol depending on what exactly happened. Sometimes The address of a symbol is in one line and the information about the line offset in the next line...

To sum up:

Is there an elegant way to get the exact line number of a function call in the backtrace from within the program during runtime?

edit: example code:

#define UNW_LOCAL_ONLY
#include <libunwind.h>
#include <execinfo.h>
#include <iostream>
#include <stdlib.h>

void show_backtrace()
{
  // get current address
  void* p = __builtin_return_address(0);
  std::cout << std::hex << p << std::endl;

  // get callee addresses
  p = __builtin_return_address(1);
  std::cout << std::hex << p << std::endl;

  p = __builtin_return_address(2);
  std::cout << std::hex << p << std::endl;
}

void show_backtrace2()
{
  void *array[10];
  size_t size;
  char **strings;
  int i;

  size = backtrace (array, 10);
  strings = backtrace_symbols ((void *const *)array, size);

  for (i = 0; i < size; i++)
    {
      std::cout << strings[i] << std::endl;
    }

  free (strings);
}

void show_backtrace3 (void)
{
  char name[256];
  unw_cursor_t cursor; unw_context_t uc;
  unw_word_t ip, sp, offp;

  unw_getcontext (&uc);
  unw_init_local (&cursor, &uc);

  while (unw_step(&cursor) > 0)
    {
      char file[256];
      int line = 0;

      name[0] = '\0';
      unw_get_proc_name (&cursor, name, 256, &offp);
      unw_get_reg (&cursor, UNW_REG_IP, &ip);
      unw_get_reg (&cursor, UNW_REG_SP, &sp);

      std::cout << std:: hex << name << " ip = " << (long) ip 
                << " , sp = " << (long) sp << std::endl;
    }
}

void dummy_function2()
{
  show_backtrace();
  show_backtrace2();
  show_backtrace3();
}

void dummy_function1()
{ 
  dummy_function2 (); 
} // line 73

int main(int argc, char **argv)
{
  dummy_function1 ();
  return 0;
}

compile and run:

g++ test_unwind.cc -g -O0 -lunwind  && ./a.out

output:

0x400edb
0x400ef0
0x400f06
./a.out() [0x400cfb]
./a.out() [0x400ee0]
./a.out() [0x400ef0]
./a.out() [0x400f06]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7f2f044ae76d]
./a.out() [0x400b79]
_Z15dummy_function2v ip = 400ee5 , sp = 7fffdb564580
_Z15dummy_function1v ip = 400ef0 , sp = 7fffdb564590
main ip = 400f06 , sp = 7fffdb5645a0
__libc_start_main ip = 7f2f044ae76d , sp = 7fffdb5645c0
_start ip = 400b79 , sp = 7fffdb564680

testing e.g. 0x400ef0 with addr2line yields

/path/to/code/test_unwind.cc:73

which is the correct file, but wrong line number. In real life applications the line number can differ by many lines, forward and backward.

edit: compiling with -S shows that the relevant part is:

.LCFI34:
  .cfi_def_cfa_register 6
  .loc 2 72 0
  call  _Z15dummy_function2v
  .loc 2 73 0
  popq  %rbp

What is displayed by addr2line and alike is the return address, as shown in the line after the call. I would like to get the "entry" line, i.e. what is shown before!

Educate

Does it get it wrong when compiled without optimisations? – Kalbli

Yes forgot to mention that... I will edit the post – Educate

Try gdbing the executable and use l *0x<address>. Does it give the correct address, or does it also give the same as addr2line? – Chiliasm

@Shahbaz: gdb and l *0x... gives the same wrong line number. – Educate

@steffen, from the man page of backtrace, first of all, it seems like ./prog() [0x402e42] is impossible. You either have in the form of ./prog [0x402e42] or ./prog(func+0x5c) [0x402e42]. The first form is for static functions. Also from the man page, you need the -rdynamic option to gcc during link. – Chiliasm

Well, i get the message with the empty parentheses. Maybe because I call backtrace from a dynamically linked library. With -rdynamic I get e.g. ./prog(main+0x1cd) [0x40b502]. Does that help me in any way? – Educate

@steffen, well, doesn't the new address reported point to the correct line? – Chiliasm

@Shahbaz: As far as I understand, this is an offset from main. If I count bytes from where main appears in the code, I don't get to the right line. Maybe I have to not count blank characters (like indents) and comments? Or how do I get the address of main? – Educate

Did you notice that the address in your original question (0x402e42) is different from the one in your previous comment (0x40b502)? Did you try addr2line with this new reported address? And no you don't need to skip any blank lines. – Chiliasm

@Shahbaz: True, but that must be due to recompiling. I still get the wrong line number. – Educate

Is there any other gdb trick to apply? – Educate

@steffen: just to be clear, a function+offset value like 'main+0x1cd' indicates the location in the executable code that is 461 bytes after the label 'main'. It has no direct relationship to lines or characters in the source code of the program. – Spheno

@antonm: Yep, that's what I assumed. But it makes no sense... For the code I posted I get main+0x14 for the function call of dummy_function1. 20 bytes after main I am in the middle of main's arguments. And if I insert blank spaces and lines, the 0x14 does not change... – Educate

Maybe a stupid comment but.. isn't line 73 the actual return address? I.e. where to return to when the call is done? In other words the address pushed to the stack? – Triode

Have you also tried to compile it with assembly output -S to see what assembly code corresponds to what line? That helped me with a similar problem (in Windows). – Triode

@Qiau: You're right, it is the return address. Now at least I understand where the number comes from. But it is still hard to tell, where the function call was. In a real world code, the displayed line could be at the for loop condition... – Educate

I couldn't test it so maybe the results would be the same you got so far, but GCC's -finstrument-functions option allowed me to obtain information about the calling line. I found some information about it here: codingrelic.geekhold.com/2010/09/… – Arlinda

@dhekir: Gee, I know the option and actually use it in another context, but it didn't occur to me to apply it here! I will try if it works. – Educate

Have you tried libbacktrace from the GCC source tree ? – Logarithmic

@user416983: isn't that what backtrace in execinfo.h does/uses? – Educate

Needing this too here, I stumbled over this question. I only need it from time to time by hand, but got good results by subtracting 1 from the address before feeding it to addr2line, this seems to make it hit the call instructions. Did you come to a solution maybe that you would like to post here too? – Scrunch

@PlasmaHH: I haven't worked on this for a while... If you subtract 1 from the adress, it will work, if the next statement to be executed is actually on the next line. This is not the case if the function you are trying to locate is the last one in a for loop, for instance. – Educate

Answer 1

This is because on most architectures, the program counter points to an instruction after the current executing instruction. You can still use addr2line, but add -1 for x86 or -8 for aarch32 to get the actual address of the source code line. Backward CPP uses the same idea. https://github.com/bombela/backward-cpp/blob/master/backward.hpp (search for ip_before_instruction). I found the only time to not subtract would be if the address is where a crash happened. This makes sense because after calling the fault handler (e.g. to service a page fault), it could resume executing the faulting instruction.

Agamogenesis

Great sum up! Thanks – Thereabout

Answer 2

You sure can do! I know of an example implementation which uses libunwind. See this blog post: http://blog.bigpixel.ro/stack-unwinding-stack-trace-with-gcc/

It comes down to this piece of code (literally copied from the article):

void show_backtrace (void)
{
    char name[256];
    unw_cursor_t cursor; unw_context_t uc;
    unw_word_t ip, sp, offp;

    unw_getcontext(&uc);
    unw_init_local(&cursor, &uc);

    while (unw_step(&cursor) > 0)
    {
        char file[256];
        int line = 0;

        name[0] = '\0';
        unw_get_proc_name(&cursor, name, 256, &offp);
        unw_get_reg(&cursor, UNW_REG_IP, &ip);
        unw_get_reg(&cursor, UNW_REG_SP, &sp);

        //printf ("%s ip = %lx, sp = %lx\n", name, (long) ip, (long) sp);
        getFileAndLine((long)ip, file, 256, &line);
        printf("%s in file %s line %d\n", name, file, line);
    }
}

Illuminative

I'm excited to try that as soon as I get to the office! – Educate

I tried it and found that in getFileAndLine they also use a popen call to addr2line. libunwind is interesting (+1 for that) but has similar functionality as backtrace. The result is the same offset line numbers. – Educate

In your question you said you're compiling with -g, maybe you should use -ggdb instead? – Illuminative

Thanks for the hint. I tried with all the debug symbol flags available and the result is the same. – Educate

is there any way to get file and line number without using popen and addr2line utility? – Salify

Answer 3

Have you tried

__LINE__

It is a preprocessor symbol, but you can compile it in.

Scrap

I have thought about __FILE__ and __LINE__ as well, but I want to get the information of foo() from a function which foo() calls. And I cannot change how foo() is called. – Educate