perf record with --call-stack fp fails to unwind main function

Question

I have a C++ test program that keeps the CPU busy:

#include <cstdint>
#include <iostream>

// Linear-feedback shift register
uint64_t lfsr1(uint64_t max_ix)
{
    uint64_t start_state = 0xACE1u;  /* Any nonzero start state will work. */
    uint64_t lfsr = start_state;
    uint64_t bit;                    /* Must be 16-bit to allow bit<<15 later in the code */

    for (uint64_t ix = 0; ix < max_ix; ++ix)
    {   /* taps: 16 14 13 11; feedback polynomial: x^16 + x^14 + x^13 + x^11 + 1 */
        bit = ((lfsr >> 0) ^ (lfsr >> 1) ^ (lfsr >> 3) ^ (lfsr >> 4)) & 1 /* & 1u */;
        lfsr = (lfsr >> 1) | (bit << 15);
    }
    return lfsr;
}

int main() {
    std::cout << lfsr1(1717986914ull) << "\n";
}

I compile it with g++ -g -O3 -fno-omit-frame-pointer cpu.cpp -o cpu.bin, then run it with perf record -F 100 --call-graph fp -- ./cpu.bin and a second time with dwarf instead of fp.

In the perf script output for fp, I can see

cpu.bin 23435 1535437.021156:   42706947 cycles: 
            5617daf4b7a1 main+0x31 (…/cpu.bin)
            7f9a95088bf7 __libc_start_main+0xe7 (/lib/x86_64-linux-gnu/libc-2.27.so)
         3fe258d4c544155 [unknown] ([unknown])

whereas for dwarf, it's

cpu.bin 23443 1535441.101859:   42952079 cycles: 
            55a3b4ffd7a1 lfsr1+0x31 (inlined)
            55a3b4ffd7a1 main+0x31 (…/cpu.bin)
            7f00bcc8ebf6 __libc_start_main+0xe6 (/lib/x86_64-linux-gnu/libc-2.27.so)
            55a3b4ffd829 _start+0x29 (…/cpu.bin)

Seems like perhaps fp is off by one byte in __libc_start_main and that causes it to miss the last unwind step. How can this be fixed?

Petaliferous

I guess this is because lfsr1 is inlined (lfsr1+0x31 (inlined) in dwarf report). You can maybe try to declare it with __attribute__((noinline)). – Peeress

@Peeress I'm interested in the missing _start frame, not the missing lsfr1 frame. – Petaliferous

Is __libc_start_main compiled to use a traditional frame-pointer in your (distro's) build of glibc? I wouldn't be surprised if libc is compiled normally (-O2 without -fno-omit-frame-pointer), including that file, and probably that function doesn't have any special override for that. – Bagging

Probably yes. When I try on Ubuntu 20.04 instead of 18.04, I get no more [unknown], but no _start either. But when I install libc6-prof and use that, _start is there, and backtraces from libc functions work much better overall. – Petaliferous

Answer 1

Like Peter said in his comment, the problem resolves itself when a version of glibc with frame pointers is used. On Ubuntu 20.04, there is a package with such a glibc.

sudo apt install libc6-prof
# To use this library:
env LD_LIBRARY_PATH=/lib/libc6-prof/x86_64-linux-gnu perf record …

Then, the [unknown] is resolved to _start as expected.

Source: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1908307

Petaliferous