Problems with using gperftools on Mac OS X

Question

I have found several conflicting answers over this topic. This blog post requires libuwind, but that doesn't work on Mac OS X. I included #include <google/profiler.h> in my code, however my compiler (g++) could not find the library. I installed gperftools via homebrew. In addition, I found this stackoverflow question showing this:

Then I ran pprof to generate the output:

[hidden ~]$ pprof --text ./a.out cpu.profile 
Using local file ./a.out.
Using local file cpu.profile.
Removing __sigtramp from all stack traces.
Total: 282 samples
     107  37.9%  37.9%      107  37.9% 0x000000010d72229e
      16   5.7%  43.6%       16   5.7% 0x000000010d721a5f
      12   4.3%  47.9%       12   4.3% 0x000000010d721de8
...

Running that command (without any of the prior steps) gets me this:

[hidden]$ pprof --text ./a.out cpu.profile 
Using remote profile at ./a.out.
Failed to get the number of symbols from http://cpu.profile/pprof/symbol

Why does it try to access an internet site on my machine and a local file on his/hers?

Attempting to link lib profiler as a dry run with g++ gets me:

[hidden]$ g++ -l libprofiler
ld: library not found for -llibprofiler
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I have looked at the man pages, the help option text, the official online guide, blog posts, and many other sources.

I am so confused right now. Can someone help me use gperftools?

The result of my conversation with @osgx was this script. I tried to clean it up a bit. It likely contains quite a few unnecessary options too.

Illegal

Hi, user uo... Can you list all files of gperftools installed via homebrew (or just full log or make install)? How did you try link your program to be profiled with cpu profiler of gperftools library? (the blog missed this part, check goog-perftools.sourceforge.net/doc/cpu_profiler.html "Linking in the Library .. add -lprofiler to the link-time step for your executable. (It's also probably possible to add in the profiler at run-time using LD_PRELOAD, but this isn't necessarily recommended.)") Did you have cpu.profile file? – Lysol

Answer 1

The blog post https://dudefrommangalore.wordpress.com/2012/02/09/profiling-c-code-using-google-performance-tools/ "Profiling C++ code using Google Performance Tools" 2012 by dudefrommangalore missed the essential step.

You should link your program (which you want to be profiled) with cpu profiler library of gperftools library.

Check official manual: http://goog-perftools.sourceforge.net/doc/cpu_profiler.html, part "Linking in the Library"

add -lprofiler to the link-time step for your executable. (It's also probably possible to add in the profiler at run-time using LD_PRELOAD, but this isn't necessarily recommended.)

Second step is to collect the profile, run the code with profiling enabled. In linux world it was done by setting controlling environment variable CPUPROFILE before running:

CPUPROFILE=name_of_profile ./program_to_be_profiled

Third step is to use pprof (google-pprof in ubuntu world). Check that there is not-empty name_of_profile profile file generated; it there is no such file, pprof will try to do remote profile fetch (you see output of such try).

pprof ./program_to_be_profiled name_of_profile

Lysol

libuwind is not supported on Mac OS X. source – Caterwaul

I also could not resolve #include <google/profiler> – Caterwaul

Can you try to compile gperftools without libunwind? It is needed to readout call stack and to draw callgraph; without it it is possible to get flat profile (reading pc on profiling signal). You may use gpreftools without including google/profiler.h (the blog post is partially incorrect), just with preloading or linking with the lib, the lib wil turn on profiling in its init if the CPUPROFILE env var is set. Please, post full log of make install of gperftools. – Lysol

I'm a real noob to C++, could you please tell me which command is the link-time step? do I do it for gcc? gperftools installed successfully. do you still want a full log? by the way, thanks for the help so far! – Caterwaul

Please, add detail about your installation of gperftools on OSX ("I installed gperftools via homebrew"), and show the OSX version. I have no live OSX at the moment and can't reproduce the installation. I need to know what and where is installed to help you (to find exact library name and path to it). Compiling step has -c option and is like gcc ... -c -o file.o and linking step has executable file name in -o option, it is like gcc file.o file2.o -o executable (sometimes both are combined). Post linking command output with -v option added (verbose - it will print library search path). – Lysol

OS X El Captain v10.11.5 (latest version). I believe I've made some progress. Linking + running appears to have succeeded. Here is the linking command output and also the profiling output. It could find -lprofiler now for some magical reason. – Caterwaul

If this works, you could adapt it to this question I put a bounty on, I will award it to that answer – Caterwaul

Update: I couldn't upload the profiling to Github Gists ("contents can't be empty" even though it's non-empty), possibly because it starts with Unicode characters. On pastebin. Stripped unicode output: [a bunch of squares that get converted to nothing when I post them on any site, including here]'7fff9ce27000-7fffa296d000 r-xp 1b120000 00:00 0 /usr/lib/libDiagnosticMessagesClient.dylib – Caterwaul

upload "profiling"? What is it? Please, don't upload library itself. If you were able to link, run CPUPROFILE=profilefile ./your_program (program should run longer than 0.5s), check that profilefile is generated and not empty and run pprof on it. The "this question" was not your, so no. Posted gist "CrazyPython/f067eaa0172621bcb5f4c13fe3906b51" is not linking step, it is compiling (-emit-obj -o ..file.o) by clang; I need linking log. What was uploaded to pastebin? It is /proc/pid/maps only... (part of profile) – Lysol

Let us continue this discussion in chat. – Caterwaul

Answer 2

First you need to run your program with profiling enabled.

This is usually first linking your program with libprofiler and then running it with CPUPROFILE=cpu.profile.

I.e.

$ CPUPROFILE=cpu.profile my_program

I think that later step is what you have been missing.

The program will create this cpu.profile file when it exits. And then you can use pprof (preferably from github.com/google/pprof) on it to visualize/analyze.

Hoenack

First is to inject cpu profiling library inside the program. Either by linking it directly with profiling library or by using LD_PRELOAD (how it is called in Darwin/OSX?). And running is the second step, CPUPROFILE env var will enable profiling, only if the library is linked into program. – Lysol

Answer 3

When I compile the program under Mac=Os, I also failed by using -lprofiler. Then I change to -Lprofiler, it works. But I still can not get the cpu.profile by using CPUPROFILE=cpu.profile my_program

Swanhilda