How do I benchmark or trace a specific function in the Linux Kernel?

Question

How do I use ftrace() (or anything else) to trace a specific, user-defined function in the Linux kernel? I'm trying to create and run some microbenchmarks, so I'd like to have the time it takes certain functions to run. I've read through (at least as much as I can) the documentation, but a step in the right direction would be awesome.

I'm leaning towards ftrace(), but having issues getting it to work on Ubuntu 14.04.

Mislike

Answer 1

Ftrace is a good option and has a good documentation.

Sinewy

Answer 2

Here are a couple of options you may have depending on the version of the kernel you are on:

Systemtap - this is the ideal way check the examples that come with the stap, you may have something ready with minimal modifications to do.

Oprofile - if you are using older versions of the kernel, stap gives better precision compared to oprofile.

debugfs with stack tracer option - good for stack overrun debugging. To do this you would need to turn on depth checking functions by mounting debugfs and then echo 1 > /proc/sys/kernel/stack_tracer_enabled.

strace - if you are looking at identifying the system calls being called by the user space program and some performance numbers. use strace -fc <program name>

Hope this helps!

Danish

Answer 3

use WARN_ON() It will print some trace of function called that.

For time tracing i think you should use time stamp showing in kernel log or use jiffies counter

Aceves

Answer 4

Also systemtap will be useful in your situation. Systemtap is some kind of tool in which you can write code like in scripting languages. It is very powerful, but if you want to only know a time of execution particular function ftrace would be better, but if you need very advanced tool to analyze e.g, performance problems in the kernel space, it may be very helpful.

Pls read more: (what you want to do is here:- 5.2 Timing function execution times) enter link description here

Amal

Answer 5

If the function's execution time is interesting because it makes subsidiary calls to slow/blocking functions, then statement-by-statement tracing could work for you, without too much distortion due to the "probe effect" overheads of the instrumentation itself.

probe kernel.statement("function_name@dir/file.c:*") { println(tid(), " ", gettimeofday_us(), " ", pn()) }

will give you a trace of each separate statement in the function_name. Deltas between adjacent statements are easily computed by hand or by a larger script. See also https://sourceware.org/systemtap/examples/#profiling/linetimes.stp

Locklin

Answer 6

To get the precision that I needed (CPU cycles), I ended up using get_cycles() which is essentially a wrappeer for RDTSC (but portable). ftrace() may still be beneficial in the future, but all I'm doing now is taking the difference between start CPU cycles and end CPU cycles and using that as a benchmark.

Update: To avoid parallelization of instructions, I actually ended up wrapping RDTSCP instead. I couldn't use RDTSC + CPUID because that caused a lot of delays from hypercalls (I'm working in a VM).

Mislike

Answer 7

Use systemtap and try this script: https://github.com/openresty/stapxx#func-latency-distr

Avocado