How to count the number of guest instructions QEMU executed from the beginning to the end of a run?

Question

I want to benchmark guest instructions per second of QEMU to compare it with other simulators.

How to obtain the guest instruction count? I'm interested both in user and full system mode.

The only solutions I have now would be to log all instructions with either simple trace exec_tb or -d in_asm: How to use QEMU's simple trace backend? and then count the instructions from there. But this would likely considerably reduce simulation performance due to the output operations, so I would likely have to run the test program twice, one with and another without the trace, and hope that both executions are similar (should be, especially for single threaded user mode simulation).

I saw the -icount option, which sounds promising from the name, but when I passed it to QEMU 4.0.0, I didn't see anything happen. Should it print an instruction count somewhere? The following patch appears unmerged and suggests not: https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg01275.html

Laurentium

Answer 1

Basic Profiling

To follow up on Peter's answer, I have recently run into a situation where I wanted to get the instruction count of a program run under QEMU (I'm using v4.2.0, the first where plugins became available).

One of the example plugins, insn.c, does exactly what you want, and returns the count of executed instructions on plugin exit.

(I assume you already know how to run QEMU, so I'll strip this down to the important flags)

qemu-system-arm ... -plugin qemu-install-dir/build/tests/plugin/libinsn.so,arg=inline -d plugin

The first part loads the plugin and passes a single argument, "inline" to it. The next part enables printing of the plugin. You can redirect the plugin output to a different file by adding -D filename to the command line invocation.

More Advanced Profiling

When I was looking for possible ways to profile a program run under QEMU, this is one of the only results of my search that was promising. In the spirit of creating a good record for other searching in the future, here are some links to code that I have written to do just that.

Profiling Plugin code, docs.

_{^{Disclaimer: I wrote the above code.}}

Stupendous

Ahh, I had forgotten about this, thanks for the followup! – Laurentium

make sure to --enable-plugins on configure. – Abortionist

Answer 2

Current released versions of QEMU don't provide any means for doing this. The upcoming "TCG plugin" support which should go out in the 4.2 release at the end of the year would allow you to write a simple "count the instructions executed" plugin, but this (as with the -d tracing) will add an overhead.

The -icount option is certainly confusing, but what it does is make the emulated CPU (try to) run at a specific number of executed instructions per second, as opposed to the default of "as fast as possible". This has higher overhead (and it will stop QEMU using multiple host threads for SMP guests), but is more deterministic.

Philosophically speaking, "instructions per second" is a rather misleading metric for emulators, because the time taken to execute an instruction can vary vastly compared to hardware. Loads and stores are rather slower than on real hardware. Floating point instructions are incredibly slow (perhaps a factor of 10 or worse of an integer arithmetic instruction, where real hardware could execute both in one cycle). JIT emulators like QEMU have a start-stop performance profile where execution stops entirely while we translate a block of code, whereas a real CPU or an interpreting emulator will not have these pauses. How much effect the JIT time has will depend on whether your code reruns previously translated hot code frequently or if it spends most of its time running "new" code, and whether it does things that result in the JIT having to discard the old code (eg self modifying code, or frequent between-process context switches). If you had an "IPS meter" on your emulator you'd see the value it reported fluctuate wildly as the guest code executed and did different things. You're probably better off just picking a benchmark which you think is representative of your actual use case, running it on various emulators, and comparing the wall-clock time it takes to complete.

Sheryllshetland

Thanks for yet another fantastic answer, Peter. I didn't know about the TCG plugin system, and I am very excited to hear about it. I wonder if a "device plugin" system with devices in shared libraries wouldn't also be feasible: #44255290 – Laurentium

Device plugins are specifically off the menu, because upstream does not want to provide a nice easy mechanism for people to use to have out-of-tree non-GPL/closed-source devices. Those land upstream with all the support burden of maintaining a stable API/ABI to devices, without giving the upstream community any benefit. – Sheryllshetland

Thanks for the info. Let me know if you can find an existing upstream talk so I can link to from that question. Stable API could not be a requirement like kernel module API. – Laurentium

The topic of plugins came up at this year's QEMU Summit, so we're planning to write the project policy/philosophy up at some point soon. – Sheryllshetland

@PeterMaydell, Concerning the upcoming "TCG plugin", will it make use of performance monitoring counters (instructions counter) available in most cpu architectures? – Myke

No, host performance monitoring counters would be of no use to a JIT based emulator like QEMU. – Sheryllshetland