Record dynamic instruction trace or histogram in QEMU?

Question

1. I've written and compiled a RISC-V Linux application.

2. I want to dump all the instructions that get executed at run-time (which cannot be achieved by static analysis).

Is it possible to get a dynamic assembly instruction execution historgram from QEMU (or other tools)?

Girvin

Did the QEMU manual contain anything useful on the subject? – Hosea

So you want counts by mnemonic (ignoring operands), not necessarily an actual trace of every instruction in execution order? Given a trace you can process it into a histogram if necessary. (I updated your question title to be more specific.) – Intermarriage

@PeterCordes Yes that's what I want. But how to obtain the trace (of executed guest assembly instructions) from QEMU ? – Girvin

I don't know, that's why I upvoted your question. But that might be a useful search term since it's standard terminology and something that other people have probably wanted at some point. – Intermarriage

Answer 1

For instruction tracing, I go with -singlestep -d nochain,cpu, combined with some awk. This can become painfully slow and large depending on the code you run.

Regarding the statistics you'd like to obtain, delegate it to R/numpy/pandas/whatever after extracting the program counter.

The presentation or video of user "yvr18" on that topic, might cover some aspects of QEMU tracing at various levels (as well as some interesting heatmap visualization).

Parallelism

Answer 2

QEMU doesn't currently support that sort of trace of all instructions executed.

1. The closest we have today is that there are various bits of debug logging under the -d switch, and you can combine the tracing of "instructions translated from guest to native" with the "blocks of translated code executed" translation to work out what was executed, but this is pretty awkward.

2. Alternatively you could try scripting the gdbstub interface to do something like "disassemble instruction at PC; singlestep" which will (slowly!) give you all the instructions executed.

Note: There ongoing work to improve QEMU's ability to introspect guest execution so that you can write a simple 'plugin' with functions that are called back on events like guest instruction execution; with that it would be fairly easy to write a dump of guest instructions executed (or do more interesting processing), but this is still work-in-progress, so not available yet.

Swords

Worth mentioning that for some ISAs (notably x86) there are other tools that do this for a single userspace process. e.g. Intel's SDE can do it, even histogramming directly like the OP asked as well as or instead of spitting out a big trace. It works by dynamic recompilation (from x86 to x86) to emulate new ISA extensions as well as doing binary instrumentation. IDK if there's anything at all similar for RISC-V. Other than of course using gdb inside the guest to single-step a process. Basically the same idea you suggested for QEMU's gdb-remote. – Intermarriage

I used the following command to obtain an instruction trace $QEMU_LINUX_USER -L $BUILDROOT_OUTPUT/images/rootfs -D logfile.log -d in_asm,cpu,fpu my_program .. But you said it's "awkward", do you mean that what's printed doesn't reflect reality? does the execution interfere with the host? or may be calls host services? .. I only need to study how many computaion-related instructions have been executed, how many memory accesses ...etc. This kind of stuff – Girvin

I mean that the trace gives you separately (1) what guest instructions go into what translation blocks and (2) tracing of what translation blocks are executed, so you have to recombine them yourself if you want to know what instructions are executed. Handling loops is also awkward as QEMU likes to optimise chaining between TBs which means repeated times around a loop may not get logged. (The 'nochain' option to -d disables that optimisation and makes QEMU slower and the logs more verbose but less confusing.) You need to know a bit about what QEMU does internally to correctly interpret the log. – Swords

Answer 3

It seems you can do something similar with rv8 (https://github.com/rv8-io/rv8), using the command:

rv-jit -l

Ought

Just in case this is still relevant, I wrote a poor man's profiler using this command, here github.com/bat52/rv8_prof – Ought

Answer 4

The "spike" RISC-V emulator allows tracing instructions executed, new values stored into registers, or just simply a histogram of PC values (from which you can extract what instruction was at each PC location).

It's not as fast as qemu, but runs at 100 to 200 MIPS on current x86 hardware (at least without tracing enabled)

Interglacial