Why is RDTSC a virtualized instruction on modern processors?

Question

I am studying RDTSC and learning about how it is virtualized for the purposes of virtual machines like VirtualBox and VMWare. Why did Intel/AMD go to all the trouble of virtualizing this instruction?

I feel like it can be easily simulated with a trap and it's not exactly a super-common instruction (I tested and there's no noticable slow-down for general usage in a virtual machine where hardware RDTSC virtualization is disabled).

However, I know Intel/AMD wouldn't have gone to all the trouble to add this instruction to the virtualizing hardware unless it was important to able to execute very fast.

Does anyone know why?

Centra

Note that according to my tests, on a standard EC2 instance RDTSC is not virtualised (I used the tests at this question about haveged - this also seems to be confirmed by the comment on this thread – Kathyrnkati

Answer 1

Its common to use RDTSC to get fine-grained timing information, where the overhead of a virtualization trap would be quite significant. Most common use is to have two RDTSC instructions with a small amount of code between them, taking the difference of the times as the elapsed time (number of cycles) for the code sequence. So even the overhead of pipeline drains/flushes is quite significant.

Also, since all the instruction does is read a continuously running counter, virtualizing it is quite easy -- the hardware only needs to allow saving/reloading the counter value on VM context switches, and not anything special for the RDTSC instruction itself.

Xanthous

Okay, I get why you would need virtualization for very fine-grained timing of a short sequence of instructions. However, the only reason I can think of for doing this is research purposes, or side-channel attacks. Are there any legitimate reasons? It almost feels like virtualization of rdtsc is evil – Centra

( not to imply research isn't legitimate :P , I meant to say, "any other legitimate reasons besides the vital but uncommon task of fine-grained timing research?" ) – Centra

Well, if you DON'T virtualize RDTSC properly, it becomes an easy way for a program to detect if its running under virtualization, which you might not want – Xanthous

@Robert Having an accurate high-frequency timer is important if you're building an instrumented profiler into your program, so you can determine exactly how long individual functions take. That's important if your servers are required to always respond within 25 milliseconds, for example, so you can track performance issues that come up after deployment and optimize them. – Equinox

It's also important if you're building industrial machinery or robots, which need to run motors or sensors at precise speeds, etc. – Equinox

Both good points, thanks. Although on a millisecond time scale, I feel like gettimeofday() or QueryPerformanceCounter would be more appropriate, right? – Centra

@RobertMartin: If what you're timing takes significantly longer than the granularity of gettimeofday() you are better off using that. RDTSC is not reliable unless you go through very specific pains (pinning to single core, disabling power management, calling cpuid before hand to prevent out of order execution...). The advantage of RDTSC is that it will have less of a probe effect (reading a register is cheap). – Chrysostom

Answer 2

VMs should be able to have separate TSCs because they start up at different times. The physical CPU just has one, so something is needed to at least get individual, per-VM TSC offsets.

Also, since VMs don't own the underlying physical CPUs fully, that is, they don't get to execute on them all the time, their TSCs should also somehow reflect the "on/off" periods and it is desirable that they don't change abruptly in value w.r.t. actual time, which the VMs should still get right from the host OS, because there's a lot of software that is virtualization-unready and can break when the numbers are too off.

I think these are the reasons why RDTSC is virtualized. But whatever you do, meeting conflicting requirements is tough and they complicate matters. You can't hide virtualization and have VMs run at near-native speed at the same time. There are trade offs and some things have to give in.

Berner

Migrating a VM to hardware with a different TSC frequency is another reason; that's why scaling as well as offsets are supported. – Deciare

I think generally you would want the guest TSC to jump if it hasn't run any instructions for a long time. Usually software uses RDTSC as an absolute timesource e.g. for clock_gettime, (not for start/stop interval timing) so any attempt to hide the fact that the VM wasn't running for some time will mean delivering incorrect times to the guest. You generally don't want that. If your VM is sensitive to long pauses, don't run it on a hypervisor that will ever pause it; shut it down instead of pausing it. Your other option is to let its wall-clock fall behind real time, I guess. – Deciare