Can constant non-invariant tsc change frequency across cpu states?

Question

I used to benchmark Linux System Calls with rdtsc to get the counter difference before and after the system call. I interpreted the result as wall clock timer since TSC increments at constant rate and does not stop when entering halt state.

The Invariant TSC concept is described as

The invariant TSC will run at a constant rate in all ACPI P-, C-. and T-states.

Can a constant non-invariant tsc change frequency when changing state from C0 (operating) to C1 (halted)?

My current view is that it cannot change frequency only across Performance(P) states. So applying rdtsc to get wall clock timer for system calls is not reliable when using non-invariant tsc.

I did not find invariant tsc flag in my /proc/cpuinfo, only constant_tsc meaning that it is not necessary invariant.

The source of confusion is the sentence from Intel System Programming manual:

The time stamp counter in newer processors may support an enhancement, referred to as invariant TSC.

So some chips (including mine) have constant, but not invariant tsc.

Freefloating

What's the last non-constant_tsc processor built? Do you care about prehistoric processors? – Magness

@Magness Well, no I do not consider non-constant_tsc. But having constant_tsc does not mean having invariant_tsc. – Freefloating

@Magness Or are invariant and constant tsc actually the same thing? – Freefloating

Ah, right. Well, for my processor there are a few TSC-related flags in /proc/cpuinfo: tsc, constant_tsc, nonstop_tsc, tsc_known_freq, tsc_deadline_timer, tsc_adjust. No "invariant", but maybe it's the "known frequency"? – Magness

@Magness At least one of them. I just ran a quick example with cpuid, eax = 0x80000008 (a leaf to determine if tsc is invariant) and got the resulting content of edx as 0b100000000 (8th bit stands for enabled invariant_tsc). So my chip actually supports invariant_tsc which was not clear from /proc/cpuinfo output. – Freefloating

cpuinfo indicates Invariant TSC by putting both constant_tsc and nonstop_tsc – Fomentation

@Fomentation Thanks much. Don't you know if cpus with constant non-invariant tsc even exist? Or can non-invariant tsc change frequency (not completely stop) in a halted state? – Freefloating

Are you sure you don't just have an older kernel? There's only one CPUID feature bit for all of these things, but the Linux kernel added a few different names so it could do some logic based on known CPU model numbers. I did some research about TSC feature bits and put my findings in this answer. But older kernels don't show the different names in /proc/cpuinfo, just constant_tsc on an old Core 2 system I have, for example. Even though it is nonstop and synced between cores. – Caisson

Answer 1

Starting with Nehalem and Saltwell, all Intel processors support invariant TSC, which means that the TSC is incremented at a constant rate across P-, C-, and T-states (but not necessarily across S-states).

Starting with Pentium 4 Family 0F Model 03, all Intel processors support constant TSC, which means that the TSC is incremented at a constant rate across P- and T-states. The TSC continues to increment in the HLT state (called Auto Halt or C1/Auto Halt). TSC doesn't increment in any other sleep state. This category of processors includes Bonnell.

Older processors don't support constant TSC. The TSC continues to increment in the HLT state, but not in deeper sleep states. On some of these processors, TSC is buggy.

The TSC value may be reinitialized (to some BIOS-dependent value) when waking up from an S-state.

Here is a summary. "Y" means that TSC continues to increment at the same rate across the specified type of states. "N" means that TSC either continues to increment at a different rate or stops incrementing. On a few processors, TSC is incremented in the S3 state and lower (this is called always-on TSC). "N/A" means that TSC is not supported.

                                  |   T   |   P   |C = HLT|C Other|S <= S3|S Other|
---------------------------------------------------
Nehalem+                          |   Y   |   Y   |   Y   |   Y   |   N   |   N   |
Silvermont Merrifield+Moorefield, |   Y   |   Y   |   Y   |   Y   |   Y   |   N   |
Saltwell Penwell+Cloverview
Other Saltwell+                   |   Y   |   Y   |   Y   |   Y   |   N   |   N   |
KNL+                              |   Y   |   Y   |   Y   |   Y   |   N   |   N   |
P4 90nm+                          |   Y   |   Y   |   Y   |   N   |   N   |   N   |
Enhanced Pentium M+               |   Y   |   Y   |   Y   |   N   |   N   |   N   |
Bonnell                           |   Y   |   Y   |   Y   |   N   |   N   |   N   |
Quark X1000                       |   Y   |   N   |   Y   |   N   |   N   |   N   |
KNC                               |   Y   |   N   |   Y   |   N   |   N   |   N   |
P5+                               |   Y   |   N   |   Y   |   N   |   N   |   N   |
Before P5                         |  N/A  |  N/A  |  N/A  |  N/A  |  N/A  |  N/A  |
Other Quark                       |  N/A  |  N/A  |  N/A  |  N/A  |  N/A  |  N/A  |

Pender

I went looking for info on invariant vs. constant TSC in the Linux kernel source last time I was editing my answer on How to get the CPU cycle count in x86_64 from C++?. I didn't find anywhere Linux was unsetting one feature flag but not the other for Intel CPUs. If I'm not mistaken, both invariant and constant TSC are the same CPUID feature bit (which is weird, right? Shouldn't there be something in CPUID that indicates that the TSC will keep ticking in deeper C-states?) – Caisson

@PeterCordes There is only a bit for invariant TSC, which is CPUID.0x80000007.EDX[8]. Yeah there is no bit to differentiate between constant and non-constant TSC. Note that invariant TSC is also constant. – Pender