How can I ensure that a process runs in a specific physical CPU core and thread?

Question

This question asks about ensuring two processes run on the same CPU. Using sched_setaffinity I can limit a process to a number of logical CPUs, but how can I ensure that these are mapped to specific physical CPUs and threads?

I expect that the mapping would be:

0 - CPU 0 thread 0
1 - CPU 0 thread 1
2 - CPU 1 thread 0
3 - CPU 1 thread 1
etc...

where the number on the left is the relevant CPU used in sched_setaffinity.

However, when I tried to test this, it appeared that this is not necessarily the case.

To test this, I used the CPUID instruction, which returns the initial APIC ID of the current core in EBX:

void print_cpu() 
{
    int cpuid_out;

    __asm__(
    "cpuid;"
        : "=b"(cpuid_out) 
        : "a"(1) 
        :);

    std::cout << "I am running on cpu " << std::hex << (cpuid_out >> 24) << std::dec << std::endl;
}

Then I looped over the bits in the cpu mask and set them one at a time so that the OS would migrate the process to each logical CPU in turn, and then I printed out the current CPU.

This is what I got:

cpu mask is 0 
I am running on cpu 0
cpu mask is 1 
I am running on cpu 4
cpu mask is 2 
I am running on cpu 2
cpu mask is 3 
I am running on cpu 6
cpu mask is 4 
I am running on cpu 1
cpu mask is 5 
I am running on cpu 5
cpu mask is 6 
I am running on cpu 3
cpu mask is 7 
I am running on cpu 7

assuming that the CPU assigns initial APIC IDs according to the scheme I listed above, it would seem that the cpu mask doesn't actually correspond to the physical core and thread.

How can I find the correct mapping of bits in the mask for sched_setaffinity to physical cores?

Kaylenekayley

Answer 1

hwloc is a portable C library for discovering hardware/NUMA topology, and also binding processes/threads to particular cores. It has functions to discover physical/logical cores, and then bind a process/thread to it.

It also looks like it can also return a cpu_set_t for use with sched_setaffinity(), if you want to keep using that directly.

Lustrate

One of hwloc's best features is the capability to show a diagram of the system topology -- how CPUs share caches, memory domains, etc. – Engrave