Getting EPERM when calling pthread_create() for SCHED_FIFO thread as root on Linux

Question

I am trying to spawn threads with SCHED_FIFO or SCHED_RR policies as root on a Linux system but my calls to pthread_create() are returning 1 (EPERM). The man page for pthread_create() says that EPERM indicates that "[t]he caller does not have appropriate permission to set the required scheduling parameters or scheduling policy." Shouldn't root be able to specify SCHED_FIFO or SCHED_RR?

I've stripped out the code that creates a thread into a small program that does only that. It looks right to me but still gets the error. What am I doing wrong?

The program:

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

static void *_Thread(void *arg)
{
    (void)arg;
    printf("Thread running!\n");
    return NULL;
}

int main(void)
{
    int retVal;
    pthread_attr_t attr;
    struct sched_param schedParam;
    pthread_t thread;

    retVal = pthread_attr_init(&attr);
    if (retVal)
    {
        fprintf(stderr, "pthread_attr_init error %d\n", retVal);
        exit(1);
    }

    retVal = pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
    if (retVal)
    {
        fprintf(stderr, "pthread_attr_setinheritsched error %d\n", retVal);
        exit(1);
    }

    retVal = pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
    if (retVal)
    {
        fprintf(stderr, "pthread_attr_setschedpolicy error %d\n", retVal);
        exit(1);
    }

    schedParam.sched_priority = 1;
    retVal = pthread_attr_setschedparam(&attr, &schedParam);
    if (retVal)
    {
        fprintf(stderr, "pthread_attr_setschedparam error %d\n", retVal);
        exit(1);
    }

    retVal = pthread_create(&thread,
                            &attr,
                            _Thread,
                            NULL);
    if (retVal)
    {
        fprintf(stderr, "pthread_create error %d\n", retVal);
        exit(1);
    }

    retVal = pthread_join(thread, NULL);
    if (retVal)
    {
        fprintf(stderr, "pthread_join error %d\n", retVal);
        exit(1);
    }

    printf("main run successfully\n");
    return 0;
}

This program has been compiled and run as root. When run, it program fails at the call to pthread_create, returning EPERM.

Changing the thread to SCHED_RR scheduling has no effect - EPERM still returned by pthread_create.

Changing the thread to SCHED_OTHER scheduling and its priority to 0 allows the program to run without error.

Zambia

I believe you need CAP_SYS_NICE capability set for the calling user. I don't know much about how this works with sudo etc. Check out man 7 capabilities. – Kenaz

It looks like you can set the nice level using ulimit -e in the same way as setting the real-time priority limit. It hasn't provided me with a "eureka!" moment yet, though. – Zambia

The Linux systems we use at work are virtualized servers. I'm grasping at straws here but is that likely to have any effect on setting real-time priorities? – Zambia

Look in /etc/security/limits.conf for an "rtprio" entry. That is probably the issue. Virtual servers are usually set up to be completely standalone in terms of things like this. Do you have a full local root filesystem (not inherited)? – Walkling

The only reference to "rtprio" in /etc/security/limits.conf is in a comment. I found someone who has Linux (Ubuntu) running on a laptop and I got him to compile and run the program above and it worked for him so it's something to do with our Linux servers that's causing the issue. I'll check with our IT guy when he comes in tomorrow about the nature of their root filesystems. – Zambia

Answer 1

It must be that your real-time priority soft limit is too restrictive.

Either call ulimit -r unlimited in shell before running your code. Or call setrlimit(RLIMIT_RTPRIO, ...) directly in your code.

System-wide limits are set in /etc/security/limits.conf.

Freezing

Calling ulimit -r did indeed show that root's real-time priority limit is 0. I have changed it first using ulimit -r unlimited and then in my program with setrlimit(...) but while I could verify that the limit had changed, I still got EPERM. – Zambia

Are you in a special envrionment, maybe like ARM? What does uname -a show? – Walkling

uname -a gives me Linux cumanta 2.6.32-042stab044.17 #1 SMP Fri Jan 13 12:53:58 MSK 2012 i686 i686 i386 GNU/Linux. The machine runs CentOS 6.2. – Zambia

I ran your code on Fedora 14 and it completes successfully. However, I have custom settings in my /etc/security/limits.conf: @wheel hard rtprio unlimited and @wheel soft rtprio 10 and my user belongs to group wheel. – Freezing

I also ran your code on CentOS 6.2 Linux 2.6.32-220.2.1.el6.x86_64 #1 SMP Tue Dec 13 16:21:34 EST 2011 x86_64 x86_64 x86_64 GNU/Linux with default /etc/security/limits.conf. It fails under my non-privileged user and succeeds when run as root, as expected. – Freezing

It seems to be something to do with the kernel build used by CentOS 6.2 for virtualized servers. My code's been tested on a virtualized CentOS 5 server and worked fine there (once the real-time priorities were tweaked) and already worked on a standalone Ubuntu box so it seems to be something to do with that kernel. Thanks for all the help, demonstrating that it was the environment and not the code is enough for now I think! – Zambia

Answer 2

Surprisingly, I ran into the same problem as the OP almost 10 years later: CentOS 7.6, spawning a realtime thread, root user, ulimit -r says "unlimited", yet the thread creation fails returning EPERM. It turned out the failure is caused by the following clause in Linux kernel:

kernel/sched/core.c

          if (user) {
  #ifdef CONFIG_RT_GROUP_SCHED
                  /*
                   * Do not allow realtime tasks into groups that have
no runtime
                   * assigned.
                   */
                  if (rt_bandwidth_enabled() && rt_policy(policy) &&
                                  task_group(p)->rt_bandwidth.rt_runtime == 0 &&
                                  !task_group_is_autogroup(task_group(p))) {
                              task_rq_unlock(rq, p, &flags);
                              return -EPERM;
                  }
  #endif

Given that, and Linux cgroup documentation https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt, and CONFIG_RT_GROUP_SCHED documentation https://www.kernel.org/doc/Documentation/scheduler/sched-rt-group.txt, the problem is resolved by adding the current shell to the existing cgroup with realtime scheduling info:

cd /sys/fs/cgroup/cpu
echo $$ > tasks

Then running the executable spawning a realtime thread from the current shell the thread creation succeeds.

Paste

Wow! I totally hit exactly this same thing. Man... just when I thought I'd seen every super obscure roadblock out there! Thank you for tracking it down! – Cai

Fascinating. We have ran into the same problem at CERN, and this fixed it. – Spry

I meet the same problem now on an openwrt embedded system, and this solution works for me. The command to give a cgroup rt permission: echo -n $runtime > /sys/fs/cgroup/$group/cpu.rt_runtime_us – Via

Answer 3

I've tested your code on Linux (2.6 kernel), Solaris 9 & 10. No problems. It has to do with your real-time priority settings. You can change that with:

ulimit -r unlimited

oops. Maxim already got here with the answer....

Walkling

Answer 4

Run the program as root user. Because we are messing with scheduling (its for FIFO and RR). you're messing also (by implication) with relative priorities, and that only root can do.

Running above program normally, pthread_create() returns EPERM, but as root user is works fine.

For more info visit: http://www.linuxforums.org/forum/programming-scripting/111359-pthread-error.html

Periwinkle

Answer 5

Met this on CentOS 7. After setting /proc/sys/kernel/sched_rt_runtime_us to -1, it works.

Hexapody