Cannot create more than 10 mqueues

Question

I am using a python module that wraps the posix real time extensions to get MessageQueues.

This is the python code

#!/usr/bin env python
import uuid
import posix_ipc
import time

def spawn():
    return posix_ipc.MessageQueue("/%s" % uuid.uuid4(), flags=posix_ipc.O_CREAT)

i = 0
while True:
    i += 1
    spawn()
    print(i)

This will create about 10 mqs before reporting OSError: This process already has the maximum number of files open

I looked into mq limits and rlimit and checked that they are all set very high. E.g.

fs.file-max = 2097152
fs.mqueue.msg_max = 1000
fs.mqueue.queues_max = 1000

And even for privileged users it will still only create about 10 queues.

The equivalent C using the realtime extensions directly is as follows

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <errno.h>
#include <mqueue.h>

#define handle_error(msg) \
    do { perror(msg); exit(EXIT_FAILURE); } while (0)

int main(int argc, char **argv)
{
    mqd_t mq;
    struct mq_attr attr;
    char buffer[1024 + 1];
    int must_stop = 0;

    /* initialize the queue attributes */
    attr.mq_flags = 0;
    attr.mq_maxmsg = 10;
    attr.mq_msgsize = 1024;
    attr.mq_curmsgs = 0;

    /* create the message queue */
    int count = 0;
    char name[5];
    while (1) {
        sprintf(name, "/%d", count);
        mq = mq_open(name, O_CREAT | O_RDONLY, 0644, &attr);
        if (mq == (mqd_t) -1)
            handle_error("mq_open");
        count++;

    }
    return 0;
}

(compile with gcc -lrt test.c) But this only gets me 20 mqs open at one time. Realistically I want to have a few hundred maybe a thousand open at a time.

Anyone got any ideas or suggestions?

EDIT: Better error checking in the C version. Still max out.

Meldon

The C program error check is incorrect — check errno if mq_open returns (mqd_t)-1. – Olnek

@Olnek Fixed it – Meldon

Answer 1

The parameter fs.mqueue.queues_max is only a global number of message queues allowed in the system. The limit you are reaching is the number of message queues per process. Because mq_open says about error codes:

[EMFILE] Too many message queue descriptors or file descriptors are currently in use by this process.

You should normally be able to read (an set) that per process limit with getrlimit/ setrlimit. The man page for rlimit says:

RLIMIT_MSGQUEUE (Since Linux 2.6.8)

Specifies the limit on the number of bytes that can be allocated for POSIX message queues for the real user ID of the calling process. This limit is enforced for mq_open(3). Each message queue that the user creates counts (until it is removed) against this limit according to the formula:

bytes = attr.mq_maxmsg * sizeof(struct msg_msg *) +
        attr.mq_maxmsg * attr.mq_msgsize

where attr is the mq_attr structure specified as the fourth argument to mq_open(3).

The first addend in the formula, which includes sizeof(struct msg_msg *) (4 bytes on Linux/i386), ensures that the user cannot create an unlimited number of zero-length messages (such messages nevertheless each consume some system memory for bookkeeping overhead).

You can also try to read the value and multiply it with what you need:

struct rlimit limit;

getrlimit(RLIMIT_MSGQUEUE, &limit);
limit.rlim_cur *= 20;
limit.rlim_max *= 20;
setrlimit(RLIMIT_MSGQUEUE, &limit);

Tweezers