mq_open() - too many open files

Question

I'm trying to write a client and server which are going to exchange data by using POSIX message queue. I tried to do it by looking at the examples I saw in the Internet and the documents of the course. But, I am stuck in. When I run it, I get "Too many open files" error. Here is my code:

Client:

int main( int argc, char *argv[]) {

    //Open its queue, which is client queue
    char cq_name[10];
    sprintf( cq_name, "/cq%i", getpid());
    printf( "Client Queue name: %s\n", cq_name);

    mqd_t cq_id = mq_open( cq_name, O_CREAT | O_RDWR, 0666, NULL);
    if( cq_id == -1) {

            printf( "Error in cq: %s\n", strerror( errno));
            return -1;
    }

    printf( "Name: %s\n", argv[1]);

    //Connect to the server message queue
    mqd_t sq_id = mq_open( argv[1], O_RDWR);

    if( sq_id == -1) {

            printf( "Error in sq: %s\n", strerror( errno));
            return -1;
    }

...

Server:

int main( int argc, char *argv[]) {

    //The server message queue
    struct mq_attr attr;
    attr.mq_flags = 0;
    attr.mq_curmsgs = 0;

    printf( "Name: %s\n", argv[1]);

    mqd_t id = mq_open( argv[1], O_CREAT | O_RDWR, 0666, NULL);

    //Check the message queue
    if( id == -1) {

            printf( "Error: %s\n", strerror(errno));
    }

    printf( "Check Point 1 - %i\n", id);

...

Can you help me to figure out what the problem is. Thanks in advance..

Mclin

Does my answer answered your question ? – Easy

Answer 1

Usually, Too Many Open files means that your are trying to open file descriptors while you already opened the maximum number of files allowed by the system.

cat /proc/sys/fs/file-max

show you the maximum number of file descriptors allowed on your system. You can try to increase this value for your current running session with:

sysctl fs.file-max = new_value

or permanently by editing /etc/sysctl.conf by adding the following line:

fs.file-max = new_value

To be sure that this limit is the one you are reaching you can run

cat /proc/sys/fs/file-nr

The third column nnumber is the same as the one reported by file-max. The first column is the number of allocated file handles and the second one is the number of unused but allocated file handles. If substracting of the first number by the second one gives you the third one (or a number close to the third one) you are reaching the limit.

As suggested by @nos in a comment, in the precise case of mq_open as specified by the man page (*ENOSPC Insufficient space for the creation of a new message queue. This probably occurred because the queues_max limit was encountered; see mq_overview(7)*) you 'll need also to check in the same way the value of fs.mqueue.queues_max

/proc/sys/fs/mqueue/queues_max

to get current value and

sysctl fs.mqueue.queues_max = new_value

to change it.

Easy

You'll need fs.mqueue.queues_max too, it's a lot lower than fs.file-max. The next problem might be the per process limit on file descriptors (look with ulimit -n ) – Illness

doesnt the message will be null after reader have recieved the message? – Keating

the message can't be sent at all because the ms_send function needs the mqd returned by mq_open which is failling – Easy

Answer 2

I hit this problem this week - that I could only open a max of 10 mqs.

My intent is to use mqs to pass notification of events to threads. In my case an event_id is just an int. By using specifying non-default attributes in the call to mq_open like so:

char mq_name[128];
sprintf(mq_name, "%s.%d", MQ_NAME, nnid);

struct mq_attr attrib;
attrib.mq_flags = 0;
attrib.mq_maxmsg = 4;
attrib.mq_msgsize = sizeof(int);
attrib.mq_curmsgs = 0;

retval = mq = mq_open(mq_name, O_RDWR | O_CREAT | O_EXCL, 0644, &attrib);

I am now able to open up to 256 mqs before mq_open() fails.

Conspiracy

Answer 3

There are a number of limits that you might be hitting here.

Assuming you are on Linux:

Since you are passing NULL in as the last argument to mq_open, you get default number of messages and max message size per queue. These are set from

/proc/sys/fs/mqueue/msg_default and

/proc/sys/fs/mqueue/msgsize_default

You can view these limits with

cat /proc/sys/fs/mqueue/msg_default

and change them with

sudo echo myNewLimit > /proc/sys/fs/mqueue/msg_default

There is also a limit on the total number of message queues that can be created on a system. This is stored at /proc/sys/fs/mqueue/queues_max Documentation for these and other limits is in MQ_OVERVIEW(7)

There is also a limit on how much memory your process can allocate to message queues. This memory is at least (Num_Messages * Message_Length + book keeping overhead) per message queue. A Detailed description of this limit is available here GETRLIMIT(2)

I would advise writing a small test program to create multiple message queues to experiment with these limits. Also remember to check for the existence of queues before you start by using

ls -l /dev/mqueue/ 

And also remember to delete your queues with mq_unlink at the end of your program, else they will persist between runs.

Girand

Answer 4

You may try following

    struct rlimit r;
    r.rlim_cur = 99999999;
    r.rlim_max = 99999999;
    if (setrlimit(RLIMIT_MSGQUEUE, &r) == 0)
    {
        printf("set RLIMIT: %d\n", r.rlim_cur);
    }
    else printf("failed to set RLIMIT", strerror(errno));

Hunyadi

Answer 5

You need to close the your client in server code every time you are done processing client request and sending response. Something like this.

// close the client
if (mq_close (qd_client) == -1) {
        perror ("Client: mq_close");
        exit (1);
}

Here's a complete working server example.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>

#include <fcntl.h>
#include <sys/stat.h>
#include <mqueue.h>

#define SERVER_QUEUE_NAME   "/sp-example-server"
#define QUEUE_PERMISSIONS 0666
#define MAX_MESSAGES 10
#define MAX_MSG_SIZE 8000
#define MSG_BUFFER_SIZE MAX_MSG_SIZE + 10

int main (int argc, char **argv)
{
    mqd_t qd_server, qd_client;   // queue descriptors
    long token_number = 1; // next token to be given to client

    struct mq_attr attr;

    attr.mq_flags = O_RDWR | O_CREAT;
    attr.mq_maxmsg = MAX_MESSAGES;
    attr.mq_msgsize = MAX_MSG_SIZE;
    attr.mq_curmsgs = 0;

    if ((qd_server = mq_open (SERVER_QUEUE_NAME, O_RDWR | O_CREAT, QUEUE_PERMISSIONS, &attr)) == -1) {
        perror ("Server: mq_open (server)");
        exit (1);
    }

    char in_buffer [MSG_BUFFER_SIZE];
    char out_buffer [MSG_BUFFER_SIZE];

    while (1) {     // while 1 starts
        // get the oldest message with highest priority
        if (mq_receive (qd_server, in_buffer, MSG_BUFFER_SIZE, NULL) == -1) {
        perror ("Server: mq_receive");
            exit (1);
        }
            // output message from client
        int iter001;

        printf("Message received from Client is following: \n");
        for (iter001 = 0; iter001 < MSG_BUFFER_SIZE; iter001++)
        {
                printf("%c", in_buffer[iter001]);
        }
        printf("\n");

        printf ("Server: message received.\n");

        // send reply message to client

        if ((qd_client = mq_open (in_buffer, O_RDWR | O_CREAT)) == 1) {
        perror ("Server: Not able to open client queue");
        continue;
        }
        printf("Value of qd_client: \n");
        printf("%d\n", qd_client);
        sprintf (out_buffer, "%s", "hello world!");

        printf("Value of out_buffer: %s", out_buffer);
        if (mq_send (qd_client, out_buffer, strlen (out_buffer), 0) == -1) {
        perror ("Server: Not able to send message to client");
        continue;
        }

        printf ("Server: response sent to client.\n");
        token_number++;

        // close the client
        if (mq_close (qd_client) == -1) {
        perror ("Client: mq_close");
        exit (1);
        }

    }   // while 1 ends
}

Kerbstone