Error in implementing message queue using redis, error in using BLPOP

Question

I am trying to build a message queue using Redis. Whenever client sends new data, they are added to a list.

Here is the code for it

$client->lPush("my_queue", $data);

Now there is a separate script slave.php which pops the newly arrived data and process it. The code for slave.php

while (true) {
   list($queue, $message)  = $client->brPop(["my_queue"], 0);

    /*
    Logic to process the data
    */
}

I have modified the apache startup script so that slave.php should start & stop with apache. It works well. But after waiting for few minutes the brPop stops listening with the error message like this :

Uncaught exception 'Predis\Connection\ConnectionException' with message 'Error while reading line from the server [tcp://127.0.0.1:6379]' in /var/www/api/lib/predis-0.8/lib/Predis/Connection/AbstractConnection.php:139
Stack trace:
#0 /var/www/api/lib/predis-0.8/lib/Predis/Connection/StreamConnection.php(205): Predis\Connection\AbstractConnection->onConnectionError('Error while rea...')
#1 /var/www/api/lib/predis-0.8/lib/Predis/Connection/AbstractConnection.php(128): Predis\Connection\StreamConnection->read()
#2 /var/www/api/lib/predis-0.8/lib/Predis/Connection/AbstractConnection.php(120): Predis\Connection\AbstractConnection->readResponse(Object(Predis\Command\ListPopLastBlocking))
#3 /var/www/api/lib/predis-0.8/lib/Predis/Client.php(227): Predis\Connection\AbstractConnection->executeCommand(Object(Predis\Command\ListPopLastBlocking))
#4 /var/www/api/lib/slave.php(7): Predis\Client->__call('brPop', Array)
#5 /var/www/api/lib/slave.php(7): Predis\Client->brPop(Array, 0)
#6 {main}
 thrown in /var/www/api/lib/predis-0.8/lib/Predis/Connection/AbstractConnection.php on line 139

According to the documentation, if list is empty then, BLPOP/BRPOP blocks the connection until another client performs an LPUSH or RPUSH operation against one of the keys. But this is not happening in my case. In my case once the brpop blocks the connection, it doesn't listen again even when new data arrives in the list.

What changes I should make to get this working?

Moorefield

I would suggest to make sure both scripts connect to the same Redis instance on the same database. It should work. Also, it is a bad practice to set an infinite timeout. It is much better to wait for say 10 seconds, and handle/ignore empty results in your code. – Tychon

Yes..both scripts connect to the same redis instance on the same database. As I have mentioned it works well for few minutes. But after waiting for few minutes, it blocks the connection and never listens again. Regarding waiting for 10 seconds, we can do so, but I think BLPOP has some features by which slave script finds out whenever new data arrive in the list. Also making unnecessary empty requests will add to the server load – Moorefield

If you check every 10 seconds, you're adding a few extra instructions every 20 or 30 billion cycles. It's not significant, but makes your service more reliable. Didier is right; this is pretty standard practice for blocking pops in Redis. – Castara

Make sure that your producer (LPUSH) and consumer (BRPOP) are using separate connections. Also, in your while loop, catch any exception, reconnect if necessary (does Predis do this automatically), and continue looping. – Castara

Answer 1

Its working for me now but I am not sure whether it is the right method to do this. Now I am catching the error and recursively calling the function in case of connection failure. My new slave.php looks like this :

function process_data()
{
    try {
        $client = new \Predis\Client();

        require_once("logger.php");

        while (true) {
            list($queue, $message) = $client->brPop(["bookmark_queue"], 0);
            // logic
        }
    } catch (Exception $ex) {
        $error = $ex->getMessage();
        log_error($error, "slave.php");
        process_data(); // call the function recursively if connection fails
    }
}
process_data(); // call the function

Moorefield

It's been more than a year of this. Did you figure out a way to (say set timeout for $client and something else) avoid Exception? – Surely

@ManuManjunath please check the answer below for reason – Stodder

Answer 2

The problem is not with your Redis configuration but you have to change your php.ini setting and set

default_socket_timeout = 90 //Or whatever you would like to set

It is PHP which disconnects the socket when reached the max socket timeout.

Stodder

Answer 3

Add ?read_write_timeout=-1 to your connection string .

Gabi

Maybe if that is the simple answer, shouldn't be deleted. I would wait for the OP to approve or downvote the answer, but still looks poor. One could have expanded a little explanation – Thirtytwo