How to have both clustered and non-clustered redis connections in laravel

Question

Background

In the past I was able to use a non-clustered redis just fine in my config like so:

'redis' => [

    'default' => [
        'host' => env('REDIS_HOST', '127.0.0.1'),
        'password' => env('REDIS_PASSWORD', null),
        'port'     => 6379,
        'database' => 0,
        'cluster' => true,
    ]
],

However due to the load on our redis servers, I have to cluster my redis, This config works fine when the only redis connection I have is clustered (figured it out after a lot of work):

'redis' => [
    'client' => 'predis',
    'cluster' => true,
    'options' => [
        'cluster' => 'redis',
        'parameters' => [
            'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_DEFAULT_PORT', 6379),
            'database' => 0,
            ],
        ],
    'clusters' => [
        'default' => [
            'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_SHARD_1_PORT', 6379),
            'database' => 0,
        ],
        'shard2' => [
            'host' => env('REDIS_SHARD_2_HOST', '127.0.01'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_SHARD_2_PORT', 6379),
            'database' => 0,
        ],
        'shard3' => [
            'host' => env('REDIS_SHARD_3_HOST', '127.0.01'),
            'password' => env('REDIS_PASSWORD', null),
            'port' => env('REDIS_SHARD_3_PORT', 6379),
            'database' => 0,
        ],
        'options' => [
            'cluster' => 'redis'
        ],
    ]
]

any my env file looks like this (For my localhost, anyway):

QUEUE_DRIVER=redis      // cluster compatible
BROADCAST_DRIVER=redis  // cluster compatible
CACHE_CONNECTION=redis  // cluster incompatible 

REDIS_CLUSTER=true
REDIS_HOST=localhost

REDIS_DEFAULT_PORT=7000

REDIS_SHARD_1_HOST=localhost
REDIS_SHARD_2_HOST=localhost
REDIS_SHARD_3_HOST=localhost

REDIS_SHARD_1_PORT=7000
REDIS_SHARD_2_PORT=7001
REDIS_SHARD_3_PORT=7002

Problem

The fact is that currently, we use the non-clustered redis for the following:

• Cache: supports redis clustering
• Queue/Jobs: supports redis clustering
• Broadcast (ie websockets): does not support redis clustering

That's why we need to have both redis connections simultaneously, so that we can use the clustered connection for caching/queues, and non-clustered connection for websockets.

But this isn't working:

'redis' => [
    'default' => [
        'host' => env('REDIS_HOST', '127.0.0.1'),
        'password' => env('REDIS_PASSWORD', null),
        'port'     => 6379,
        'database' => 0,
        'cluster' => true,
    ],
    'clustered' => [
        'client' => 'predis',
        'cluster' => true,
        'options' => [
            'cluster' => 'redis',
            'parameters' => [
                'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
                'password' => env('REDIS_PASSWORD', null),
                'port' => env('REDIS_DEFAULT_PORT', 6379),
                'database' => 0,
                ],
            ],
        'clusters' => [
            'default' => [
                'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
                'password' => env('REDIS_PASSWORD', null),
                'port' => env('REDIS_SHARD_1_PORT', 6379),
                'database' => 0,
            ],
            'shard2' => [
                'host' => env('REDIS_SHARD_2_HOST', '127.0.01'),
                'password' => env('REDIS_PASSWORD', null),
                'port' => env('REDIS_SHARD_2_PORT', 6379),
                'database' => 0,
            ],
            'shard3' => [
                'host' => env('REDIS_SHARD_3_HOST', '127.0.01'),
                'password' => env('REDIS_PASSWORD', null),
                'port' => env('REDIS_SHARD_3_PORT', 6379),
                'database' => 0,
            ],
            'options' => [
                'cluster' => 'redis'
            ],
        ]

further, some users state that such a task is simply impossible for redis. Is that true?

update

I tried this

in Cache.php

    'redis' => [
        'driver' => 'redis',
        'connection' => 'clustered',
    ],

note: in the above connection i simply couldn't just copy/paste the cluster options, since it would crash if i didn't put a driver option

In database.php I was inspired by this answer and simply put different connections under the redis key: (ie `database.redis.connection-1, database.redis.connection-2 etc)

'redis' => [
    'clustered' => [
         // clustered settings copied from above
        ],
    ], 

    'default' => [
         // non clustered settings
    ],
]

To test, I ran the following tinker

>>> use Illuminate\Support\Facades\Cache;
>>> Cache::put('foo','bar',1);
Predis/Response/ServerException with message 'MOVED 7837 127.0.0.1:7001'

The move error is a known one, it's simply saying that i'm dealing with a non-clustered redis connection.

thoughts?

Darindaring

I'd tackle it like this: I'd create additional redis config, in cache.php I'd call it redis_nonclustered. In database.php I'd create additional redis config, also naming the key redis_nonclustered. This should let you use two redis connections, one clustered and other nonclustered, both distinct and separate from each other. – Simonnesimonpure

@Simonnesimonpure have you tried this before? – Darindaring

I didn't try with redis per-se, but I use this method to add various additional multiple connections with other database-like services. What you want to do is use redis clustered and nonclustered by using a single service provider in Laravel which you can't, you have to create two distinct connections - so just do that, configure two different connections and you're done. – Simonnesimonpure

@Simonnesimonpure If you can show me an example of how to create two different connections (for redis or otherwise) i would appreciate it (enough to merit a correct answer if it works). inside my config/cache.php i have 'redis' => [ 'driver' => 'redis', 'connection' => 'default', ], , where connection points to default, which is in turn defined in my config/database.php, so not sure how to add another connection. – Darindaring

@Simonnesimonpure I followed your advice above but still didn't work. Is that what you had in mind? – Darindaring

I'll reply with an answer once I get to a PC that lets me test the solution, it appears that one needs to bootstrap a new Redis service provider manually, it's not the same as adding additional MySQL connections. – Simonnesimonpure

Ok thanks for the effort in advance @N.B.. if you may please take a look at this thread as well as it has more details of what I tried: github.com/nrk/predis/issues/480#issuecomment-526890740 – Darindaring

thanks @abbood. I just switched to phpredis instead of predis and it's fine now. – Dissentious

@Dissentious can you set both clustering AND replication with phpredis? see here: #57761828 – Darindaring

Answer 1

I was able to fix this problem with this PR.

This is what my config looks like now:

'redis' => [

    'clustered' => [
        'client' => 'predis',
        'cluster' => true,
        'options' => [ 'cluster' => 'redis' ],
        'clusters' => [
                    [
                        'host' => env('REDIS_SHARD_1_HOST', '127.0.01'),
                        'password' => env('REDIS_PASSWORD', null),
                        'port' => env('REDIS_SHARD_1_PORT', 6379),
                        'database' => 0,
                    ],
                    [
                        'host' => env('REDIS_SHARD_2_HOST', '127.0.01'),
                        'password' => env('REDIS_PASSWORD', null),
                        'port' => env('REDIS_SHARD_2_PORT', 6379),
                        'database' => 0,
                    ],
                    [
                        'host' => env('REDIS_SHARD_3_HOST', '127.0.01'),
                        'password' => env('REDIS_PASSWORD', null),
                        'port' => env('REDIS_SHARD_3_PORT', 6379),
                        'database' => 0,
                    ],
        ],
    ], 

    'default' => [
        'host' => env('REDIS_HOST', '127.0.0.1'),
        'password' => env('REDIS_PASSWORD', null),
        'port'     => 6379,
        'database' => 0,
        'cluster' => false,
    ],
]

Will be sending PR to Laravel itself shortly.

Darindaring