Redis how to store associative array? Set or Hash or List?

Question

I'm a bit confused with all the available storing options of Redis. I want to do something simple and I don't want to over engineer it. I'm working with phpredis and Redis v2.8.6.

I have this simple associative array that I need to store. I also need to be able to retrieve an item by its key and loop over all the items.

$a = array(
    '12345' => array(
        'name' => 'Post A',
        'val2' => 'blah blah',
        'val3' => 'blah blah blah',
    ),
    '54321' => array(
        'name' => 'Post B',
        'val2' => 'blah blah',
        'val3' => 'blah blah blah',
    ),
    '998877' => array(
        'name' => 'Post C',
        'val2' => 'blah blah',
        'val3' => 'blah blah blah',
    )
);

So what I was doing till now was using hash type. storing my array like this:

foreach ($a as $key => $value) {
    $this->redis->hSet('posts', $key, json_encode($value));
}

Like that I could access the key easily like this:

public function getPost($postId)
{
    return json_decode($this->redis->hGet('posts', $postId), true);
}

// This is returning the information of Post A
$post = getPost(12345);

But now I need to loop over all the posts I don't know how to do it and if I can do it with my current structure. I don't know if I need to store all the post_id in another list to be able to loop over all the posts?

So my question is which data type(s) should I use to store my list of posts, allowing me to fetch a single post by its id and looping over all the posts?

Thanks, Maxime

Catenate

Answer 1

You can use SET and Hash and SORT in combination

redis 127.0.0.1:6379> HMSET TEST_12345 name "Post A" val2 "Blah Blah" val3 "Blah Blah Blah"
OK
redis 127.0.0.1:6379> HMSET TEST_54321 name "Post B" val2 "Blah Blah" val3 "Blah Blah Blah"
OK
redis 127.0.0.1:6379> HMSET TEST_998877 name "Post C" val2 "Blah Blah" val3 "Blah Blah Blah"
OK
redis 127.0.0.1:6379> SADD All_keys TEST_12345 TEST_54321 TEST_998877
(integer) 3
redis 127.0.0.1:6379> HGETALL TEST_12345

To GET one HASH:

redis 127.0.0.1:6379> HGETALL TEST_12345
1) "name"
2) "Post A"
3) "val2"
4) "Blah Blah"
5) "val3"
6) "Blah Blah Blah"

TO GET All HASH

redis 127.0.0.1:6379> SORT All_keys BY nosort GET *->name GET *->val2 GET *->val3
1) "Post A"
2) "Blah Blah"
3) "Blah Blah Blah"
4) "Post B"
5) "Blah Blah"
6) "Blah Blah Blah"
7) "Post C"
8) "Blah Blah"
9) "Blah Blah Blah"

If you don't want to use sort you can use Fetch All the key names from SET using SMEMBERS and then use Redis Pipeline to fetch all the keys

Negligence

So there is no other solutions than to store the hash of the post in a SET? What is the best options SORT or SMEMBERS if I have millions entries in the SET? – Catenate

If you notice for sorting i am using nosort so actually it will not sort. Also, Ideal structure according to me is HASH to keep key value pair, I am sure you wont be fetching millions of values at one go, you will fetch only few lets say 50 then you can use Limit with sort . redis.io/commands/SORT – Isreal

What if I want to delete all the hash from this set? Do I have to loop over all of them to delete them ? – Catenate

You mean you want to empty the SET or you also want to remove the HASH? – Isreal

Both of them. I want to delete the SET and all the HASH it contains. I'm just using Redis as a buffer while doing calculations, after the calculations I'm saving the result in MySQL and I want to delete the Redis stored objects that I won't be using anymore. – Catenate

In that case, redis Database 0 to 12 . You can choose a database using SELECT (e.g. Select 10) command. Load your data in one database and then use FLUSHDB to to remove all keys from that DATABASE. Make sure you use FLUSHDB not FLUSHALL because FLUSHALL will delete all the keys from ALL DATABASE and FLUSHDB will delete all keys from SELECTED DATABASE. – Isreal

Answer 2

Just for the folks looking for the PHP code, here is what I've ended up using:

// Create a post hash
$key = 'post:'.$post->getId();
$this->redis->hSet($key, 'data', serialize($post->toArray()));

// Add a post in the account posts SET
$this->redis->sAdd($account->getId().':posts', $post->getId());

// You can execute the above code as many time as you need 
// to add an object in a SET

// Fetch the first $limit posts for this account
// SORT <account_id>:posts BY nosort GET <account_id>:post:*->data
$key = $account->getId().':posts';
$keys = $this->redis->sort($key, array(
    'by' => 'nosort',
    'limit' => array($offset, $limit),
    'get' => 'post:*->data'
));

// All Good !
var_dump($keys);

I hope this will help some of you ;)

Catenate

You should check for igbinary: $igbinary = function_exists('igbinary_serialize'); $encoded = $igbinary ? igbinary_serialize($data) : serialize($data); It should be slightly faster than the regular serialize. – Unappealable

Answer 3

In PHP you can just do

$redis->set($key, json_encode($value));

Then

$value = json_decode($redis->get($key));

Or use whatever serialization technique you prefer. JSON encode/decode was performant enough for me not to care.

Cordless

But when you need to do this for thousands of results it would probably use more resources than necessary. Using a redis-only solution will less likely get you into performance trouble. – Alec

@BobKruithof Yeah, abso-fuckin-lutely. This answer is a bit of a dirty hack and I'm somewhat ashamed of it. It did work fine for what I was doing at the time though. – Cordless