Redis & Node.js: All keys

Question

Basic question: Using Node.js I would like to get all the keys in my redis db. My redis db looks like this when I call keys *;

1. aXF
2. x9U
3. lOk

So each record I have, has a unique key, generated as a random string. Now I would like to call something like foreach(key in Redis) and get all keys in the redis. Would it be possible to accomplish a "SELECT * FROM Redis"-like query with Node.js & Redis

Codex

Answer 1

Sure, you'll need to install the redis module for nodejs which can be found at https://github.com/redis/node-redis.

npm install redis

Then you would do:

var redis = require('redis'),
    client = redis.createClient();

client.keys('*', function (err, keys) {
  if (err) return console.log(err);
   
  for(var i = 0, len = keys.length; i < len; i++) {
    console.log(keys[i]);
  }
});        

Generally speaking you won't want to always return all of the keys (performance will be bad for larger data sets), but this will work if you are just testing things out. There is even a nice warning in the Redis documentation:

Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases. This command is intended for debugging and special operations, such as changing your keyspace layout. Don't use KEYS in your regular application code. If you're looking for a way to find keys in a subset of your keyspace, consider using sets.

Knoll

First of all, great answer! Just wanted to ask one thing though. And even though you might know the exact answer, an approximation from a redis programmer would still do good. By LARGE db, can you give me an estimate number of keys? – Codex

If this is for a production site, don't use keys. That's pretty much the recommendation. If you are finding that you need to use it, then you probably need to rethink how you are using redis. You might want to look at sets and lists as an alternative. – Knoll

I'm sorry if I'm asking too much but in this application I have to assign every object with a unique key and later on will query them according to their(let's say age). I know that the number of key's wont go over 3K. If I have to apply this structure like "generate unique key for an object, insert. And later on get all the objects to check in O(N)" to set, what would I have to do? – Codex

For something like age, you could create a key for each age (such as age:32) and then push the object keys into a set client.sadd('age:32', key) and then store the actual object in its own key client.set(key, JSON.stringify(obj)). Now when you want to find all of the people that are 32 you can just call client.get('age:32') which will be O(1) instead of O(N). If you find that you are doing lots of queries like this, a database like mySQL or MongoDB might make more sense than Redis. – Knoll

Funny thing is we are actually using the MongoDB to store the static data of the users. However we have a constantly changing state. Assume that our users can go offline, invisible, online. In order to keep up with constant changes in these modes I'm using redis, so that I don't have to query the db even though its Mongo. But in fact we are going to be making tons of queries like this. But isn't Redis more efficient in this scenario? – Codex

No, that makes sense. You just need to figure out the right things to cache to make lookups fast. If you want to track status you should have three keys users:offline, users:online, and users:invis which use sets to store the users that are in that state. As users change state, remove them from the key of their current state and push them onto the key of their new state. – Knoll

That's great. One more thing though. If I learned Redis correctly, every db is a hashtable that you get the push key and a corresponding object. In this scenario, will the set take one key cell in hashtable, or set corresponds to a Redis DB? – Codex

One key in Redis. Keys in Redis can store a bunch of different types of values like strings, hashes, lists, and sets. This is one of the great things about Redis unlike something like Memcached which can only store string values. – Knoll

Answer 2

Install redis client for nodejs

npm install redis

Then I do the following to get all key's data

var redis =   require('redis'),
    client =  redis.createClient();

client.multi()
    .keys('*', function (err, replies) {
        // NOTE: code in this callback is NOT atomic
        // this only happens after the the .exec call finishes.

        console.log("MULTI got " + replies.length + " replies");

        replies.forEach(function (reply, index) {
            console.log("Reply " + index + ": " + reply.toString());
            client.get(reply, function(err, data){
                    console.log(data);
            });
        });

    })
    .exec(function (err, replies) {});

Foam

Answer 3

Node-Redis v4 Update:

The redis package for node on npm has recently seen a major update (v4), and there's a much easier way to accomplish this.

Using the sendCommand function, you can run any redis command you wish without needing a dedicated function for it in the package, and can retrieve the output.

const keys = await client.sendCommand(["keys","*"]);
console.log(keys); // ["aXF","x9U","lOk",...]

Note:

Please do consider the point mentioned in Bill's answer:

Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases. This command is intended for debugging and special operations, such as changing your keyspace layout. Don't use KEYS in your regular application code. If you're looking for a way to find keys in a subset of your keyspace, consider using sets.

Falsecard

Answer 4

npm install node_redis 

is no more available now. Use this instead -

npm install redis

Lanctot