node.js store objects in redis

Question

Here is the thing - I want to store native JS (node.js) objects (flash sockets references) in redis under a certain key. When I do that with simple client.set() it's stored as a string. When I try to get value I get [object Object] - just a string.

Any chance to get this working? Here's my code:

  addSocket : function(sid, socket) {
    client.set(sid, socket);
  },

  getSocket : function(sid) {
    client.get(sid, function(err, reply) {
      // cant't get an object here. All I get is useless string
    });
  },

Maladapted

you cant store references to javascript in some form of database because the objects dissappear when the server goes down – Mathamathe

Answer 1

Downvoters: the context here is SET command and ability to store arbitrary objects.

No, you can't do that. You should accept the fact that Redis stores everything as a string (the protocol is text-based, after all). Redis may perform some optimizations and convert some values to integers, but that's its business, not yours.

If you want to store arbitrary objects in Redis, make sure that you serialize them before saving and de-serialize after retrieving.

I'm not sure if you can do that with socket objects, though. They simply describe a system resource (an open connection), after all (TCP sockets, for example). If you manage to serialize the description and deserialize it on another machine, that other machine won't have the connection.

Phrenology

Not sure how it was back in 2012, but at least now in 2016 Redis does store arbitrary objects, it does not store everything as a string. You just need to use different commands/methods to use that functionality. See Jason Loveman's answer for a better solution: https://mcmap.net/q/273037/-node-js-store-objects-in-redis – Kasten

@JMTyler: links to the docs? Redis does not store arbitrary objects. Just a few pre-determined types (string, hash, list, etc.). And if you store something via SET command, it'll be stored as a string. No way around this. – Phrenology

Yes, SET stores as a string, but that is obviously not the only command Redis has. You can't claim that how SET works is how all of Redis works. hash is another word for object depending on what language/platform you're using, so I don't see what your argument is. The fact that you mention hashes means you obviously know that you can store more complex data than just strings. – Kasten

@JMTyler: for starters, clearly the "everything as a string" part is in the context of SET command. Next, redis hash is not an equivalent for JS object, not even close. The only way to store arbitrary things in redis is to serialize and SET it (given that the things in question can be serialized/deserialized, of course). – Phrenology

Convert the object to string using JSON.stringify() before store it to the cache did the trick for me – Horticulture

Answer 2

Since the socket is of type Object, you need to convert the object to a string before storing and when retrieving the socket, need to convert it back to an object.

You can use

JSON.stringify(socket) 

to convert to a string and

JSON.parse(socketstr) 

to convert back to an object.

Edit:

Since the release of version 2.0.0, we are able to store objects as hashes into Redis.

client.hmset("hosts", "mjr", "1", "another", "23", "home", "1234");

client.hgetall("hosts", function (err, obj) {
    console.dir(obj);
});

https://redis.io/commands/hset

https://github.com/NodeRedis/node_redis

Weed

Now that's an answer! You can't do something is never an answer. tnx Shawn. – Ivon

Does it work with sockets, though? Can you deserialize a socket on another machine (or even the same one) and start, say, pushing data into it? – Phrenology

@moeiscool there are many answers on SO that should be accepted answers but question OPs do not always care about it and they do not change accepted answers ever. There should be moderator privilege to change accepted answers. – Lycian

A caveat about the edit, you can store multiple key-value pairs in the hash, but each of these keys and values are still strings. As such, arbitrary JS objects that contain non-string values still cannot be stored as-is, so stringifying them into JSON may still be the preferable choice. – Uniflorous

Answer 3

The solution below doesn't solve the whole point of using redis -- to share data across cluster instances. Storing an instance-specific id in redis will be meaningless to another instance that tries to use that id.

However, there is "hmset" which can be called with an object and it will set each object field as a separate redis field in the same key. And it will be converted back to an object when you call hgetall. Unfortunately, I don't think it handles nested objects or arrays within an object, only simple properties whose values can be stored by "toString()".

So an object like

client.hmset("myhashkey",{a:1, b:2, c:'xxx'})

works great and can be retrieved with

client.hmget("myhashkey", function(obj) {
   console.log(obj);
});

Not so much for:

client.hmset("myhashkeynested",{a:1, b:2, c:'xxx', d: { d1: 'x', d2: 'y'}});

Boiling

Retrieve by: client.hgetall("the-key-name", function(err,obj){//...}) , for hmget need field names. – Ungovernable

This answer needs more upvotes! The accepted answer is false. – Kasten

@JMTyler: this won't help in storing arbitrary js objects (sockets, in this case). – Phrenology

This is specifically for storing JS objects. It would successfully store any and all primitive type top-level properties of a socket object, since every object in JS extends the base Object. Worst case, if the socket object is nested, convert it I to a form that is not nested and you can still save it as an Object this way. – Kasten

Sadly a and b will return as strings when using hgetall, which makes this even more awkward. – Neelyneeoma

Answer 4

I also found this to be an incredibly useful tool, especially if you are channeling JSON from an API to your front-end:

node-redis-jsonify

If you're receiving a huge block of JSON and can't store as a specific hash, stringifying it while storing will allow you to retrieve the whole json instead of just "[object Object]".

Madwort

This package is a long winded way of simply doing JSON.stringify and JSON.parse. Check the source. – Limnology

Answer 5

I belive that when you store the object, internally, before storage object.toString() is called and that is the value that is stored.

({a: 1}).toString() # "[object Object]"

What you need to do is use JSON.encode and JSON.parse.

You cannot store (hidden, binary) references.
Otherwise, you might be able to make a correspondence between integers and sockets, and store integers.

Caucus

Answer 6

We can store object as a string using JSON.stringyfy(obj); and after we could retrieve original object using JSON.parse(obj);

const obj = {
   userId: socket.user._id,
   socketId: socket.id,
};
await client.set("user", JSON.stringify(obj));
const usersData = await client.get("user");

Peraea

Please don't post only code as an answer, but also provide an explanation what your code does and how it solves the problem of the question. Answers with an explanation are usually more helpful and of better quality, and are more likely to attract upvotes. – Raybin

Answer 7

You can set a JSON value in redis by using '.' in path param in (json.set and json.get)

example :

await client.json.set(TEST_KEY, '.', { example: 123 });
const value = await client.json.get(TEST_KEY, {
  path: '.',
});

Giulio

Answer 8

well if you think about it javascript objects are keys who's values may reference other objects, and in the case of sockets possibly native objects. So if redis is external to the executing javascript how will it be able to store a reference to to that object?

// a and b exist inside the executing javascript, not in the external O/S
a = {}
b = {}

var obj = {
  'one': a,
  'two': b
}

// how can redis possibly store references to obj, a or b when this application exits?
redis.set('obj', obj)

// same as..
redis.set('obj', obj.toString()) /*or*/ redis.set('obj', "" + obj)

// same as..
redis.set('obj', "[object Object]")


// the trick is to stringify the object
redis.set('obj', JSON.stringify(obj))

// same as..
redis.set('obj', "{'one':{},'two':{}}")

// here redis does not have to store the actual object references but rather a json string representing the object

// this could also work
redis.set('obj', "{'one':a,'two':b}")

// where you would then do:
obj = eval(redis.get('obj')) 
// provided that a and b have already been initialized

Decern

use eval is always a bad idea – Deon

Answer 9

You can save a reference to the JavaScript object, using a technique like this. Basically, it extends Object.prototype (doesn't have to) and you can call radd() to add a custom reference to rhash{}, then retrieve the object with rget(). The "reference" is a string so it will fit right into Redis. This means you won't have to .stringify() and .parse() to put/get a copy in the database. However, the object's data will be destroyed when Node shuts down unless its serialized.

var OBJECT = Object.prototype;
OBJECT.rhash = {};
OBJECT.rset = function(id, object) {
  OBJECT.rhash[id] = object;
  return id;
};
OBJECT.rget = function(id) {
  return OBJECT.rhash[id];
};

var dog = {
  name: "Skippy",
  food: "Bacon",
  bark: function() {
    alert(this.name + "!");
  }
};

var id = OBJECT.rset("a123", dog);
var ref = OBJECT.rget("a123");
ref.bark();

Nadenenader

Don't modify Object.prototype! – Dugaid

why do you need to bind this to the Object.prototype and make these get/set methods instead of just doing hash = {}, hash[key] = object? – Decern

Also, no serialization is happening here. The reference is to the original object, not a string. – Limnology