Convert buffer to array

Question

I am setting memcached with

$memcached->set("item" , ["1" => "hello"]);

anything work in PHP ,

In Node.js with memcached plugin , I get a buffer instead of array in result

<Buffer 61 3a 25 61 34 3a>

I can not convert such buffer to array

In Node.js :

memcached.get("item" , function(err, data) {
  console.log(data);
}

Do you have any way ?

Schiro

What happens if you just call $memcached->set("item" , "hello"); in php? – Phonic

it get string and no problem – Schiro

how about $memcached->set("item" , ["1", "hello"]); I wonder if PHP is not writing the array properly – Phonic

I can ge that as array well , may be the php serilize that , also I got array with this code phpjs.unserialize(data.toString()); – Schiro

Answer 1

arr = [...buffer]

ES6 introduced a lot of other features, besides buffers.

You can even easily append like this:

arr.push(...buffer)

The ... operator expands enumerables such as arrays and buffers when used in array. It also expands them into separate function arguments.

---

Yes, it's also faster:

... : x100000: 835.850ms

Slice call from prototype : x100000: 2118.513ms

var array,
    buffer = new Buffer([1, 4, 4, 5, 6, 7, 5, 3, 5, 67, 7, 4, 3, 5, 76, 234, 24, 235, 24, 4, 234, 234, 234, 325, 32, 6246, 8, 89, 689, 7687, 56, 54, 643, 32, 213, 2134, 235, 346, 45756, 857, 987, 0790, 89, 57, 5, 32, 423, 54, 6, 765, 65, 745, 4, 34, 543, 43, 3, 3, 3, 34, 3, 63, 63, 35, 7, 537, 35, 75, 754, 7, 23, 234, 43, 6, 247, 35, 54, 745, 767, 5, 3, 2, 2, 6, 7, 32, 3, 56, 346, 4, 32, 32, 3, 4, 45, 5, 34, 45, 43, 43]),
    iter = 100000;

array = buffer;

console.time("... : x" + iter);
for (var i = iter; i--;) array = [...buffer]
console.timeEnd("... : x" + iter);

console.time("Apply/call/etc : x" + iter);
for (var i = iter; i--;) array = Array.prototype.slice.call(buffer, 0)
console.timeEnd("Apply/call/etc : x" + iter);

Unblown

Answer 2

There is another way to convert to array of integers

Using toJSON()

Buffer.from('Text of example').toJSON()
{ type: 'Buffer',data: [ 84, 101, 120, 116, 32, 111, 102, 32, 101, 120, 97, 109, 112, 108, 101 ] }

// simple get data
Buffer.from('Text of example').toJSON().data
[ 84, 101, 120, 116, 32, 111, 102, 32, 101, 120, 97, 109, 112, 108, 101 ]

Example of benchmark

// I took this from @user4584267's answer
const buffer = new Buffer([1, 4, 4, 5, 6, 7, 5, 3, 5, 67, 7, 4, 3, 5, 76, 234, 24, 235, 24, 4, 234, 234, 234, 325, 32, 6246, 8, 89, 689, 7687, 56, 54, 643, 32, 213, 2134, 235, 346, 45756, 857, 987, 0790, 89, 57, 5, 32, 423, 54, 6, 765, 65, 745, 4, 34, 543, 43, 3, 3, 3, 34, 3, 63, 63, 35, 7, 537, 35, 75, 754, 7, 23, 234, 43, 6, 247, 35, 54, 745, 767, 5, 3, 2, 2, 6, 7, 32, 3, 56, 346, 4, 32, 32, 3, 4, 45, 5, 34, 45, 43, 43]);
let array = null;
const iterations = 100000;

console.time("...buffer");
for (let i = iterations; i=i-1;) array = [...buffer]
console.timeEnd("...buffer");

console.time("array.prototype.slice.call");
for (let i = iterations; i=i-1;) array = Array.prototype.slice.call(buffer, 0)
console.timeEnd("array.prototype.slice.call");

console.time("toJSON().data");
for (let i = iterations; i=i-1;) array = buffer.toJSON().data
console.timeEnd("toJSON().data");

OUTPUT

...buffer: 559.932ms
array.prototype.slice.call: 1176.535ms
toJSON().data: 30.571ms

or if you want more profesional and custom function in Buffer use this:

Buffer.prototype.toArrayInteger = function(){
    if (this.length > 0) {
        const data = new Array(this.length);
        for (let i = 0; i < this.length; i=i+1)
            data[i] = this[i];
        return data;
    }
    return [];
}

Example of benchmark:

const buffer = new Buffer([1, 4, 4, 5, 6, 7, 5, 3, 5, 67, 7, 4, 3, 5, 76, 234, 24, 235, 24, 4, 234, 234, 234, 325, 32, 6246, 8, 89, 689, 7687, 56, 54, 643, 32, 213, 2134, 235, 346, 45756, 857, 987, 0790, 89, 57, 5, 32, 423, 54, 6, 765, 65, 745, 4, 34, 543, 43, 3, 3, 3, 34, 3, 63, 63, 35, 7, 537, 35, 75, 754, 7, 23, 234, 43, 6, 247, 35, 54, 745, 767, 5, 3, 2, 2, 6, 7, 32, 3, 56, 346, 4, 32, 32, 3, 4, 45, 5, 34, 45, 43, 43]);
let array = null;
const iterations = 100000;

console.time("toArrayInteger");
for (let i = iterations; i=i-1;) buffer.toArrayInteger();
console.timeEnd("toArrayInteger");

Ouput:

toArrayInteger: 28.714ms

Note: In the last example I copied a function from Buffer.toJSON and custom it a lite

Vaginitis

Answer 3

Here you go:

var buffer = new Buffer([1,2,3])
var arr = Array.prototype.slice.call(buffer, 0)
console.log(arr)

Ethelinda

Could you add a little more detail to your answer? Thanks! – Arrear

What additional detail is needed?! – Berryman

Super concise answer that works! Thanks I was struggling to read the silly paragraphs. – Goar

Answer 4

I haven't used memcached so I am not sure just what this buffer represents or what you want to have instead. Sorry. Here is a function to split a buffer up into an array of bytes. More at node.js Buffer docs, hope it helps!

var hex = new Buffer("613a2561343a", "hex");
var l = hex.length; // in bytes
var output = [];
for(var i = 0; i < l; i++){
  var char = hex.toString('hex',i,i+1); // i is byte index of hex
  output.push(char);
};
console.log(output);
// output: [ '61', '3a', '25', '61', '34', '3a' ]

Boscage

it just split the buffer data , it do not convert each item to used character – Schiro

Gotcha. Maybe there is a way to make PHP save a non-encoded string into memcached? I don't know either very well, sry - good luck! – Boscage

Answer 5

You can also use Array.from:

memcached.get("item" , function(err, data) {
  console.log(Array.from(data));
}

Brockington

Answer 6

I have a solution, although I am currently trying to find a better one:

function bufToArray(buffer) {
  let array = new Array();
  for (data of buffer.values()) array.push(data);
  return array;
}

EDIT : I found a simpler way:

var buffer = Buffer.from('NodeJS rocks!')
var array = new Function(`return [${Array.prototype.slice.call(buffer, 0)}]`)

But, like someone already said, [...buffer] is faster (and more code efficient).

You can also use new Uint8Array(buffer [, byteOffset [, length]]);

Inexpensive

'a simpler way' is eval which is totally uncalled-for here. – Revulsion

Why in the world do you need a function constructor?? Is it more performant or in any way better than var array = Array.prototype.slice.call(buffer, 0);? – Commingle

Answer 7

In interent , there was no information about that , but I have found the convert way

In nodejs , I have to use :

var arrayobject = phpjs.unserialize(data.toString());

but , it is very stupid way for getting array , it seem that php serilzie the data when setting memcache .

Schiro

You could try to store the array as JSON. $memcached->set("item" , json_encode(["1" => "hello"]));. Then decode the JSON in node.js. – Rothko

Yes , that is way , but think we get high rate on encode decode per sec , in such way , that is not efficent way – Schiro