Typed Arrays in Gecko 2: Float32Array concatenation and expansion

Question

I'm a bit confused with Javascript Typed Arrays.

What I have are several Float32Array s, that have no concat method. I don't know how many are them in advance, btw. I'd like to concatenate them all inside another Float32Array, but:

• as I said before, there is no concatenation method
• if I try to write past the array length, the array is not expanded (aka this won't work - please note that event.frameBuffer and buffer are both Float32Array and that I don't know what the final length of my buffer will be):

---

var length_now = buffer.length;
for (var i = 0; i < event.frameBuffer.length; i += 1) {
      buffer [length_now + i] = event.frameBuffer[i];
}

The only solution I found is to copy the Float32Array in a regular array, that's definitely not what I want. How would you do, stackoverflowers?

Sorenson

Answer 1

Typed arrays are based on array buffers, which cannot be resized dynamically, so writing past the end of the array or using push() is not possible.

One way to achieve what you want would be to allocate a new Float32Array, large enough to contain both arrays, and perform an optimized copy:

function Float32Concat(first, second)
{
    var firstLength = first.length,
        result = new Float32Array(firstLength + second.length);

    result.set(first);
    result.set(second, firstLength);

    return result;
}

That would allow you to write:

buffer = Float32Concat(buffer, event.frameBuffer);

Emaciated

This is really great. Two questions: continuosly re-create a new typed array won't impact performances? and where did you find documentation about the .set function member? It's not in the page you linked. – Sorenson

@janesconference, well, it won't necessarily impact performance since set() is probably implemented natively and, as such, blindingly fast with memory blits, but it will impact memory since you can't just extend an existing type array. Depending on array size, if memory becomes scarce, thrashing might occur and performance will degrade tremendously as a result. – Turbosupercharger

@FrédéricHamidi: There is another issue than "native" implementation: Assume that you have n arrays with m elements you want to concatenate. Your complexity is then O(m^2), since you will copy larger and larger data blocks. The optimal solution is amortized O(m). – Barbicel

if anyone comes across this as I did wondering about the answer to @janesconference's 2nd question - where is there documentation for .set() - see developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… – Impassion

buffer = Float32Array.from([...buffera, ...bufferb]). spread operator works just fine no need to write anything. – Rimester

Answer 2

Or if you're trying to join N arrays:

// one-liner to sum the values in an array
function sum(a){
  return a.reduce(function(a,b){return a+b;},0);
}

// call this with an array of Uint8Array objects
function bufjoin(bufs){
  var lens=bufs.map(function(a){return a.length;});
  var aout=new Uint8Array(sum(lens));
  for (var i=0;i<bufs.length;++i){
    var start=sum(lens.slice(0,i));
    aout.set(bufs[i],start); // copy bufs[i] to aout at start position
  }
  return aout;
}

Snailpaced

Answer 3

I had the same issue, you can add the following to the prototype

Float32Array.prototype.concat = function() {
    var bytesPerIndex = 4,
        buffers = Array.prototype.slice.call(arguments);

    // add self
    buffers.unshift(this);

    buffers = buffers.map(function (item) {
        if (item instanceof Float32Array) {
            return item.buffer;
        } else if (item instanceof ArrayBuffer) {
            if (item.byteLength / bytesPerIndex % 1 !== 0) {
                throw new Error('One of the ArrayBuffers is not from a Float32Array');  
            }
            return item;
        } else {
            throw new Error('You can only concat Float32Array, or ArrayBuffers');
        }
    });

    var concatenatedByteLength = buffers
        .map(function (a) {return a.byteLength;})
        .reduce(function (a,b) {return a + b;}, 0);

    var concatenatedArray = new Float32Array(concatenatedByteLength / bytesPerIndex);

    var offset = 0;
    buffers.forEach(function (buffer, index) {
        concatenatedArray.set(new Float32Array(buffer), offset);
        offset += buffer.byteLength / bytesPerIndex;
    });

    return concatenatedArray;
};

now you can simply do

var array1 = new Float32Array(10000000),
    array2 = new Float32Array(10000000);

var array3 = array1.concat(array2);

Continuant