How to handle passing/returning array pointers to emscripten compiled code?

Question

From a windows 7 environment, I've used emcc to compile a simple c program which accepts an array and modifies it (see below).

double* displayArray(double *doubleVector) {

   for (int cnt = 0; cnt < 3; cnt++) 
       printf("doubleVector[%d] = %f\n", cnt, doubleVector[cnt]);

   doubleVector[0] = 98;
   doubleVector[1] = 99;
   doubleVector[2] = 100;

   for (int cnt1 = 0; cnt1 < 3; cnt1++) 
       printf("modified doubleVector[%d] = %f\n", cnt1, doubleVector[cnt1]);
    
   return doubleVector;
}

int main() {

   double d1, d2, d3;
   double array1[3];
   double *array2;

   array1[0] = 1.00000;
   array1[1] = 2.000000;
   array1[2] = 3.000000;

   array2 = displayArray(array1);

   for (int cntr =0; cntr < 3; cntr++)
       printf("array1[%d] = %f\n", cntr, array1[cntr]);
    
   for (int cnt = 0; cnt < 3; cnt++)
       printf("array2[%d] = %f\n", cnt, array2[cnt]);
    
   return 1;
}

Using the -o options for emcc, I generated a .html file which I loaded to a browser (Chrome).

python emcc displayArray7.c -o displayArray7.html -s EXPORTED_FUNCTIONS="['_main', '_displayArray'

Upon loading, I see that the output being generated within the browser window is as expected (see below).

doubleVector[0] = 1.000000
doubleVector[1] = 2.000000
doubleVector[2] = 3.000000
modified doubleVector[0] = 98.000000
modified doubleVector[1] = 99.000000
modified doubleVector[2] = 100.000000
array1[0] = 98.000000
array1[1] = 99.000000
array1[2] = 100.000000
array2[0] = 98.000000
array2[1] = 99.000000
array2[2] = 100.000000

However, when using the module.cwrap() command via javascript console and attempting to invoke the function directly (outside of main()) ,

> displayArray=Module.cwrap('displayArray', '[number]', ['[number]'])

> result = displayArray([1.0,2.0,3.0])
[1, 2, 3]
> result
[1, 2, 3]

I am seeing the following being generated/displayed in the browser which is NOT what I expect to see.

doubleVector[0] = 0.000000
doubleVector[1] = 0.000000
doubleVector[2] = 0.000000
modified doubleVector[0] = 100.000000
modified doubleVector[1] = 100.000000
modified doubleVector[2] = 100.000000   

I have the following questions:

1. Do I have the syntax correct for the return type and parameter listing correct in my call to Module.cwrap()? I've successfully run the simple, straight-forward example of int_sqrt() in the "Interacting with code" section of the tutorial which deals with passing non-pointer variables to the int_sqrt() routine.

2. Is there something different that is happening when arrays and/or pointers are passed to (or returned from) the emscripten-generated javascript code?

3. How is that the generated output in the browser of the function, displayArray(), works (as expected) when called from main(); but not via the javascript console?

Doit

I am having the same issue in angular 8. I tried to import WASM files using WebAssembly module. But i am not sure how you get the "Module" object. For me its says undefined. – Dollop

Answer 1

The expected format of Module.cwrap does allow for 'array's to be passed into the function, but will assert on result and fail if you attempt to return an array

displayArrayA=Module.cwrap('displayArray','array',['array'])
displayArrayA([1,2,3]) 
// Assertion failed: ccallFunc, fromC assert(type != 'array')

A second restriction of this is that incoming arrays are expected to be byte arrays, meaning you would need to convert any incoming double arrays into unsigned 8-bit numbers

displayArrayA=Module.cwrap('displayArray','number',['array'])
displayArrayA(new Uint8Array(new Float64Array([1,2,3]).buffer))

Calling the method this way will invoke your function, temporarily copying your arrays to the Emscripten stack which will be reset after your invoked function's execution, making the returned Array offset potentially unusable as it is freed stackspace.

It is much more preferable, if you want the results of your function, to allocate and preserve an array inside Emscriptens Heap system.

Emscripten code is only able to access memory that has been allocated within Emscripten's Heap space. The arrays that you are attempting to pass into the function are being allocated outside the heap that the Emscripten code is running against, and do not match the raw pointer type expected in the incoming arguments.

There are several ways that you can gain access to an array to pass data to your functions. All of these require Emscripen having knowledge of the location of your memory inside the emscripten Module.HEAP*, so the initial step is at some point to call the Emscripten "_malloc" function.

var offset = Module._malloc(24)

This would allow you to allocate the required 24 bytes in the Emscripten heap needed for your 3x 8-byte double array, and returns a Number offset in the Emscripten heap denoting the U8 TypedArray offset reserved for your array. This offset is your pointer, and will automatically work being passed into your cwrap displayArray function when it is configured to use the raw pointer offsets.

displayArray=Module.cwrap('displayArray','number',['number'])

At this point, if you wish to access or modify the contents of the array, as long as the malloc is valid, you have at least the following options:

1. Set the memory using a temporarily wrapped Float64 array, with no easy way to recover the value except the following 2 methods of access

   Module.HEAPF64.set(new Float64Array([1,2,3]), offset/8);
   displayArray(offset);
   

2. Module.setValue will use the 'double' hint to automatically modify the HEAPF64 offset, divided by 8.

   Module.setValue(offset, 1, 'double')
   Module.setValue(offset+8, 2, 'double')
   Module.setValue(offset+16, 3, 'double')
   displayArray(offset)
   var result = [];
   result[0] = Module.getValue(offset,'double'); //98
   result[1] = Module.getValue(offset+8,'double') //99
   result[2] = Module.getValue(offset+16,'double') //100
   

3. If you wish to use your pointer more extensively on the Javascript side, you can pull a subarray TypedArray off the HEAPF64 entry manually. This allows you to easily read the values once you have finished executing your function. This TypedArray is backed by the same heap as the rest of the Emscripten, so all changes performed on the Javascript side will be reflected on the Emscripten side and vice-versa:

   var doublePtr = Module.HEAPF64.subarray(offset/8, offset/8 + 3);
   doublePtr[0] = 1;
   doublePtr[1] = 2;
   doublePtr[2] = 3;
   // Although we have access directly to the HEAPF64 of the pointer,
   // we still refer to it by the pointer's byte offset when calling the function
   displayArray(offset);
   //doublePtr[] now contains the 98,99,100 values
   

Holly

Very helpful indeed, but what about "the other way around"? How can you modify the array from C code and use it in Javascript? – Plasmosome

I see a Module.malloc() but no Module.free() ? Is it handled by JS's GC ? – Exhalation

@Exhalation No, you have to call Module._free(...). – Supplement

Why do you divide the offset by 8? Doesn't the offset already points to the correct location? – Manche

@savram, HEAPF64 is an array of 64 bit numbers. offset is in bytes. – Commorancy

Answer 2

As a bonus to the other answer, here is a single convenience function for allocating a float64 array

function cArray(size) {
    var offset = Module._malloc(size * 8);
    Module.HEAPF64.set(new Float64Array(size), offset / 8);
    return {
        "data": Module.HEAPF64.subarray(offset / 8, offset / 8 + size),
        "offset": offset
    }
}
var myArray = cArray(3) // {data: Float64Array(3), offset: 5247688}


var displayArray = Module.cwrap('displayArray','number',['number'])
displayArray(myArray.offset)

Gives the output from the original function:

doubleVector[0] = 98.000000
doubleVector[1] = 99.000000
doubleVector[2] = 100.000000
modified doubleVector[0] = 98.000000
doubleVector[1] = 99.000000
modified doubleVector[2] = 100.000000

Otology