How can I pass an ArrayBuffer from JS to AssemblyScript/Wasm?

Question

I have a pretty straightforward piece of Typescript code that parses a specific data format, the input is a UInt8Array. I've optimized it as far as I can, but I think this rather simple parser should be able to run faster than I can make it run as JS. I wanted to try out writing it in web assembly using AssemblyScript to make sure I'm not running into any quirks of the Javascript engines.

As I figured out now, I can't just pass a TypedArray to Wasm and have it work automatically. As far as I understand, I can pass a pointer to the array and should be able to access this directly from Wasm without copying the array. But I can't get this to work with AssemblyScript.

The following is a minimal example that shows how I'm failing to pass an ArrayBuffer to Wasm.

The code to set up the Wasm export is mostly from the automatically generated boilerplate:

const fs = require("fs");
const compiled = new WebAssembly.Module(
  fs.readFileSync(__dirname + "/build/optimized.wasm")
);
const imports = {
  env: {
    abort(msgPtr, filePtr, line, column) {
      throw new Error(`index.ts: abort at [${line}:${column}]`);
    }
  }
};
Object.defineProperty(module, "exports", {
  get: () => new WebAssembly.Instance(compiled, imports).exports
});

The following code invokes the WASM, index.js is the glue code above.

const m = require("./index.js");
const data = new Uint8Array([1, 2, 3, 4, 5, 6, 7, 8]);
const result = m.parse(data.buffer);

And the AssemblyScript that is compiled to WASM is the following:

import "allocator/arena";

export function parse(offset: usize): number {
  return load<u8>(offset);
}

I get a "RuntimeError: memory access out of bounds" when I execute that code.

The major problem is that the errors I get back from Wasm are simply not helpful to figure this out on my own. I'm obviously missing some major aspects of how this actually works behind the scenes.

How do I actually pass a TypedArray or an ArrayBuffer from JS to Wasm using AssemblyScript?

Hyphen

Is offset expected to be a property of the Unit8Array? Are you trying to get the byteOffset? Does parse expect the parameter to be a Unit8Array or an integer? – Rajah

@Rajah the ArrayBuffer isn't actually passed to the WASM, that's what I originally thought. But every time I tried to access a property like length of the array it threw an error. As far as I understand, what is passed to WASM is only a pointer to the location of the ArrayBuffer in memory. But I'm probably wrong on that, though I'm pretty sure I'm right that the real ArrayBuffer isn't passed in. – Hyphen

What does parse expect as argument? – Rajah

@Rajah the address in memory which I can use to execute the load command. In the real code I'd iterate over the array, so I'd also pass in the length. I tried treating the parameter as a TypedArray, and that didn't work. As far as I've read, if I pass in an ArrayBuffer I get a pointer to the array on the WASM side. – Hyphen

I don't familiar with AssemblyScript but much experienced with C/C++ for WASM. In any language you use, you cannot pass an array object, but copy it to WAS's heap region. This is because WASM is strictly forbidden to touch any memory outside of WASM's heap and WASM only cannot understand arrary as a type but only number types. In WASM for C, you should call malloc of the same size of ArrayBuffer, pass the pointer to JS, and then convert it Uint8Array, and copy the ArrayBuffer to the Uint8Array using Uinit8Array.set() method. There is really no other way. I hope this would help. – Trottier

probably related to this issue – Hooper

Answer 1

In AssemblyScript, there are many ways to read data from the memory. The quickest and fastest way to get this data is to use a linked function in your module's function imports to return a pointer to the data itself.

let myData = new Float64Array(100); // have some data in AssemblyScript

// We should specify the location of our linked function
@external("env", "sendFloat64Array")
declare function sendFloat64Array(pointer: usize, length: i32): void;

/**
 * The underlying array buffer has a special property called `data` which
 * points to the start of the memory.
 */
sendFloat64Data(myData.buffer.data, myData.length);

Then in JavaScript, we can use the Float64Array constructor inside our linked function to return the values directly.

/**
 * This is the fastest way to receive the data. Add a linked function like this.
 */
imports.env.sendFloat64Array = function sendFloat64Array(pointer, length) {
  var data = new Float64Array(wasmmodule.memory.buffer, pointer, length);
};

However, there is a much clearer way to obtain the data, and it involves returning a reference from AssemblyScript, and then using the AssemblyScript loader.

let myData = new Float64Array(100); // have some data in AssemblyScript

export function getData(): Float64Array {
  return myData;
}

Then in JavaScript, we can use the ASUtil loader provided by AssemblyScript.

import { instantiateStreaming } from "assemblyscript/lib/loader";

let wasm: ASUtil = await instantiateStreaming(fetch("myAssemblyScriptModule.wasm"), imports);

let dataReference: number = wasm.getData();
let data: Float64Array = wasm.getArray(Float64Array, dataReference);

I highly recommend using the second example for code clarity reasons, unless performance is absolutely critical.

Good luck with your AssemblyScript project!

Endsley

Stupid question, perhaps, but where is "getArray" supposed to come from (or what should it look like, since it appears that it's something defined in our AS module)? – Dallas

Thanks! That is indeed what I needed (and discovered)! – Dallas