How to produce a `ArrayBuffer` from `bytes` using `js_of_ocaml`

Question

I am building a JavaScript library that is implemented in Ocaml and compiled to JavaScript using js_of_ocaml.

One of my Ocaml function returns a string with binary data. How can I expose that using js_of_ocaml as a ArrayBuffer?

Rerun

Answer 1

When you compile to javascript, manipulating binary data in strings is extremely bug prone!

The underlying reason is questionable choice of js_of_ocaml: Because javascript strings are encoded in UTF16 whereas OCaml ones are (implicitly) encoded in UTF8, js_of_ocaml tries to navigate in between the 2. Therefore, when it encounters a "character" whose code is > 127, js_of_ocaml converts it which is a disaster if it is, in fact, raw binary data!

The solution is to manipulate bigstrings instead of strings. Bigstrings are (char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t in raw OCaml but more and more libraries aliases them. Especially, they are Typed_array.​Bigstring.t in js_of_ocaml (where you can see functions to convert from and to ArrayBuffer)

If your function does work by magic on string once compiled in javascript, there are translation function in between bigstrings and strings in several places. For example the bigstring library: http://c-cube.github.io/ocaml-bigstring/ but these functions are also available in Lwt_bytes of lwt

You can see an other question on the same subject (including ways to manipulate OCaml string in javascript while not touching them at all using gen_js_api) at https://discuss.ocaml.org/t/handling-binary-data-in-ocaml-and-javascript/1519

Pickaback

Thanks! So far it all seems to work using string, somehow… – Rerun

@JoachimBreitner, you might not run into issues as long as binary strings are only passed from JS to ML, whereas the conversion pitfall would be in the other direction. – Pallet