Javascript ArrayBuffer to Hex

Question

I've got a Javascript ArrayBuffer that I would like to be converted into a hex string.

Anyone knows of a function that I can call or a pre written function already out there?

I have only been able to find arraybuffer to string functions, but I want the hexdump of the array buffer instead.

Fosterfosterage

How about number.toString(16) – Garlic

Answer 1

function buf2hex(buffer) { // buffer is an ArrayBuffer
  return [...new Uint8Array(buffer)]
      .map(x => x.toString(16).padStart(2, '0'))
      .join('');
}

// EXAMPLE:
const buffer = new Uint8Array([ 4, 8, 12, 16 ]).buffer;
console.log(buf2hex(buffer)); // = 04080c10

This function works in four steps:

1. Converts the buffer into an array.
2. For each x the array, it converts that element to a hex string (e.g., 12 becomes c).
3. Then it takes that hex string and left pads it with zeros (e.g., c becomes 0c).
4. Finally, it takes all of the hex values and joins them into a single string.

Below is another longer implementation that is a little easier to understand, but essentially does the same thing:

function buf2hex(buffer) { // buffer is an ArrayBuffer
  // create a byte array (Uint8Array) that we can use to read the array buffer
  const byteArray = new Uint8Array(buffer);
  
  // for each element, we want to get its two-digit hexadecimal representation
  const hexParts = [];
  for(let i = 0; i < byteArray.length; i++) {
    // convert value to hexadecimal
    const hex = byteArray[i].toString(16);
    
    // pad with zeros to length 2
    const paddedHex = ('00' + hex).slice(-2);
    
    // push to array
    hexParts.push(paddedHex);
  }
  
  // join all the hex values of the elements into a single string
  return hexParts.join('');
}

// EXAMPLE:
const buffer = new Uint8Array([ 4, 8, 12, 16 ]).buffer;
console.log(buf2hex(buffer)); // = 04080c10

Euphemism

Not really sure how this works, but seems to match the same output as the c# ... string myString = BitConverter.ToString(new byte[] {120,144,107}); – Kutenai

@Kutenai See the code snippet at the bottom, it's a more detailed explanation of how the code works. – Euphemism

@etienne-martin Great example, ES6 works. I'm trying your ES5 example, i run it with node.js and answer coming back as 04040404. Any idea ??? In ES6 example you have Array.prototype.map.call, but not in ES5, is that supposed to be so ? – Concertize

Hi, when i run ES5, it's coming back as 04040404, any suggestions ? – Concertize

@Concertize It's not an ES5 example, and I've never claimed or intended it to be (it's just longer for clarity). The problem is the ES6 let and const statements, which are partially but not fully supported in Node.js 4 and 5 (which leads to incorrect behavior). Just replace all of them with var statements instead, or upgrade to Node.js 6 or 7, and it should work fine. – Euphemism

@Concertize And the Array.prototype.map.call call in the first example is replaced with a for loop in the second; they're functionally equivalent, though. – Euphemism

@Frxstrem, thanks very much for the reply, let me do more research for ES5 for IE. – Concertize

@Frxstrem I think you can pad with '0' rather than '00' because the shortest x.toString(16) is still one character – Incarcerate

This has become a very popular snippet using the ES6 style. How about the reverse function of this? hex2buf? – Gummosis

Answer 2

Here are several methods for encoding an ArrayBuffer to hex, in order of speed. All methods were tested in Firefox initially, but afterwards I went and tested in Chrome (V8). In Chrome the methods were mostly in the same order but it did have slight differenences--the important thing is that #1 is the fastest method in all environments by a huge margin.

If you want to see how slow the currently selected answer is, you can go ahead and scroll to the bottom of this list.

TL;DR

Method #1 (just below this) is the fastest method I tested for encoding to a hex string. If, for some very good reason, you need to support IE, you may need to replace the .padStart call with the .slice trick used in method #6 when precomputing the hex octets to make sure every octet is 2 characters.

1. Precomputed Hex Octets w/ for Loop (Fastest/Baseline)

This approach computes the 2-character hex octets for every possible value of an unsigned byte: [0, 255], and then just maps each value in the ArrayBuffer through the array of octet strings. Credit to Aaron Watters for the original answer using this method.

NOTE: as mentioned by Cref, you may get a performance boost in V8 (Chromium/Chrome/Edge/Brave/etc.) by using the loop to just concatenate hex octets into one big string as you go and then returning the string after the loop. V8 seems to optimize string concatenation very well while Firefox performed better with building up an array and then .joining it into a string at the end as I did in the code below. That would probably be a micro-optimization subject to change with the whims of optimizing JS compilers though..

const byteToHex = [];

for (let n = 0; n <= 0xff; ++n)
{
    const hexOctet = n.toString(16).padStart(2, "0");
    byteToHex.push(hexOctet);
}

function hex(arrayBuffer)
{
    const buff = new Uint8Array(arrayBuffer);
    const hexOctets = []; // new Array(buff.length) is even faster (preallocates necessary array size), then use hexOctets[i] instead of .push()

    for (let i = 0; i < buff.length; ++i)
        hexOctets.push(byteToHex[buff[i]]);

    return hexOctets.join("");
}

2. Precomputed Hex Octets w/ Array.map (~30% slower)

Same as the above method, where we precompute an array in which the value for each index is the hex string for the index's value, but we use a hack where we call the Array prototype's map() method with the buffer. This is a more functional approach, but if you really want speed you will always use for loops rather than ES6 array methods, as all modern JS engines optimize them much better.

IMPORTANT: You cannot use new Uint8Array(arrayBuffer).map(...). Although Uint8Array implements the ArrayLike interface, its map method will return another Uint8Array which cannot contain strings (hex octets in our case), hence the Array prototype hack.

function hex(arrayBuffer)
{
    return Array.prototype.map.call(
        new Uint8Array(arrayBuffer),
        n => byteToHex[n]
    ).join("");
}

3. Precomputed ASCII Character Codes (~230% slower)

Well this was a disappointing experiment. I wrote up this function because I thought it would be even faster than Aaron's precomputed hex octets--boy was I wrong LOL. While Aaron maps entire bytes to their corresponding 2-character hex codes, this solution uses bitshifting to get the hex character for the first 4 bits in each byte and then the one for the last 4 and uses String.fromCharCode(). Honestly I think String.fromCharCode() must just be poorly optimized, since it is not used by very many people and is low on browser vendors' lists of priorities.

const asciiCodes = new Uint8Array(
    Array.prototype.map.call(
        "0123456789abcdef",
        char => char.charCodeAt()
    )
);

function hex(arrayBuffer)
{
    const buff = new Uint8Array(arrayBuffer);
    const charCodes = new Uint8Array(buff.length * 2);

    for (let i = 0; i < buff.length; ++i)
    {
        charCodes[i * 2] = asciiCodes[buff[i] >>> 4];
        charCodes[i * 2 + 1] = asciiCodes[buff[i] & 0xf];
    }

    return String.fromCharCode(...charCodes);
}

4. Array.prototype.map() w/ padStart() (~290% slower)

This method maps an array of bytes using the Number.toString() method to get the hex and then padding the octet with a "0" if necessary via the String.padStart() method.

IMPORTANT: String.padStart() is a relative new standard, so you should not use this or method #5 if you are planning on supporting browsers older than 2017 or so or Internet Explorer. TBH if your users are still using IE you should probably just go to their houses at this point and install Chrome/Firefox. Do us all a favor. :^D

function hex(arrayBuffer)
{
    return Array.prototype.map.call(
        new Uint8Array(arrayBuffer),
        n => n.toString(16).padStart(2, "0")
    ).join("");
}

5. Array.from().map() w/ padStart() (~370% slower)

This is the same as #4 but instead of the Array prototype hack, we create an actual number array from the Uint8Array and call map() on that directly. We pay in speed though.

function hex(arrayBuffer)
{
    return Array.from(new Uint8Array(arrayBuffer))
        .map(n => n.toString(16).padStart(2, "0"))
        .join("");
}

6. Array.prototype.map() w/ slice() (~450% slower)

This is the selected answer, do not use this unless you are a typical web developer and performance makes you uneasy (answer #1 is supported by just as many browsers).

function hex(arrayBuffer)
{
    return Array.prototype.map.call(
        new Uint8Array(arrayBuffer),
        n => ("0" + n.toString(16)).slice(-2)
    ).join("");
}

Lesson #1

Precomputing stuff can be a very effective memory-for-speed tradeoff sometimes. In theory, the array of precomputed hex octets can be stored in just 1024 bytes (256 possible hex values ⨉ 2 characters/value ⨉ 2 bytes/character for a UTF-16 string representation used by most/all browsers), which is nothing in a modern computer. Realistically there are some more bytes in there used for storing the array and string lengths and maybe type information since this is JavaScript, but the memory usage is still negligible for a massive performance improvement.

Lesson #2

Help out the optimizing compiler. The browser's JavaScript compiler regularly attempts to understand your code and break it down to the fastest possible machine code for your CPU to execute. Because JavaScript is a very dynamic language, this can be hard to do and sometimes the browser just gives up and leaves all sorts of type checks and worse under-the-hood because it can't be sure that x will indeed be a string or number, and vice versa. Using modern functional programming additions like the .map method of the built-in Array class can cause headaches for the browser because callback functions can capture outside variables and do all sorts of other things that often hurt performance. For-loops are well-studied and relatively simple constructs, so the browser developers have incorporated all sorts of tricks for the compiler to optimize your JavaScript for-loops. Keep it simple.

Trill

why not use reduce? – Lingenfelter

@Lingenfelter You absolutely could use reduce. In fact, I'm guessing it's much better than the solutions I included that use map. I never tested reduce but unfortunately I'm guessing it's STILL slower than the plain old for-loop. From my experience, JS engines just have a hard time optimize the functional programming methods like map/reduce/forEach/etc. I have other things to do right now, but if you test and find that reduce does perform as well as or better than the for-loop please tell me and I will edit the post. – Trill

Very nice, I'd say – Doubler

const byteToHex = Array(256).fill(0).map((_, n) => n.toString(16).padStart(2, "0")) – Ciceronian

Answer 3

Here is a sweet ES6 solution, using padStart and avoiding the quite confusing prototype-call-based solution of the accepted answer. It is actually faster as well.

function bufferToHex (buffer) {
    return [...new Uint8Array (buffer)]
        .map (b => b.toString (16).padStart (2, "0"))
        .join ("");
}

How this works:

1. An Array is created from a Uint8Array holding the buffer data. This is so we can modify the array to hold string values later.
2. All the Array items are mapped to their hex codes and padded with 0 characters.
3. The array is joined into a full string.

Mussorgsky

The spread syntax [...new Uint8Array(buffer)] could be slightly faster (if other tests are an indication). – Reincarnation

After someone's edit, accepted answer code is now identical to this =D – Bulge

Answer 4

The simplest way to convert arraybuffer to hex:

const buffer = new Uint8Array([ 4, 8, 12, 16 ]);
console.log(Buffer.from(buffer).toString("hex")); // = 04080c10

For supporting Buffer in Broswer, check this: https://github.com/feross/buffer

Seism

Works great for Node. Sadly, unavailable in browser. – Langlauf

Worked for me in 2021 – Foretopsail

there's buffer polyfill for browser to support node API, should solve issue – Bearberry

For something like const arr = new Uint16Array([3,2,1]) use Buffer.from(arr.buffer).toString('hex'). Note if I am reading the docs correctly, the result you see depends on the endianess of your host's architecture! Hence on Intel you see 030002000100 and on Motorola 000300020001 (but I have not tested the latter as I lack the hardware for this). So do not be puzzled (like me) when Buffer.from(new Uint32Array([0x12345678]).buffer).toString('hex') gives string "78563412" ;) – Watchtower

Answer 5

Here is another solution which is, on Chrome (and probably node too) about 3x faster than the other suggestions using map and toString:

function bufferToHex(buffer) {
    var s = '', h = '0123456789ABCDEF';
    (new Uint8Array(buffer)).forEach((v) => { s += h[v >> 4] + h[v & 15]; });
    return s;
}

Additional bonus: you can easily choose uppercase/lowercase output.

See bench here: http://jsben.ch/Vjx2V

Grazia

Interesting approach! +1 – Mussorgsky

Nice solution! To be compatible with common output as e.g. Buffer.from(buffer).toString('hex') you should use lower case characters i.e. '0123456789abcdef'. – Hainaut

Answer 6

uint8array.reduce((a, b) => a + b.toString(16).padStart(2, '0'), '')

Surprisingly, it is important to use reduce instead of map. This is because map is reimplemented for typed arrays to return a typed array for each element, instead of a uint8.

Foreplay

Answer 7

The following solution uses precomputed lookup tables for both forward and backward conversion.

// look up tables
var to_hex_array = [];
var to_byte_map = {};
for (var ord=0; ord<=0xff; ord++) {
    var s = ord.toString(16);
    if (s.length < 2) {
        s = "0" + s;
    }
    to_hex_array.push(s);
    to_byte_map[s] = ord;
}

// converter using lookups
function bufferToHex2(buffer) {
    var hex_array = [];
    //(new Uint8Array(buffer)).forEach((v) => { hex_array.push(to_hex_array[v]) });
    for (var i=0; i<buffer.length; i++) {
        hex_array.push(to_hex_array[buffer[i]]);
    }
    return hex_array.join('')
}
// reverse conversion using lookups
function hexToBuffer(s) {
    var length2 = s.length;
    if ((length2 % 2) != 0) {
        throw "hex string must have length a multiple of 2";
    }
    var length = length2 / 2;
    var result = new Uint8Array(length);
    for (var i=0; i<length; i++) {
        var i2 = i * 2;
        var b = s.substring(i2, i2 + 2);
        result[i] = to_byte_map[b];
    }
    return result;
}

This solution is faster than the winner of the previous benchmark: http://jsben.ch/owCk5 tested in both Chrome and Firefox on a Mac laptop. Also see the benchmark code for a test validation function.

[edit: I change the forEach to a for loop and now it's even faster.]

Allopatric

This should be the accepted answer, as it is certainly the fastest. I am not sure, but you might even squeeze a little more speed by mapping to ASCII char codes and using String.fromCharCodes() instead of joining an array of strings. That is pure speculation though and I need to test. – Trill

^^^ I went and tested mapping to ASCII code points (using a Uint8Array and going the full 9 yards regarding optimization) is WAY SLOWER than all the methods on this page and more that I tested. Color me surprised.. – Trill

Answer 8

This one's inspired by Sam Claus' #1 which is indeed the fastest method on here. Still, I've found that using plain string concatenation instead of using an array as a string buffer is even faster! At least it is on Chrome. (which is V8 which is almost every browser these days and NodeJS)

const len = 0x100, byteToHex = new Array(len), char = String.fromCharCode;
let n = 0;
for (; n < 0x0a; ++n) byteToHex[n] = '0' + n;
for (; n < 0x10; ++n) byteToHex[n] = '0' + char(n + 87);
for (; n < len; ++n) byteToHex[n] = n.toString(16);
function byteArrayToHex(byteArray) {
    const l = byteArray.length;
    let hex = '';
    for (let i = 0; i < l; ++i) hex += byteToHex[byteArray[i] % len];
    return hex;
}
function bufferToHex(arrayBuffer) {
    return byteArrayToHex(new Uint8Array(arrayBuffer));
}

Joyce

Thanks, I should have mentioned that. I had the same result on Chrome, but it's worth noting that on Firefox, .join with an array of strings was faster (don't know if that's still the case). – Trill

Answer 9

If you find this and need to encode / decode even faster and potentially reduce the amount of memory needed, then the already provided answers, then this might work for you.

It leverages the TextEncoder, which is present in any browser (https://caniuse.com/textencoder) and in nodejs, to concat the resulting hex string or to get the hex charcodes.

In nodejs you should use the option already provided like so:

function nodeEncode(arr: Uint8Array) {
  return Buffer.from(arr).toString('hex');
}

function nodeDecode(hexString: string) {
  return Uint8Array.from(Buffer.from(hexString, 'hex'));
}

But in the browser environment you can use the TextEncoder like so:

const nibbleIntegerToHexCharCode = new TextEncoder().encode("0123456789abcdef");

function uint8ArrayToHexString(input: Uint8Array) {
  const output = new Uint8Array(input.length * 2);

  for (let i = 0; i < input.length; i++) {
    const v = input[i];
    output[i * 2 + 0] = nibbleIntegerToHexCharCode[(v & 0xf0) >> 4];
    output[i * 2 + 1] = nibbleIntegerToHexCharCode[(v & 0x0f)];
  }

  return new TextDecoder().decode(output);
}

const charCodeToNibbleInteger = new Uint8Array(0xff + 1);

for (let i = 0; i < charCodeToNibbleInteger.length; i++)
  charCodeToNibbleInteger[i] = nibbleIntegerToHexCharCode.findIndex(v => v == i);

function hexStringToUInt8Array(input: string) {
  const encodedInput = new TextEncoder().encode(input);
  const output = new Uint8Array(encodedInput.length / 2);

  for (let i = 0; i < output.length; i++) {
    const upper = charCodeToNibbleInteger[encodedInput[i * 2 + 0]] << 4;
    const lower = charCodeToNibbleInteger[encodedInput[i * 2 + 1]];
    output[i] = upper + lower;
  }

  return output;
}

The output of the hex function the nodeEncoder function and the uint8ArrayToHexString function are identical but the time to compute them differs.

For 22 MB of UInt8Array:

• nodeEncoder takes 70 ms
• uint8ArrayToHexString 280 ms
• hex takes 1400 ms

There might also be a stark difference in the amount of memory used.

Andra

Answer 10

In Node, we can use Buffer.from(uint8array, "hex")

Anastigmatic

Answer 11

I use this to hexdump ArrayBuffers the same way that Node dumps Buffers.

function pad(n: string, width: number, z = '0') {
    return n.length >= width ? n : new Array(width - n.length + 1).join(z) + n;
}
function hexdump(buf: ArrayBuffer) {
    let view = new Uint8Array(buf);
    let hex = Array.from(view).map(v => this.pad(v.toString(16), 2));
    return `<Buffer ${hex.join(" ")}>`;
}

Example (with transpiled js version):

const buffer = new Uint8Array([ 4, 8, 12, 16 ]).buffer;
console.log(hexdump(buffer)); // <Buffer 04 08 0c 10>

Odelsting

is it TypeScript? :/ – Progenitor

Yes, but just take out the type annotations and it's javascript again – Odelsting

Answer 12

I had the same task, here is what I found to be the best solution

const buff = new Uint8Array(arrayBuffer);
const buffLen = buff.length;
const hex = new Array(buffLen);
for(let i=0; i<buffLen; i++){
    hex[i] = ("0"+ buff[i].toString(16)).slice(-2);
}
console.log(hex);

Goodden