How to message child process in Firefox add-on like Chrome native messaging

Question

I am trying to emulate Chrome's native messaging feature using Firefox's add-on SDK. Specifically, I'm using the child_process module along with the emit method to communicate with a python child process.

I am able to successfully send messages to the child process, but I am having trouble getting messages sent back to the add-on. Chrome's native messaging feature uses stdin/stdout. The first 4 bytes of every message in both directions represents the size in bytes of the following message so the receiver knows how much to read. Here's what I have so far:

Add-on to Child Process

var utf8 = new TextEncoder("utf-8").encode(message);
var latin = new TextDecoder("latin1").decode(utf8);

emit(childProcess.stdin, "data", new TextDecoder("latin1").decode(new Uint32Array([utf8.length])));
emit(childProcess.stdin, "data", latin);
emit(childProcess.stdin, "end");

Child Process (Python) from Add-on

text_length_bytes = sys.stdin.read(4)
text_length = struct.unpack('i', text_length_bytes)[0]
text = sys.stdin.read(text_length).decode('utf-8')

Child Process to Add-on

sys.stdout.write(struct.pack('I', len(message)))
sys.stdout.write(message)
sys.stdout.flush()

Add-on from Child Process

This is where I'm struggling. I have it working when the length is less than 255. For instance, if the length is 55, this works:

childProcess.stdout.on('data', (data) => { // data is '7' (55 UTF-8 encoded)
    var utf8Encoded = new TextEncoder("utf-8).encode(data);
    console.log(utf8Encoded[0]); // 55
}

But, like I said, it does not work for all numbers. I'm sure I have to do something with TypedArrays, but I'm struggling to put everything together.

Rudyrudyard

What if you send it without encoding it to utf8? Do the encoding on the other side? In OS.File we have to deal with arraybuffer's only, and handle encoding on the ends. – Meanie

Answer 1

The problem here, is that Firefox is trying to read stdout as UTF-8 stream by default. Since UTF-8 doesn't use the full first byte, you get corrupted characters for example for 255. The solution is to tell Firefox to read in binary encoding, which means you'll have to manually parse the actual message content later on.

var childProcess = spawn("mybin", [ '-a' ], { encoding: null });

Your listener would then work like

var decoder = new TextDecoder("utf-8");
var readIncoming = (data) => {
    // read the first four bytes, which indicate the size of the following message
    var size = (new Uint32Array(data.subarray(0, 4).buffer))[0];
    //TODO: handle size > data.byteLength - 4
    // read the message
    var message = decoder.decode(data.subarray(4, size));
    //TODO: do stuff with message
    // Read the next message if there are more bytes.
    if(data.byteLength > 4 + size)
        readIncoming(data.subarray(4 + size));
};
childProcess.stdout.on('data', (data) => {
    // convert the data string to a byte array
    // The bytes got converted by char code, see https://dxr.mozilla.org/mozilla-central/source/addon-sdk/source/lib/sdk/system/child_process/subprocess.js#357
    var bytes = Uint8Array.from(data, (c) => c.charCodeAt(0));
    readIncoming(bytes);
});

Launder

does not help as Firefox continues to send utf8 encoded data to EXE. – Donadonadee

@AlexanderDyagilev This solution is not about the sending bit, it describes how to decode the incoming stream. See the question for the sending bit ("Add-on to Child Process"). But I would recommend to use WebExtensions or at least an embedded WebExtension to do this now and avoid using sdk/child-process – Launder

Answer 2

Maybe is this similar to this problem: Chrome native messaging doesn't accept messages of certain sizes (Windows)

Windows-only: Make sure that the program's I/O mode is set to O_BINARY. By default, the I/O mode is O_TEXT, which corrupts the message format as line breaks (\n = 0A) are replaced with Windows-style line endings (\r\n = 0D 0A). The I/O mode can be set using __setmode.

Closefisted

Thanks. I think that's some valuable information but not applicable to this particular problem. First, I'm not on Windows. Second, I'm getting the correct data back to the add-on, it's just in the wrong format (by default character-encoded to UTF-8 instead of raw bytes). – Rudyrudyard