NodeJs: binary fs.createReadStream streamed as UTF-8

Question

My task at hand was to read a jpeg file in NodeJs and send it as http-response to answer a server request. Seemed to be trivial. However, my first solution failed. The browser did receive some binary gibrish, that was about 30% larger than the original file.

My code was (simplified; res is of type SeverResponse):

...
var fs = require('fs');
var stream = fs.createReadStream(pathToJPEG, {encoding: 'binary'});
res.setHeader('Content-Type', "image/jpeg");
stream.pipe(res);
...

As it turned out, what arrived at the browser was the UTF-8 encoded version of my source data. I also was able to exlude the response object to be the culprit. When I gave it an alternative stream (from Buffer, not file) it worked just fine.

Turns out the solution to my problem was to drop the option {encoding: 'binary'}. With that my browser received the right picture:

...
var fs = require('fs');
var stream = fs.createReadStream(pathToJPEG);
res.setHeader('Content-Type', "image/jpeg");
stream.pipe(res);
...

My question is: Why?

It seems intuitive that the first non-working version should be the correct one since it explicitly declares how to read the file.

Helluva

Answer 1

This is because the binary encoding is not really binary. createReadStream uses the same encoding parameters that accepted by Buffer. From the Node Buffer Docs:

'binary' - A way of encoding the buffer into a one-byte (i.e. latin-1) encoded string. The string 'latin-1' is not supported. Instead simply pass 'binary' to use 'latin-1' encoding.

Just set encoding to null to get the raw stream or buffer, or don't specify anything at all, as you did in your second example.

Subternatural

This didn't work for me. I'm still getting the file re-encoded and corrupt. – Swisher

Answer 2

Ixe is correct, changing encoding to null worked, but only after upgrading to a newer node/express package. Here is my code that correctly uploads a tar file:

    fs.exists(filePath, function(exists){
    if (exists) {
        var stat = fs.statSync(filePath);
        console.log('sending file, size %d', stat.size);

        res.writeHead(200, {
            "Content-Type": "application/x-tar",
            "Content-Disposition": "attachment; filename=" + filePath,
            "Content-Length": stat.size,
            "Content-Transfer-Encoding": "binary"
        });
        fs.createReadStream(filePath, { encoding: null }).pipe(res);    //must set encoding to null, as binary is not treated correctly

    } else {
        console.log('file not exist.');
        res.writeHead(400, {"Content-Type": "text/plain"});
        res.end("ERROR File does not exist");
    }
});

Retardment