Node.js encrypts large file using AES

Question

I try to use following code to encrypt a file of 1 GB. But Node.js abort with "FATAL ERROR: JS Allocation failed - process out of memory". How can I deal with it?

var fs = require('fs');
var crypto = require('crypto');
var key = "14189dc35ae35e75ff31d7502e245cd9bc7803838fbfd5c773cdcd79b8a28bbd";
var cipher = crypto.createCipher('aes-256-cbc', key);
var file_cipher = "";
var f = fs.ReadStream("test.txt");
f.on('data', function(d) {
    file_cipher = file_cipher + cipher.update(d, 'utf8', 'hex');
});
f.on('end', function() {  
    file_cipher = file_cipher + cipher.final('hex');
});   

Selfrevealing

Answer 1

You could write the encrypted file back to disk instead of buffering the entire thing in memory:

var fs = require('fs');
var crypto = require('crypto');

var key = '14189dc35ae35e75ff31d7502e245cd9bc7803838fbfd5c773cdcd79b8a28bbd';
var cipher = crypto.createCipher('aes-256-cbc', key);
var input = fs.createReadStream('test.txt');
var output = fs.createWriteStream('test.txt.enc');

input.pipe(cipher).pipe(output);

output.on('finish', function() {
  console.log('Encrypted file written to disk!');
});

Sharpwitted

Thanks! BTW, how can I know when the encryption is done? – Selfrevealing

You can listen for the finish event on output. – Sharpwitted

How to decrypt the same file? – Chondriosome

You can decrypt in the exact same way by using crypto.createDecipher instead of crypto.createCipher. – Crape

During decryption, how would I handle the wrong key? @RyanPlant – Ciracirca

This work great but when i minimize my window streaming is stoped – Aintab

createCipher is deprecated – Levant

For a similar program using up-to-date methods, see this response. – Jetport

Answer 2

crypto.createCipher() without initialization vector is deprecated since NodeJS v10.0.0 use crypto.createCipheriv() instead.

You can also pipe streams using stream.pipeline() instead of pipe method and then promisify it (so the code will easily fit into promise-like and async/await flow).

const {createReadStream, createWriteStream} = require('fs');
const {pipeline} = require('stream');
const {randomBytes, createCipheriv} = require('crypto');
const {promisify} = require('util');

const key = randomBytes(32); // ... replace with your key
const iv = randomBytes(16); // ... replace with your initialization vector

promisify(pipeline)(
        createReadStream('./text.txt'),
        createCipheriv('aes-256-cbc', key, iv),
        createWriteStream('./text.txt.enc')
)
.then(() => {/* ... */})
.catch(err => {/* ... */});

With NodeJS 15+ you could simplify it (skip promisify part)

const {createReadStream, createWriteStream} = require('fs');
const {pipeline} = require('stream/promises');
const {randomBytes, createCipheriv} = require('crypto');

const key = randomBytes(32); // ... replace with your key
const iv = randomBytes(16); // ... replace with your initialization vector

pipeline(
  createReadStream('./text.txt'),
  createCipheriv('aes-256-cbc', key, iv),
  createWriteStream('./text.txt.enc')
)
.then(() => {/* ... */})
.catch(err => {/* ... */});

Carbonyl

Me getting this error while decrypting the file Error: Unsupported state or unable to authenticate data – Unabridged

@PrathameshMore It's difficult to say where the issue is without seeing the code for file decryption. Please note, you should store your key and initialization vector somewhere (e.g. file), randomBytes was used only for demonstration purpose. – Carbonyl

Stream Promises API is available since Node Version 15 (docu) – Smilacaceous

Answer 3

I would simply use the Fileger to do it. It's a promise-based package and a clean alternative to the NodeJS filesystem.

const fileger = require("fileger")
const file = new fileger.File("./your-file-path.txt");

file.encrypt("your-password") // this will encrypt the file
   .then(() => {
      file.decrypt("your-password") // this will decrypt the file
   })

Stercoricolous