Need to ZIP an entire directory using Node.js

Question

I need to zip an entire directory using Node.js. I'm currently using node-zip and each time the process runs it generates an invalid ZIP file (as you can see from this Github issue).

Is there another, better, Node.js option that will allow me to ZIP up a directory?

EDIT: I ended up using archiver

writeZip = function(dir,name) {
var zip = new JSZip(),
    code = zip.folder(dir),
    output = zip.generate(),
    filename = ['jsd-',name,'.zip'].join('');

fs.writeFileSync(baseDir + filename, output);
console.log('creating ' + filename);
};

sample value for parameters:

dir = /tmp/jsd-<randomstring>/
name = <randomstring>

UPDATE: For those asking about the implementation I used, here's a link to my downloader:

Dineen

Someone on Twitter suggested the child_process API, and simply call the system ZIP: nodejs.org/api/child_process.html – Dineen

I've tried the child_process approach. It's got two caveats. 1) unix zip command includes all parent folder hierarchy of the current working directory in the zipped file. This might be ok for you, it wasn't for me. Also changing the current working directory in child_process somehow doesn't effect the results. 2) To overcome this problem, you have to use pushd to jump into the folder you will zip and zip -r , but since pushd is built into bash and not /bin/sh you need to use /bin/bash also. In my specific case this wasn't possible. Just a heads up. – Warrenwarrener

@Warrenwarrener node's child_process.exec api lets you specify the cwd from where you want to run the command. Changing the CWD does fix the issue of the parent folder hierarchy. It also fixes the issue of not needing pushd. I fully recommend child_process. – Gaspard

https://mcmap.net/q/142619/-need-to-zip-an-entire-directory-using-node-js native nodejs solution using child_process api. 2 lines of code. No third party libs. – Gaspard

@GovindRai Many thanks! – Warrenwarrener

Answer 1

I ended up using archiver lib. Works great.

Example

var file_system = require('fs');
var archiver = require('archiver');

var output = file_system.createWriteStream('target.zip');
var archive = archiver('zip');

output.on('close', function () {
    console.log(archive.pointer() + ' total bytes');
    console.log('archiver has been finalized and the output file descriptor has closed.');
});

archive.on('error', function(err){
    throw err;
});

archive.pipe(output);

// append files from a sub-directory, putting its contents at the root of archive
archive.directory(source_dir, false);

// append files from a sub-directory and naming it `new-subdir` within the archive
archive.directory('subdir/', 'new-subdir');

archive.finalize();

Dineen

There don't seem to be any examples of how to do this, do you mind sharing what you did? – Alit

archiver, unfortunately, doesn't support Unicode characters in filenames as of now. Reported to github.com/ctalkington/node-archiver/issues/90. – Custumal

How do I include all files and directories, recursively (also the hidden files/directories)? – Dorolisa

Archiver makes this even simpler now. Rather than using the bulk() method, you can now use directory(): npmjs.com/package/archiver#directory-dirpath-destpath-data – Honeyman

The .directory() method does work, but when the archive is unzipped the file tree to the directory exists. Is it possible to avoid this? – Commodity

Shall I call output.close() somewhere? – Tirzah

.bulk is deprecated – Hoedown

Thanks for this, I had troubles both with adm-zip and node-zip (corrupted binary files), this seem to work fast and clean (my application builds XXX MB zips composed by a lot of small videos) – Concatenate

Bulk has been deprecated. github.com/archiverjs/node-archiver/compare/1.3.0...2.0.0 – Cholula

How to add source directory to bulk zip? – Break

I still have an issue, where when I open the zip I created, all the files are there, except the image files, and if the images are there, most of the pixels are the same solid color – Phyle

Answer 2

I'm not going to show something new, just wanted to summarise the solutions above for those who like Promises as much as I do 😉.

const archiver = require('archiver');

/**
 * @param {String} sourceDir: /some/folder/to/compress
 * @param {String} outPath: /path/to/created.zip
 * @returns {Promise}
 */
function zipDirectory(sourceDir, outPath) {
  const archive = archiver('zip', { zlib: { level: 9 }});
  const stream = fs.createWriteStream(outPath);

  return new Promise((resolve, reject) => {
    archive
      .directory(sourceDir, false)
      .on('error', err => reject(err))
      .pipe(stream)
    ;

    stream.on('close', () => resolve());
    archive.finalize();
  });
}

Hope it will help someone 🤞

Hertfordshire

what exactly is "out" here? i assume source is the path of directory – Arrangement

@Tarun full zip's path like: /User/mypc/mydir/test.zip – Hertfordshire

@ekaj_03 please make sure that you have enough rights for specified directory – Hertfordshire

I found the easiest way to add files and zip in Nodejs at this link : jsonworld.wordpress.com/2019/09/07/… – Shipper

Can someone please share code that can unzip the above zip? I have tried many things and always end up with "there are no headers at the end of the file / headers are incorrect" error. Thanks. – Catacomb

@AmeyParundekar could you please share your code so we can investigate the issue? – Hertfordshire

Make sure to catch write error from stream too stream.on('error', ex => reject(ex)); – Dysthymia

Thanks!!! This helped me!!! Saved time!!! Here's an adaptation using import from: stackoverflow.com/a/70958944 – Southwestward

If you are using Typescript, you might need an additional dependencies: @types/archiver – Voidable

I needed to archive files in a node prebuild script, but kept ending up with archives that were 0 bytes long. Wrapping everything in a Promise worked like a charm. Thanks! – Kelle

Answer 3

Use Node's native child_process api to accomplish this.

No need for third party libs. Two lines of code.

const child_process = require("child_process");
child_process.execSync(`zip -r <DESIRED_NAME_OF_ZIP_FILE_HERE> *`, {
  cwd: <PATH_TO_FOLDER_YOU_WANT_ZIPPED_HERE>
});

The example above showcases the synchronous API. You can also use child_process.exec(path, options, callback) if you want async behavior. There are a lot more options you can specify other than cwd to further fine-tune your request.

---

If you don't have the ZIP utility:

This question is specifically asks about the zip utility for archiving/compression purposes. Therefore, this example assumes you have the zip utility installed on your system. For completeness sakes, some operating systems may not have utility installed by default. In that case you have at least three options:

1. Work with the archiving/compression utility that is native to your platform

   Replace the shell command in the above Node.js code with code from your system. For example, linux distros usually come with tar/gzip utilities:

   tar -cfz <DESIRED_NAME_OF_ZIP_FILE_HERE> <PATH_TO_FOLDER_YOU_WANT_ZIPPED_HERE>.

   This is a nice option as you don't need to install anything new onto your operating system or manage another dependency (kind of the whole point for this answer).

2. Obtain the zip binary for your OS/distribution.

   For example on Ubuntu: apt install zip.

   The ZIP utility is tried and tested for decades, it's fairly ubiquitous and it's a safe choice. Do a quick google search or go to the creator, Info-ZIP's, website for downloadable binaries.

3. Use a third party library/module (of which there are plenty on NPM).

   I don't prefer this option. However, if you don't really care to understand the native methods and introducing a new dependency is a non-issue, this is also a valid option.

Gaspard

Unfortunately only works on systems that have zip. – Soften

Went for this solution just for the sake of avoiding dozens of external libraries on my project – Ark

it makes sense, but if i'm not incorrect this is screwing over windows users again. Please think of the windows users! – Vendace

@MathijsSegers haha! that's why i included a link to the binary so windows users can get it too! :) – Gaspard

Is there a way to get this to work for a directory within a project rather than a computer directory? – Misdeed

What do you mean by "within a project"? – Gaspard

@GovindRai What I meant was how to access a directory within my app and use fs.readdir on let's say 'project_root/server/modules/auth' but I was actually able to make it work using path.resolve(__dirname) + '/rest_of_path'. – Misdeed

Assuming that a given library is installed in the underlying platform defeats the purpose of the package.json dependency manifest. Even more importantly, if Zip is only available on MacOS and you deploy to a linux environment you are doomed :D – Bedwell

Just a reminder for folks who find this solution inadequate because it relies of the zip binary: this question is specifically asking how to zip files. It's not asking for a cross platform archiving solution. The last paragraph is dedicated for this purpose of obtaining the zip utility across platforms. – Gaspard

All the Node zip libs I checked out were like toys compared to the real thing. +1 for the reminder, thanks! – Closing

Is there a way to stream the input and stream the resulting zip archive out for an API route? I am using streaming with a Node library to stream results but curious if it is possible using exec and not using the tool. – Schedule

Getting error with zip command, although when i use zip command with terminal it works fine!! /bin/sh: 1: zip: not found – Electric

Answer 4

This is another library which zips the folder in one line : zip-local

var zipper = require('zip-local');

zipper.sync.zip("./hello/world/").compress().save("pack.zip");

Arrangement

Worked like a charm, unlike dozen of others available on internet or mentioned above, which always generated 'zero bytes' file for me – Catalonia

Answer 5

Archive.bulk is now deprecated, the new method to be used for this is glob:

var fileName =   'zipOutput.zip'
var fileOutput = fs.createWriteStream(fileName);

fileOutput.on('close', function () {
    console.log(archive.pointer() + ' total bytes');
    console.log('archiver has been finalized and the output file descriptor has closed.');
});

archive.pipe(fileOutput);
archive.glob("../dist/**/*"); //some glob pattern here
archive.glob("../dist/.htaccess"); //another glob pattern
// add as many as you like
archive.on('error', function(err){
    throw err;
});
archive.finalize();

Olein

Was wondering about this, they said bulk was deprecated but didn't suggest which function to use instead. – Paternoster

How do you specify the "source" directory? – Arrangement

Try once the below approach: jsonworld.wordpress.com/2019/09/07/… – Shipper

2020: archive.directory() is much simpler! – Boxfish

Answer 6

To include all files and directories:

archive.bulk([
  {
    expand: true,
    cwd: "temp/freewheel-bvi-120",
    src: ["**/*"],
    dot: true
  }
]);

It uses node-glob(https://github.com/isaacs/node-glob) underneath, so any matching expression compatible with that will work.

Nerin

.bulk is deprecated – Bewley

Answer 7

To pipe the result to the response object (scenarios where there is a need to download the zip rather than store locally)

 archive.pipe(res);

Sam's hints for accessing the content of the directory worked for me.

src: ["**/*"]

Embassy

Answer 8

Since archiver is not compatible with the new version of webpack for a long time, I recommend using zip-lib.

var zl = require("zip-lib");

zl.archiveFolder("path/to/folder", "path/to/target.zip").then(function () {
    console.log("done");
}, function (err) {
    console.log(err);
});

Impressionist

You have an example to integrate zip-lib into my webpack project? 😊😊😊 – Hydrastine

Answer 9

As today, I'm using AdmZip and works great:

const AdmZip = require('adm-zip');
export async function archiveFile() {
  try {
    const zip = new AdmZip();
    const outputDir = "/output_file_dir.zip";
    zip.addLocalFolder("./yourFolder")
    zip.writeZip(outputDir);
  } catch (e) {
    console.log(`Something went wrong ${e}`);
  }
}

Magnificat

Answer 10

Adm-zip has problems just compressing an existing archive https://github.com/cthackers/adm-zip/issues/64 as well as corruption with compressing binary files.

I've also ran into compression corruption issues with node-zip https://github.com/daraosn/node-zip/issues/4

node-archiver is the only one that seems to work well to compress but it doesn't have any uncompress functionality.

Leucopenia

About which node-archiver are you talking about? : github.com/archiverjs/node-archiver ; github.com/richardbolt/node-archiver – Cates

@firian He did not say Archiver, he said Adm-zip. – Forsterite

@FrancisPelland Umm, in the last sentence he wrote "node-archiver is the only one that seems to work" - that's what I'm refering to. – Cates

i think he meatn npmjs.com/package/archiver – Boxfish

Answer 11

I have found this small library that encapsulates what you need.

npm install zip-a-folder

const zipAFolder = require('zip-a-folder');
await zipAFolder.zip('/path/to/the/folder', '/path/to/archive.zip');

https://www.npmjs.com/package/zip-a-folder

Housebound

Is it possible to add parameters to make zip folder ? like compressed level and size if so how to do that? – Cellulose

Downvoting because zip-a-folder is not a vaild identifier. – Tripalmitin

Answer 12

import ... from answer based on https://stackoverflow.com/a/51518100

To zip single directory

import archiver from 'archiver';
import fs from 'fs';

export default zipDirectory;

/**
 * From: https://stackoverflow.com/a/51518100
 * @param {String} sourceDir: /some/folder/to/compress
 * @param {String} outPath: /path/to/created.zip
 * @returns {Promise}
 */
function zipDirectory(sourceDir, outPath) {
  const archive = archiver('zip', { zlib: { level: 9 }});
  const stream = fs.createWriteStream(outPath);

  return new Promise((resolve, reject) => {
    archive
      .directory(sourceDir, false)
      .on('error', err => reject(err))
      .pipe(stream)
    ;

    stream.on('close', () => resolve());
    archive.finalize();
  });
}

To zip multiple directories:

import archiver from 'archiver';
import fs from 'fs';

export default zipDirectories;

/**
 * Adapted from: https://stackoverflow.com/a/51518100
 * @param {String} sourceDir: /some/folder/to/compress
 * @param {String} outPath: /path/to/created.zip
 * @returns {Promise}
 */
function zipDirectories(sourceDirs, outPath) {
  const archive = archiver('zip', { zlib: { level: 9 }});
  const stream = fs.createWriteStream(outPath);

  return new Promise((resolve, reject) => {
    var result = archive;
    sourceDirs.forEach(sourceDir => {
      result = result.directory(sourceDir, false);
    });
    result
      .on('error', err => reject(err))
      .pipe(stream)
    ;

    stream.on('close', () => resolve());
    archive.finalize();
  });
}

Southwestward

Answer 13

You can try in a simple way:

Install zip-dir :

npm install zip-dir

and use it

var zipdir = require('zip-dir');

let foldername =  src_path.split('/').pop() 
    zipdir(<<src_path>>, { saveTo: 'demo.zip' }, function (err, buffer) {

    });

Devilish

is it possible to add parameters to make zip folder ? like compressed level and size if so how to do that? – Cellulose

Answer 14

I ended up wrapping archiver to emulate JSZip, as refactoring through my project woult take too much effort. I understand Archiver might not be the best choice, but here you go.

// USAGE:
const zip=JSZipStream.to(myFileLocation)
    .onDone(()=>{})
    .onError(()=>{});

zip.file('something.txt','My content');
zip.folder('myfolder').file('something-inFolder.txt','My content');
zip.finalize();

// NodeJS file content:
    var fs = require('fs');
    var path = require('path');
    var archiver = require('archiver');

  function zipper(archive, settings) {
    return {
        output: null,
        streamToFile(dir) {
            const output = fs.createWriteStream(dir);
            this.output = output;
            archive.pipe(output);

            return this;
        },
        file(location, content) {
            if (settings.location) {
                location = path.join(settings.location, location);
            }
            archive.append(content, { name: location });
            return this;
        },
        folder(location) {
            if (settings.location) {
                location = path.join(settings.location, location);
            }
            return zipper(archive, { location: location });
        },
        finalize() {
            archive.finalize();
            return this;
        },
        onDone(method) {
            this.output.on('close', method);
            return this;
        },
        onError(method) {
            this.output.on('error', method);
            return this;
        }
    };
}

exports.JSzipStream = {
    to(destination) {
        console.log('stream to',destination)
        const archive = archiver('zip', {
            zlib: { level: 9 } // Sets the compression level.
        });
        return zipper(archive, {}).streamToFile(destination);
    }
};

Leekgreen

Answer 15

const express = require("express");

const { createGzip } = require("node:zlib");
const { pipeline } = require("node:stream");
const { createReadStream, createWriteStream } = require("node:fs");

const app = express();
app.use(express.json());
app.get("/", async (req, res) => {
  const gzip = createGzip();
  const source = createReadStream("hotellists.json");
  const destination = createWriteStream("hotellists.json.gz");

  pipeline(source, gzip, destination, (err) => {
    if (err) {
      console.error("An error occurred:", err);
      process.exitCode = 1;
    }
    
  });
});
app.listen(2020, () => console.log("server in on"));

Aeneous

Your answer could be improved by adding more information on what the code does and how it helps the OP. – Otolith