How to speed up unzipping time in Java / Android?

Question

Unzipping files on android seems to be dreadfully slow. At first I thought this was just the emulator but it appears to be the same on the phone. I've tried different compression levels, and eventually dropped down to storage mode but it still takes ages.

Anyway, there must be a reason! Does anyone else have this problem? My unzip method looks like this:

public void unzip()
{
try{
        FileInputStream fin = new FileInputStream(zipFile);
        ZipInputStream zin = new ZipInputStream(fin);
        File rootfolder = new File(directory);
        rootfolder.mkdirs();
        ZipEntry ze = null;
        while ((ze = zin.getNextEntry())!=null){

            if(ze.isDirectory()){
                dirChecker(ze.getName());
            }
            else{
                FileOutputStream fout = new FileOutputStream(directory+ze.getName());

            for(int c = zin.read();c!=-1;c=zin.read()){
                fout.write(c);
            }
                //Debug.out("Closing streams");
                zin.closeEntry();
                fout.close();

        }
    }
    zin.close();
}
catch(Exception e){
            //Debug.out("Error trying to unzip file " + zipFile);

}
    }

Alphonsoalphonsus

consider where you are unzipping. Context.getCacheDir() is going to be internal storage, where Environment.getDataDirectory() may be internal (or not) and Environment.getExternalDirectory() will be on the SD card. internal memory is almost certainly going to be faster. – Grouchy

Answer 1

I don't know if unzipping on Android is slow, but copying byte for byte in a loop is surely slowing it down even more. Try using BufferedInputStream and BufferedOutputStream - it might be a bit more complicated, but in my experience it is worth it in the end.

BufferedInputStream in = new BufferedInputStream(zin);
BufferedOutputStream out = new BufferedOutputStream(fout);

And then you can write with something like that:

byte b[] = new byte[1024];
int n;
while ((n = in.read(b,0,1024)) >= 0) {
  out.write(b,0,n);
}

Phosgene

Why not put the BufferedInputStream between the FileInputStream and the ZipInputStream? I.e., FileInputStream fin = new FileInputStream(zipFile); BufferedInputStream bin = new BufferedInputStream(fin); ZipInputStream zin = new ZipInputStream(bin); – Vend

awesome - I'll give it a shot – Alphonsoalphonsus

If the reads and writes are performed on arrays like the sample loop, then the benefit of wrapping the streams with buffered streams is diminished. The buffering streams provide performance improvements when the consumer/producer is incapable of working with byte arrays, and performs operations byte-by-byte. – Geez

You can create BufferedInputStream between FileInputStream and ZipInputStream and it works as well with better performace – Tamarah

Brilliant. Went from about 5 mins per item unzipped 5 seconds. Lifesaver... thanks Robert – Fronia

Mine went from 40 minutes to 3 secs. o_O – Anklebone

Dilum Ranatunga was right, if already copied files by arrays instead of byte-by-byte, buffered streams resulted in same speed, I've just tested. – Shun

Unzipping 8101 png files went from 19 minutes < 1 min. Thanks! – Giguere

How do you know the length of the byte array? – Algarroba

I tested and found that BufferedOutputStream make the process faster. The BufferedInputStream seem not change the speed, and it may make the process a litle bit slower. – Alodi

Answer 2

Thanks for the solution Robert. I modified my unzip method and now it takes only a few seconds instead of 2 minutes. Maybe someone's interested in my solution. So here you go:

public void unzip() {

    try {
        FileInputStream inputStream = new FileInputStream(filePath);
        ZipInputStream zipStream = new ZipInputStream(inputStream);
        ZipEntry zEntry = null;
        while ((zEntry = zipStream.getNextEntry()) != null) {
            Log.d("Unzip", "Unzipping " + zEntry.getName() + " at "
                    + destination);

            if (zEntry.isDirectory()) {
                hanldeDirectory(zEntry.getName());
            } else {
                FileOutputStream fout = new FileOutputStream(
                        this.destination + "/" + zEntry.getName());
                BufferedOutputStream bufout = new BufferedOutputStream(fout);
                byte[] buffer = new byte[1024];
                int read = 0;
                while ((read = zipStream.read(buffer)) != -1) {
                    bufout.write(buffer, 0, read);
                }

                zipStream.closeEntry();
                bufout.close();
                fout.close();
            }
        }
        zipStream.close();
        Log.d("Unzip", "Unzipping complete. path :  " + destination);
    } catch (Exception e) {
        Log.d("Unzip", "Unzipping failed");
        e.printStackTrace();
    }

}

public void hanldeDirectory(String dir) {
        File f = new File(this.destination + dir);
        if (!f.isDirectory()) {
            f.mkdirs();
        }
}

Heed

Hi, what's handleDirectory? – Offcenter

Oh sorry i overlooked it. It's an simple method to create a new folder in the case that the file which is unzipped, is an folder. I added it above – Heed

How do you know the length of the byte array? (1024) – Algarroba

Sorry for the late response. The size of the buffer was chosen randomly. The buffer is filled by the ZipInputStream and emptied by the FileOutputStream until all bytes are transferred. So the buffer's size only influence the number of times, the buffer is filled and emptied. There are probably pros and cons of small or large buffers. But to be honest, i didn't spend that many thoughts into this. – Heed

This is very fast :) – Braille

Brilliant. Thanks a lot. Works like a charm and speed is like a rocket.! – Priestcraft

It is really fast, @thoxxer Thank you very much can you please explain, what exactly you did? – Enviable

beautifully implemented <3 – Lepley

Answer 3

Using above ideas and ideas from some other sources I have created this class

Create this new class

import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import android.util.Log;

public class DecompressFast {
 private String _zipFile; 
  private String _location; 

  public DecompressFast(String zipFile, String location) { 
    _zipFile = zipFile; 
    _location = location; 
    _dirChecker(""); 
  } 

  public void unzip() { 
    try  { 
      FileInputStream fin = new FileInputStream(_zipFile); 
      ZipInputStream zin = new ZipInputStream(fin); 
      ZipEntry ze = null; 
      while ((ze = zin.getNextEntry()) != null) { 
        Log.v("Decompress", "Unzipping " + ze.getName()); 
        if(ze.isDirectory()) { 
          _dirChecker(ze.getName()); 
        } else { 
          FileOutputStream fout = new FileOutputStream(_location +  ze.getName()); 
          BufferedOutputStream bufout = new BufferedOutputStream(fout);
          byte[] buffer = new byte[1024];
          int read = 0;
          while ((read = zin.read(buffer)) != -1) {
              bufout.write(buffer, 0, read);
          }
          bufout.close();
          zin.closeEntry(); 
          fout.close(); 
        }    
      } 
      zin.close(); 
      Log.d("Unzip", "Unzipping complete. path :  " +_location );
    } catch(Exception e) { 
      Log.e("Decompress", "unzip", e); 
      Log.d("Unzip", "Unzipping failed");
    } 
  } 

  private void _dirChecker(String dir) { 
    File f = new File(_location + dir); 

    if(!f.isDirectory()) { 
      f.mkdirs(); 
    } 
  } 
}

USAGE

just pass your file location of zip file and your destination Location to this class
example

  String zipFile = Environment.getExternalStorageDirectory() + "/the_raven.zip"; //your zip file location
  String unzipLocation = Environment.getExternalStorageDirectory() + "/unzippedtestNew/"; // unzip location
  DecompressFast df= new DecompressFast(zipFile, unzipLocation);
    df.unzip();

Dont Forget to add following permissions in manifest(also Run time permission if version higher than marshmellow)

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

 <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

hope this helps

Compelling

Answer 4

The URL that helped me learn how to zip and unzip can be found here.

I used that URL in conjuction with user3203118's answer above for unzipping. This is for future references for people who run in to this issue and need help solving it.

Below is the ZipManager code I am using:

public class ZipManager {

    private static final int BUFFER = 80000;

    public void zip(String[] _files, String zipFileName) {
        try {
            BufferedInputStream origin = null;
            FileOutputStream dest = new FileOutputStream(zipFileName);
            ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(
                dest));
            byte data[] = new byte[BUFFER];

            for (int i = 0; i < _files.length; i++) {
                Log.v("Compress", "Adding: " + _files[i]);
                FileInputStream fi = new FileInputStream(_files[i]);
                origin = new BufferedInputStream(fi, BUFFER);

                ZipEntry entry = new ZipEntry(_files[i].substring(_files[i]
                    .lastIndexOf("/") + 1));
                out.putNextEntry(entry);
                int count;

                while ((count = origin.read(data, 0, BUFFER)) != -1) {
                    out.write(data, 0, count);
                }
                origin.close();
            }
            out.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public void unzip(String _zipFile, String _targetLocation) {


        // create target location folder if not exist
        dirChecker(_targetLocation);

        try {
            FileInputStream fin = new FileInputStream(_zipFile);
            ZipInputStream zin = new ZipInputStream(fin);
            ZipEntry ze = null;
            while ((ze = zin.getNextEntry()) != null) {

                // create dir if required while unzipping
                if (ze.isDirectory()) {
                    dirChecker(ze.getName());
                } else {
                    FileOutputStream fout = new FileOutputStream(
                    _targetLocation + "/" + ze.getName());
                    BufferedOutputStream bufout = new BufferedOutputStream(fout);
                    byte[] buffer = new byte[1024];
                    int read = 0;
                    while ((read = zin.read(buffer)) != -1) {
                        bufout.write(buffer, 0, read);
                    }

                    zin.closeEntry();
                    bufout.close();
                    fout.close();
                }
            }
            zin.close();
        } catch (Exception e) {
            System.out.println(e);
        }
    }

    private void dirChecker(String dir) {
        File f = new File(dir);
        if (!f.isDirectory()) {
            f.mkdirs();
        }
    }
}

Enlighten

I think users also having this issue would be very happy if you could include the code in your answer, if possible ;-) – Eventually

I have added my zip manager that i am currently using and works like a champ – Enlighten

I got an error from your code because in a peculiar situation the file entry was coming before the folder entry. Therefore, file write was failing. I have introduced a new function before declaring fout variable: File f = new File(this.destinationFolder + filePath); File parentDir = f.getParentFile(); if (parentDir!=null) { if (parentDir.isDirectory() == false) parentDir.mkdirs(); } – Tomsk

Answer 5

Just call this method and it will give you much better performance..

    public boolean unzip(Context context) {
    try {
        FileInputStream fin = new FileInputStream(_zipFile);
        ZipInputStream zin = new ZipInputStream(fin);
        BufferedInputStream in = new BufferedInputStream(zin);
        ZipEntry ze = null;
        while ((ze = zin.getNextEntry()) != null) {
            Log.v("Decompress", "Unzipping " + ze.getName());

            if (ze.isDirectory()) {
                _dirChecker(ze.getName());
            } else {
                FileOutputStream fout = new FileOutputStream(_location
                        + ze.getName());
                    BufferedOutputStream out = new BufferedOutputStream(fout);
                    byte b[] = new byte[1024];
                for (int c = in.read(b,0,1024); c != -1; c = in.read()) {
                    out.write(b,0,c);
                }
                zin.closeEntry();
                fout.close();
            }
        }
        zin.close();
        return true;
    } catch (Exception e) {
        Log.e("Decompress", "unzip", e);
        return false;
    }
}

    private void _dirChecker(String dir) {
    File f = new File(_location + dir);
    if (!f.isDirectory()) {
        f.mkdirs();
    }
}

Ricotta

Answer 6

In case of using BufferedOutputStream be sure to flush it. If you do not do it, size smaller than buffer will not be unzipped properly

if (ze.isDirectory()) {
                _dirChecker(ze.getName());
            } else {
                FileOutputStream fout = new FileOutputStream(_location
                        + ze.getName());
                    BufferedOutputStream out = new BufferedOutputStream(fout);
                    byte buffer[] = new byte[1024];
                for (int c = in.read(buffer,0,1024); c != -1; c = in.read()) {
                    out.write(buffer,0,c);
                }
                out.flush();//flush it......
                zin.closeEntry();
                fout.close();
            }

Imbed

Thanks a lot. I upvoted your answer. My audio files were cut short after unzipping. Forgot to call out.flush() like you said – Darryldarryn