GZIP compression to a byte array

Question

I am trying to write a class that can compress data. The below code fails (no exception is thrown, but the target .gz file is empty.)
Besides: I don't want to generate the .gz file directly like it is done in all examples. I only want to get the compressed data, so that I can e.g. encrypt it before writting the data to a file.

If I write directly to a file everything works fine:

import java.io.*;
import java.util.zip.*;
import java.nio.charset.*;

public class Zipper
{
  public static void main(String[] args)
  {    
    byte[] dataToCompress = "This is the test data."
      .getBytes(StandardCharsets.ISO_8859_1);

    GZIPOutputStream zipStream = null;
    FileOutputStream fileStream = null;
    try
    {
      fileStream = new FileOutputStream("C:/Users/UserName/Desktop/zip_file.gz");
      zipStream = new GZIPOutputStream(fileStream);
      zipStream.write(dataToCompress);

      fileStream.write(compressedData);
    }
    catch(Exception e)
    {
      e.printStackTrace();
    }
    finally
    {
      try{ zipStream.close(); }
        catch(Exception e){ }
      try{ fileStream.close(); }
        catch(Exception e){ }
    }
  }
}

But, if I want to 'bypass' it to the byte array stream it does not produce a single byte - compressedData is always empty.

import java.io.*;
import java.util.zip.*;
import java.nio.charset.*;

public class Zipper
{
  public static void main(String[] args)
  {    
    byte[] dataToCompress = "This is the test data."
      .getBytes(StandardCharsets.ISO_8859_1);
    byte[] compressedData = null;

    GZIPOutputStream zipStream = null;
    ByteArrayOutputStream byteStream = null;
    FileOutputStream fileStream = null;
    try
    {
      byteStream = new ByteArrayOutputStream(dataToCompress.length);
      zipStream = new GZIPOutputStream(byteStream);
      zipStream.write(dataToCompress);

      compressedData = byteStream.toByteArray();

      fileStream = new FileOutputStream("C:/Users/UserName/Desktop/zip_file.gz");
      fileStream.write(compressedData);
    }
    catch(Exception e)
    {
      e.printStackTrace();
    }
    finally
    {
      try{ zipStream.close(); }
        catch(Exception e){ }
      try{ byteStream.close(); }
        catch(Exception e){ }
      try{ fileStream.close(); }
        catch(Exception e){ }
    }
  }
}

Lowney

"The below code fails" is not a valid Java error message. – Reber

Please also include the stack trace from the error. – Ossuary

It just generates a file that is not a ZIP archive. There is no error message. – Lowney

duh. there is no error message because you do'nt log anything in your exceptions. – Occurrence

It's generally true in programming that if you can't clearly describe what it is you want to do, you can't do it. – Chanell

It does not produce any exception that could be logged. The 'assembling' of the ZIP file must be done wrong. – Lowney

So, you're trying to generate a ZIP archive, but also not generate a ZIP archive – Leialeibman

Your code produces a perfectly fine ZIP archive for me btw. – Leialeibman

Also, to get just a stream of compressed data, use DeflaterOutputStream or GZipOutputStream – Leialeibman

Answer 1

The problem is that you are not closing the GZIPOutputStream. Until you close it the output will be incomplete.

You just need to close it before reading the byte array. You need to reorder the finally blocks to achieve this.

import java.io.*;
import java.util.zip.*;
import java.nio.charset.*;

public class Zipper
{
  public static void main(String[] args)
  {    
    byte[] dataToCompress = "This is the test data."
      .getBytes(StandardCharsets.ISO_8859_1);

    try
    {
      ByteArrayOutputStream byteStream =
        new ByteArrayOutputStream(dataToCompress.length);
      try
      {
        GZIPOutputStream zipStream =
          new GZIPOutputStream(byteStream);
        try
        {
          zipStream.write(dataToCompress);
        }
        finally
        {
          zipStream.close();
        }
      }
      finally
      {
        byteStream.close();
      }

      byte[] compressedData = byteStream.toByteArray();

      FileOutputStream fileStream =
        new FileOutputStream("C:/Users/UserName/Desktop/zip_file.gz");
      try
      {
        fileStream.write(compressedData);
      }
      finally
      {
        try{ fileStream.close(); }
          catch(Exception e){ /* We should probably delete the file now? */ }
      }
    }
    catch(Exception e)
    {
      e.printStackTrace();
    }
  }
}

I do not recommend inititalizing the stream variables to null, because it means your finally block can also throw a NullPointerException.

Also note that you can declare main to throw IOException (then you would not need the outermost try statement.)

There is little point in swallowing exceptions from zipStream.close();, because if it throws an exception you will not have a valid .gz file (so you should not proceed to write it.)

Also I would not swallow exceptions from byteStream.close(); but for a different reason - they should never be thrown (i.e. there is a bug in your JRE and you would want to know about that.)

Hypozeuxis

you need to close GZIPOutputStream or else will get partial data – Armillary

Answer 2

I've improved JITHINRAJ's code - used try-with-resources:

private static byte[] gzipCompress(byte[] uncompressedData) {
        byte[] result = new byte[]{};
        try (ByteArrayOutputStream bos = new ByteArrayOutputStream(uncompressedData.length);
             GZIPOutputStream gzipOS = new GZIPOutputStream(bos)) {
            gzipOS.write(uncompressedData);
            // You need to close it before using bos
            gzipOS.close();
            result = bos.toByteArray();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return result;
    }

private static byte[] gzipUncompress(byte[] compressedData) {
        byte[] result = new byte[]{};
        try (ByteArrayInputStream bis = new ByteArrayInputStream(compressedData);
             ByteArrayOutputStream bos = new ByteArrayOutputStream();
             GZIPInputStream gzipIS = new GZIPInputStream(bis)) {
            byte[] buffer = new byte[1024];
            int len;
            while ((len = gzipIS.read(buffer)) != -1) {
                bos.write(buffer, 0, len);
            }
            result = bos.toByteArray();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return result;
    }

Amends

gzipOS.close(); is not necessary here since GZIPOutputStream is AutoCloseable. – Bitter

gzipOS.close(); is still required! Without it, Gzip compression will be invalid. Please try and fix – Kick

@YanislavKornev you're absolutely right! If I don't close gzipOS I'll get incorrect array – Amends

How do I know how long the compressed array is? toByteArray compresses my array, but does not remove the trailing zeros. – Macropterous

@Macropterous you can use result.length. You can get more information about gzip alg-m here - en.wikipedia.org/wiki/Gzip – Amends

@slartidan what happens if an exception arises? – Amends

True... Forget what I said :) – Dot

You don't need to use gzipOS.close(); You only need gzipOS.finish();, and let java close it for you. – Gormandize

Answer 3

If you are still looking an answer you can use the below code to get the compressed byte[] using deflater and decompress it using inflater.

public static void main(String[] args) {
        //Some string for testing
        String sr = new String("fsdfesfsfdddddddsfdsfssdfdsfdsfdsfdsfdsdfggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggghghghghggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggfsdfesfsfdddddddsfdsfssdfdsfdsfdsfdsfdsdfggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggghghghghggggggggggggggggggggggggggggggggggggggggg");
        byte[] data = sr.getBytes();
        System.out.println("src size "+data.length);
        try {
            compress(data);
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
    public static byte[] compress(byte[] data) throws IOException { 
        Deflater deflater = new Deflater(); 
        deflater.setInput(data); 
        ByteArrayOutputStream outputStream = new ByteArrayOutputStream(data.length);  

        deflater.finish(); 
        byte[] buffer = new byte[1024];  
        while (!deflater.finished()) { 
        int count = deflater.deflate(buffer);  
        outputStream.write(buffer, 0, count);  
        } 
        outputStream.close(); 
        byte[] output = outputStream.toByteArray(); 

        System.out.println("Original: " + data.length  ); 
        System.out.println("Compressed: " + output.length ); 
        return output; 
        }   

Each

This is an invalid answer, as Inflater and Deflater use Zlib instead of GZip. – Connieconniption

Answer 4

To compress

private static byte[] compress(byte[] uncompressedData) {
        ByteArrayOutputStream bos = null;
        GZIPOutputStream gzipOS = null;
        try {
            bos = new ByteArrayOutputStream(uncompressedData.length);
            gzipOS = new GZIPOutputStream(bos);
            gzipOS.write(uncompressedData);
            gzipOS.close();
            return bos.toByteArray();

        } catch (IOException e) {
            e.printStackTrace();
        }
        finally {
            try {
                assert gzipOS != null;
                gzipOS.close();
                bos.close();
            }
            catch (Exception ignored) {
            }
        }
        return new byte[]{};
    }

To uncompress

private byte[] uncompress(byte[] compressedData) {
        ByteArrayInputStream bis = null;
        ByteArrayOutputStream bos = null;
        GZIPInputStream gzipIS = null;

        try {
            bis = new ByteArrayInputStream(compressedData);
            bos = new ByteArrayOutputStream();
            gzipIS = new GZIPInputStream(bis);

            byte[] buffer = new byte[1024];
            int len;
            while((len = gzipIS.read(buffer)) != -1){
                bos.write(buffer, 0, len);
            }
            return bos.toByteArray();
        } catch (IOException e) {
            e.printStackTrace();
        }
        finally {
            try {
                assert gzipIS != null;
                gzipIS.close();
                bos.close();
                bis.close();
            }
            catch (Exception e) {
                e.printStackTrace();
            }
        }
        return new byte[]{};
    }

Raposa

Answer 5

Most of the examples have wrong exception handling.

public static byte[] gzipBytes(byte[] payload) {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    try (GZIPOutputStream gzip = new GZIPOutputStream(baos)) {
        gzip.write(payload);
    } catch (IOException e) {
        throw new UncheckedIOException(e);
    }
    // note: toByteArray should be called after try-with-resources, not inside
    return baos.toByteArray();
}

public static byte[] gunzipBytes(byte[] gzPayload) {
    ByteArrayInputStream bais = new ByteArrayInputStream(gzPayload);
    try (GZIPInputStream gzip = new GZIPInputStream(bais)) {
        // java 9+ required for this method
        return gzip.readAllBytes();
    } catch (IOException e) {
        throw new UncheckedIOException("Error while unpacking gzip content", e);
    }
}

Nigritude

Answer 6

You can use the below function, it is tested and working fine.

In general, your code has serious problem of ignoring the exceptions! returning null or simply not printing anything in the catch block will make it very difficult to debug

You do not have to write the zip output to a file if you want to process it further (e.g. encrypt it), you can easily modify the code to write the output to in-memory stream

public static String zip(File inFile, File zipFile) throws IOException {        
    FileInputStream fis = new FileInputStream(inFile);
    FileOutputStream fos = new FileOutputStream(zipFile);
    ZipOutputStream zout = new ZipOutputStream(fos);

    try {
        zout.putNextEntry(new ZipEntry(inFile.getName()));
        byte[] buffer = new byte[BUFFER_SIZE];
        int len;
        while ((len = fis.read(buffer)) > 0) {
            zout.write(buffer, 0, len);
        }
        zout.closeEntry();
    } catch (Exception ex) {
        ex.printStackTrace();
        return null;
    } finally {
        try{zout.close();}catch(Exception ex){ex.printStackTrace();}
        try{fis.close();}catch(Exception ex){ex.printStackTrace();}         
    }
    return zipFile.getAbsolutePath();
}

Meit

It is possible to add each of these close() statement in a try/catch block and ignore the error – Meit

Answer 7

Try with this code..

try {
    String inputFileName = "test.txt";  //may use your file_Path
    String zipFileName = "compressed.zip";

    //Create input and output streams
    FileInputStream inStream = new FileInputStream(inputFileName);
    ZipOutputStream outStream = new ZipOutputStream(new FileOutputStream(zipFileName));

    // Add a zip entry to the output stream
    outStream.putNextEntry(new ZipEntry(inputFileName));

    byte[] buffer = new byte[1024];
    int bytesRead;

    //Each chunk of data read from the input stream
    //is written to the output stream
    while ((bytesRead = inStream.read(buffer)) > 0) {
        outStream.write(buffer, 0, bytesRead);
    }

    //Close zip entry and file streams
    outStream.closeEntry();

    outStream.close();
    inStream.close();

} catch (IOException ex) {
    ex.printStackTrace();
}

Also may be helpful this one..

• http://www.java-samples.com/java/zip_files_in_a_folder_using_java.htm

Vaccinia

This code writes directly to a ZIP file; which is what I don't want. But thank you anyway. – Lowney