Reading a binary input stream into a single byte array in Java

Question

The documentation says that one should not use available() method to determine the size of an InputStream. How can I read the whole content of an InputStream into a byte array?

InputStream in; //assuming already present
byte[] data = new byte[in.available()];
in.read(data);//now data is filled with the whole content of the InputStream

I could read multiple times into a buffer of a fixed size, but then, I will have to combine the data I read into a single byte array, which is a problem for me.

Public

Answer 1

The simplest approach IMO is to use Guava and its ByteStreams class:

byte[] bytes = ByteStreams.toByteArray(in);

Or for a file:

byte[] bytes = Files.toByteArray(file);

Alternatively (if you didn't want to use Guava), you could create a ByteArrayOutputStream, and repeatedly read into a byte array and write into the ByteArrayOutputStream (letting that handle resizing), then call ByteArrayOutputStream.toByteArray().

Note that this approach works whether you can tell the length of your input or not - assuming you have enough memory, of course.

Lowery

thank you. Made it work now. I guess I will have to read in chunks and then merge the resulting byte arrays into a single one. – Public

@Qiang: That's basically what all of the solutions I've presented do - you don't need to do much work yourself for this. – Lowery

Answer 2

Please keep in mind that the answers here assume that the length of the file is less than or equal to Integer.MAX_VALUE(2147483647).

If you are reading in from a file, you can do something like this:

    File file = new File("myFile");
    byte[] fileData = new byte[(int) file.length()];
    DataInputStream dis = new DataInputStream(new FileInputStream(file));
    dis.readFully(fileData);
    dis.close();

---

UPDATE (May 31, 2014):

Java 7 adds some new features in the java.nio.file package that can be used to make this example a few lines shorter. See the readAllBytes() method in the java.nio.file.Files class. Here is a short example:

import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;

// ...
        Path p = FileSystems.getDefault().getPath("", "myFile");
        byte [] fileData = Files.readAllBytes(p);

Android has support for this starting in Api level 26 (8.0.0, Oreo).

Enaenable

unforuntately, I am not reading from a file. I am reading from a ZipEntry. Is there anyway to get the file length similar to here? – Public

Answer 3

You can use Apache commons-io for this task:

Refer to this method:

public static byte[] readFileToByteArray(File file) throws IOException

Update:

Java 7 way:

byte[] bytes = Files.readAllBytes(Paths.get(filename));

and if it is a text file and you want to convert it to String (change encoding as needed):

StandardCharsets.UTF_8.decode(ByteBuffer.wrap(bytes)).toString()

Dyer

Answer 4

You can read it by chunks (byte buffer[] = new byte[2048]) and write the chunks to a ByteArrayOutputStream. From the ByteArrayOutputStream you can retrieve the contents as a byte[], without needing to determine its size beforehand.

Dredi

Answer 5

I believe buffer length needs to be specified, as memory is finite and you may run out of it

Example:

InputStream in = new FileInputStream(strFileName);
    long length = fileFileName.length();

    if (length > Integer.MAX_VALUE) {
        throw new IOException("File is too large!");
    }

    byte[] bytes = new byte[(int) length];

    int offset = 0;
    int numRead = 0;

    while (offset < bytes.length && (numRead = in.read(bytes, offset, bytes.length - offset)) >= 0) {
        offset += numRead;
    }

    if (offset < bytes.length) {
        throw new IOException("Could not completely read file " + fileFileName.getName());
    }

    in.close();

Lucie

Answer 6

Max value for array index is Integer.MAX_INT - it's around 2Gb (2^31 / 2 147 483 647). Your input stream can be bigger than 2Gb, so you have to process data in chunks, sorry.

        InputStream is;
        final byte[] buffer = new byte[512 * 1024 * 1024]; // 512Mb
        while(true) {
            final int read = is.read(buffer);
            if ( read < 0 ) {
                break;
            }
            // do processing 
        }

Supersaturate

I would -1 weren't for the info on max int - you should not write unnecessarily complicated code - while(true) for gosh sake - please edit : int read; while(read = is.read(buffer) > 0 ) – Frantz