Easy way to write contents of a Java InputStream to an OutputStream

Question

I was surprised to find today that I couldn't track down any simple way to write the contents of an InputStream to an OutputStream in Java. Obviously, the byte buffer code isn't difficult to write, but I suspect I'm just missing something which would make my life easier (and the code clearer).

So, given an InputStream in and an OutputStream out, is there a simpler way to write the following?

byte[] buffer = new byte[1024];
int len = in.read(buffer);
while (len != -1) {
    out.write(buffer, 0, len);
    len = in.read(buffer);
}

Boschbok

You mentioned in a comment that this is for a mobile app. Is it native Android? If so, let me know and I'll post another answer (it can be done is a single line of code in Android). – Masse

Answer 1

Java 9

Since Java 9, InputStream provides a method called transferTo with the following signature:

public long transferTo(OutputStream out) throws IOException

As the documentation states, transferTo will:

Reads all bytes from this input stream and writes the bytes to the given output stream in the order that they are read. On return, this input stream will be at end of stream. This method does not close either stream.

This method may block indefinitely reading from the input stream, or writing to the output stream. The behavior for the case where the input and/or output stream is asynchronously closed, or the thread interrupted during the transfer, is highly input and output stream specific, and therefore not specified

So in order to write contents of a Java InputStream to an OutputStream, you can write:

input.transferTo(output);

Flintlock

You should preferFiles.copy as much as possible. It is implemented in native code and therefore can be faster. transferTo should be used only if both streams are not FileInputStream/FileOutputStream. – Homologate

@Homologate Unfortunately Files.copy does not handle any input/output streams but it's specifically designed for file streams. – Verism

Also only available on >API 26 – Vinita

@Homologate It seems that Files.copy(in, out) is also using the transferTo method under the hood. So it seems there is no native code unless JVM provides an instrinsic for Files.copy(in, out) – Flintlock

Answer 2

As WMR mentioned, org.apache.commons.io.IOUtils from Apache has a method called copy(InputStream,OutputStream) which does exactly what you're looking for.

So, you have:

InputStream in;
OutputStream out;
IOUtils.copy(in,out);
in.close();
out.close();

...in your code.

Is there a reason you're avoiding IOUtils?

Wino

I'm avoiding it for this mobile app I'm building cause it'd quintuple the size of the app to save a measly 5 lines of code. – Quennie

@basZero Or using a try with resources block. – Brittney

If you're already using the Guava library, Andrejs has recommended the ByteStreams class below. Similar to what IOUtils does, but avoids adding Commons IO to your project. – Brahmaputra

@fiXedd You can use Maven Shade to strip unneeded classes from the final .jar, thus incurring only a modest increase in file jar size – Psychodrama

Kind of obvious, but if the class doesn't have too many dependencies, you can also simply copy the source code for liberal licenses (like those used for both Guava and Apache). First read the license though (disclaimer, IANAL etc.). – She

If you are using a dependency manager, like Maven, and you're avoiding IOUtils and Guava for size reasons, you might want to have a look at your dependency hierarchy. Maybe one of your dependencies transitively includes one of these without notice, they are very famous after all. If that's the case, you might as well use them yourself , you're including them anyway. – Hausa

You have the answer in your own question. Just create your own Utility class and have a static method with the example code you have in your question. Any library that does what you want is gonna do that behind the scenes anyways. Why get messy with libraries? – Supernational

Answer 3

If you are using Java 7, Files (in the standard library) is the best approach:

/* You can get Path from file also: file.toPath() */
Files.copy(InputStream in, Path target)
Files.copy(Path source, OutputStream out)

Edit: Of course it's just useful when you create one of InputStream or OutputStream from file. Use file.toPath() to get path from file.

To write into an existing file (e.g. one created with File.createTempFile()), you'll need to pass the REPLACE_EXISTING copy option (otherwise FileAlreadyExistsException is thrown):

Files.copy(in, target, StandardCopyOption.REPLACE_EXISTING)

Negativism

I don't think this actually solves the problem since one end is a path. While you can get a path for a file, as far as I am aware you can't get one for any generic stream (e.g. one over the network). – Boschbok

+1 still turned out to be useful in my case though, as I have a real file on the output side! – Alena

As far as I'm concerned, te first method you say is not available with that signature. You have to add the CopyOptions parameter. – Buenrostro

CopyOptions is arbitrary! You can put it here if you want it. – Negativism

now this is what I was looking for! JDK to the rescue, no need for another library – Notarial

FYI, Files is NOT available in Android's Java 1.7. I got stung by this: #24869823 – Valdavaldas

Amusingly, the JDK also has a Files.copy() which takes two streams, and is what all the other Files.copy() functions forward to in order to do the actual work of copying. However, it's private (since it doesn't actually involve Paths or Files at that stage), and looks exactly like the code in the OP's own question (plus a return statement). No opening, no closing, just a copy loop. – Goulette

For Java 9 and later, use InputStream.transferTo(OutputStream). https://mcmap.net/q/73847/-easy-way-to-write-contents-of-a-java-inputstream-to-an-outputstream – Gutshall

Files.copy doesn't have backward support for API below 26 @Negativism – Shf

I'm curious as to why Files doesn't have a copy(InputStream, OutputStream) variant. I assume there's a good reason for that, like maybe something will deadlock on a full buffer or something, but I'm not seeing it. – Bibliotherapy

Answer 4

I think this will work, but make sure to test it... minor "improvement", but it might be a bit of a cost at readability.

byte[] buffer = new byte[1024];
int len;
while ((len = in.read(buffer)) != -1) {
    out.write(buffer, 0, len);
}

Xantha

I suggest a buffer of at least 10KB to 100KB. That's not much and can speed up copying large amounts of data tremendously. – Shearer

This did the trick for me when writing a file. I did not realize I needed to get the actual length readd from the input stream and use that to output to a file. – Wahkuna

you might want to say while(len > 0) instead of != -1, because the latter could also return 0 when using the read(byte b[], int off, int len)-method, which throws an exception @ out.write – Monas

@Blauhirn: That would be incorrect, as it is entirely legal according to the InputStream contract for read to return 0 any number of times. And according to the OutputStream contract, the write method must accept a length of 0, and should only throw an exception when len is negative. – Utmost

You can save a line by changing the while to a for and putting one of the variables in for's init section: e.g., for (int n ; (n = in.read(buf)) != -1 ;) out.write(buf, 0, n);. =) – Elanorelapid

@Blauhim read() can only return zero if you supplied a length of zero, which would be a programming error, and a stupid condition to loop forever on. And write() does not throw an exception if you provide a zero length. – Gober

@ChristofferHammarström It is not legal for InputStream.read(byte[] b) to return 0 (unless you're stupid like EJP said): If the length of b is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at the end of the file, the value -1 is returned; otherwise, at least one byte is read and stored into b. – Marginal

@Andreas: Thanks! I never realized. I could imagine use cases though where you'd want to pass a zero length buffer, or when it happens for other reasons. So just like i always wear my seat belt, i will still always check for -1. Stopping reading and discarding remaining data is at least as much an error as passing an empty buffer. – Utmost

If the length of buffer is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at the end of the file, the value -1 is returned; otherwise, at least one byte is read and stored into buffer. - so indeed, in this code, a return of 0 cannot occur as the size of the buffer is larger. – Retuse

Answer 5

Using Guava's ByteStreams.copy():

ByteStreams.copy(inputStream, outputStream);

Heckman

Do not forget to close the streams after that! – Crafton

This is the best answer if you are already using Guava which has become indispensable for me. – Yunyunfei

@Yunyunfei You should use Files.copy as much as possible. Use ByteStreams.copy only if both of streams are not FileInputStream/FileOutputStream. – Homologate

Answer 6

Simple Function

If you only need this for writing an InputStream to a File then you can use this simple function:

private void copyInputStreamToFile( InputStream in, File file ) {
    try {
        OutputStream out = new FileOutputStream(file);
        byte[] buf = new byte[1024];
        int len;
        while((len=in.read(buf))>0){
            out.write(buf,0,len);
        }
        out.close();
        in.close();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Frausto

Great function, thanks. Would you need to put the close() calls in finally blocks, though? – Valdavaldas

@JoshPinter It wouldn't hurt. – Frausto

You probably should both include a finally block and not swallow exceptions in an actual implementation. Also, closing an InputStream passed to a method is sometimes unexpected by the calling method, so one should consider if it's the behavior they want. – Griz

Why catch the Exception when IOException suffices? – Obturate

Might cause Vulnerability issues if we use this code. One of the best practice is found here.Please modify accordingly. – Angie

Answer 7

The JDK uses the same code so it seems like there is no "easier" way without clunky third party libraries (which probably don't do anything different anyway). The following is directly copied from java.nio.file.Files.java:

// buffer size used for reading and writing
private static final int BUFFER_SIZE = 8192;

/**
  * Reads all bytes from an input stream and writes them to an output stream.
  */
private static long copy(InputStream source, OutputStream sink) throws IOException {
    long nread = 0L;
    byte[] buf = new byte[BUFFER_SIZE];
    int n;
    while ((n = source.read(buf)) > 0) {
        sink.write(buf, 0, n);
        nread += n;
    }
    return nread;
}

Inveigh

Aye. Shame this particular call is private and there's no other option but to copy it out into your own utilities class, since it's possible that you're not dealing with files, but rather 2 sockets at once. – Viquelia

Answer 8

For those who use Spring framework there is a useful StreamUtils class:

StreamUtils.copy(in, out);

The above does not close the streams. If you want the streams closed after the copy, use FileCopyUtils class instead:

FileCopyUtils.copy(in, out);

Carolus

Answer 9

PipedInputStream and PipedOutputStream should only be used when you have multiple threads, as noted by the Javadoc.

Also, note that input streams and output streams do not wrap any thread interruptions with IOExceptions... So, you should consider incorporating an interruption policy to your code:

byte[] buffer = new byte[1024];
int len = in.read(buffer);
while (len != -1) {
    out.write(buffer, 0, len);
    len = in.read(buffer);
    if (Thread.interrupted()) {
        throw new InterruptedException();
    }
}

This would be an useful addition if you expect to use this API for copying large volumes of data, or data from streams that get stuck for an intolerably long time.

Synopsis

Answer 10

There's no way to do this a lot easier with JDK methods, but as Apocalisp has already noted, you're not the only one with this idea: You could use IOUtils from Jakarta Commons IO, it also has a lot of other useful things, that IMO should actually be part of the JDK...

Excurrent

Answer 11

Using Java7 and try-with-resources, comes with a simplified and readable version.

try(InputStream inputStream = new FileInputStream("C:\\mov.mp4");
    OutputStream outputStream = new FileOutputStream("D:\\mov.mp4")) {

    byte[] buffer = new byte[10*1024];

    for (int length; (length = inputStream.read(buffer)) != -1; ) {
        outputStream.write(buffer, 0, length);
    }
} catch (FileNotFoundException exception) {
    exception.printStackTrace();
} catch (IOException ioException) {
    ioException.printStackTrace();
}

Wattle

Flushing inside the loop is highly counter-productive. – Gober

Answer 12

Here comes how I'm doing with a simplest for loop.

private void copy(final InputStream in, final OutputStream out)
    throws IOException {
    final byte[] b = new byte[8192];
    for (int r; (r = in.read(b)) != -1;) {
        out.write(b, 0, r);
    }
}

Introvert

Answer 13

Use Commons Net's Util class:

import org.apache.commons.net.io.Util;
...
Util.copyStream(in, out);

Supplant

Answer 14

I use BufferedInputStream and BufferedOutputStream to remove the buffering semantics from the code

try (OutputStream out = new BufferedOutputStream(...);
     InputStream in   = new BufferedInputStream(...))) {
  int ch;
  while ((ch = in.read()) != -1) {
    out.write(ch);
  }
}

Bourque

Why is 'removing the buffering semantics from the code' a good idea? – Gober

It means I don't write the buffering logic myself, I use the one built into the JDK which is usually good enough. – Bourque

Answer 15

A IMHO more minimal snippet (that also more narrowly scopes the length variable):

byte[] buffer = new byte[2048];
for (int n = in.read(buffer); n >= 0; n = in.read(buffer))
    out.write(buffer, 0, n);

As a side note, I don't understand why more people don't use a for loop, instead opting for a while with an assign-and-test expression that is regarded by some as "poor" style.

Olfe

Your suggestion causes a 0-byte write on the first iteration. Perhaps least do: for(int n = 0; (n = in.read(buffer)) > 0;) { out.write(buffer, 0, n); } – Alex

@BriandeAlwis You are right about the first iteration being incorrect. The code has been fixed (IMHO in a cleaner way than your suggestion) - see edited code. Thx for caring. – Olfe

Answer 16

This is my best shot!!

And do not use inputStream.transferTo(...) because is too generic. Your code performance will be better if you control your buffer memory.

public static void transfer(InputStream in, OutputStream out, int buffer) throws IOException {
    byte[] read = new byte[buffer]; // Your buffer size.
    while (0 < (buffer = in.read(read)))
        out.write(read, 0, buffer);
}

I use it with this (improvable) method when I know in advance the size of the stream.

public static void transfer(int size, InputStream in, OutputStream out) throws IOException {
    transfer(in, out,
            size > 0xFFFF ? 0xFFFF // 16bits 65,536
                    : size > 0xFFF ? 0xFFF// 12bits 4096
                            : size < 0xFF ? 0xFF // 8bits 256
                                    : size
    );
}

Fineman

"And do not use inputStream.transferTo(...) because is too generic. Your code performance will be better if you control your buffer memory." That sounds plausible, and indeed my own code originally tried to pick buffer sizes based upon known transfer size. But I'm reading the answer may be more complicated based in part upon drive block sizes and CPU cache. Have you done any real-world tests to back up your claim that custom buffer sizes perform better than InputStream.transferTo(OutputStream)? If so I'd be interested to see them. Performance is tricky. – Balbur

Have you actually seen how transferTo is implemented? – Clerissa

Answer 17

I think it's better to use a large buffer, because most of the files are greater than 1024 bytes. Also it's a good practice to check the number of read bytes to be positive.

byte[] buffer = new byte[4096];
int n;
while ((n = in.read(buffer)) > 0) {
    out.write(buffer, 0, n);
}
out.close();

Conant

Using a large buffer is indeed a good idea but not because files are mostly > 1k, it is to amortize the cost of system calls. – Gober

Might cause Vulnerability issues if we use this code. One of the best practice is found here.Please modify accordingly. – Angie

Answer 18

Not very readable, but effective, has no dependencies and runs with any java version

byte[] buffer=new byte[1024];
for(int n; (n=inputStream.read(buffer))!=-1; outputStream.write(buffer,0,n));

Sled

!= -1 or > 0? Those predicates are not quite the same. – Verism

!= -1 means not-end-of-file. This is not an iteration but a while-do-loop in disguise: while((n = inputStream.read(buffer)) != -1) do { outputStream.write(buffer, 0,n) } – Sled

Answer 19

PipedInputStream and PipedOutputStream may be of some use, as you can connect one to the other.

Tortoise

This is not good for single-threaded code as it could deadlock; see this question #484619 – Cent

May be of some use how? He already has an input stream and an output stream. How will adding another one of each help exactly? – Gober

Answer 20

Another possible candidate are the Guava I/O utilities:

http://code.google.com/p/guava-libraries/wiki/IOExplained

I thought I'd use these since Guava is already immensely useful in my project, rather than adding yet another library for one function.

Sinless

There are copy and toByteArray methods in docs.guava-libraries.googlecode.com/git-history/release/javadoc/… (guava calls input/output streams as "byte streams" and readers/writers as "char streams") – Cent

if you already use guava libraries it's a good idea, but if not, they are a mammoth library with thousands of methods 'google-way-of-doing-everything-different-to-the-standard'. I'd keep away from them – Paperboard

"mammoth"? 2.7MB with a very small set of dependencies, and an API that carefully avoids duplicating the core JDK. – Webster

Answer 21

I used ByteStreamKt.copyTo(src, dst, buffer.length) method

Here is my code

public static void replaceCurrentDb(Context context, Uri newDbUri) {
    try {
        File currentDb = context.getDatabasePath(DATABASE_NAME);
        if (currentDb.exists()) {
            InputStream src = context.getContentResolver().openInputStream(newDbUri);
            FileOutputStream dst = new FileOutputStream(currentDb);
            final byte[] buffer = new byte[8 * 1024];
            ByteStreamsKt.copyTo(src, dst, buffer.length);
            src.close();
            dst.close();
            Toast.makeText(context, "SUCCESS! Your selected file is set as current menu.", Toast.LENGTH_LONG).show();
        }
        else
            Log.e("DOWNLOAD:::: Database", " fail, database not found");
    }
    catch (IOException e) {
        Toast.makeText(context, "Data Download FAIL.", Toast.LENGTH_LONG).show();
        Log.e("DOWNLOAD FAIL!!!", "fail, reason:", e);
    }
}

Mcswain

Answer 22

public static boolean copyFile(InputStream inputStream, OutputStream out) {
    byte buf[] = new byte[1024];
    int len;
    long startTime=System.currentTimeMillis();

    try {
        while ((len = inputStream.read(buf)) != -1) {
            out.write(buf, 0, len);
        }

        long endTime=System.currentTimeMillis()-startTime;
        Log.v("","Time taken to transfer all bytes is : "+endTime);
        out.close();
        inputStream.close();

    } catch (IOException e) {

        return false;
    }
    return true;
}

Matrimony

Can you please explain why this is the right answer? – Denbrook

Answer 23

Try Cactoos:

new LengthOf(new TeeInput(input, output)).value();

More details here: http://www.yegor256.com/2017/06/22/object-oriented-input-output-in-cactoos.html

Inwardness

Answer 24

you can use this method

public static void copyStream(InputStream is, OutputStream os)
 {
     final int buffer_size=1024;
     try
     {
         byte[] bytes=new byte[buffer_size];
         for(;;)
         {
           int count=is.read(bytes, 0, buffer_size);
           if(count==-1)
               break;
           os.write(bytes, 0, count);
         }
     }
     catch(Exception ex){}
 }

Echo

catch(Exception ex){} — this is top-notch – Phenology