Download binary file from OKHTTP
Asked Answered
D

11

90

I am using OKHTTP client for networking in my android application.

This example shows how to upload binary file. I would like to know how to get inputstream of binary file downloading with OKHTTP client.

Here is the listing of the example :

public class InputStreamRequestBody extends RequestBody {

    private InputStream inputStream;
    private MediaType mediaType;

    public static RequestBody create(final MediaType mediaType, 
            final InputStream inputStream) {
        return new InputStreamRequestBody(inputStream, mediaType);
    }

    private InputStreamRequestBody(InputStream inputStream, MediaType mediaType) {
        this.inputStream = inputStream;
        this.mediaType = mediaType;
    }

    @Override
    public MediaType contentType() {
        return mediaType;
    }

    @Override
    public long contentLength() {
        try {
            return inputStream.available();
        } catch (IOException e) {
            return 0;
        }
    }

    @Override
    public void writeTo(BufferedSink sink) throws IOException {
        Source source = null;
        try {
            source = Okio.source(inputStream);
            sink.writeAll(source);
        } finally {
            Util.closeQuietly(source);
        }
    }
}

Current code for simple get request is:

OkHttpClient client = new OkHttpClient();
request = new Request.Builder().url("URL string here")
                    .addHeader("X-CSRFToken", csrftoken)
                    .addHeader("Content-Type", "application/json")
                    .build();
response = getClient().newCall(request).execute();

Now how do I convert the response to InputStream. Something similar to response from Apache HTTP Client like this for OkHttp response:

InputStream is = response.getEntity().getContent();

EDIT

Accepted answer from below. My modified code:

request = new Request.Builder().url(urlString).build();
response = getClient().newCall(request).execute();

InputStream is = response.body().byteStream();

BufferedInputStream input = new BufferedInputStream(is);
OutputStream output = new FileOutputStream(file);

byte[] data = new byte[1024];

long total = 0;

while ((count = input.read(data)) != -1) {
    total += count;
    output.write(data, 0, count);
}

output.flush();
output.close();
input.close();
Distillation answered 17/9, 2014 at 14:14 Comment(3)
check my edited answer i'm waiting your feedbackLinkboy
glad it worked for you man :DLinkboy
as a side note, your InputStreamRequestBody is not going to work if there is an ioexception in the request and the HttpEngine is set to retry. See github.com/square/okhttp/blob/master/okhttp/src/main/java/com/… line 202, writeTo is called in a while loop. It will cause an error. (I stumbled upon this while uploading from a content:// inputstream)Alight
L
41

Getting ByteStream from OKHTTP

I've been digging around in the Documentation of OkHttp you need to go this way

use this method :

response.body().byteStream() wich will return an InputStream

so you can simply use a BufferedReader or any other alternative

OkHttpClient client = new OkHttpClient();
request = new Request.Builder().url("URL string here")
                     .addHeader("X-CSRFToken", csrftoken)
                     .addHeader("Content-Type", "application/json")
                     .build();
response = getClient().newCall(request).execute();

InputStream in = response.body().byteStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
String result, line = reader.readLine();
result = line;
while((line = reader.readLine()) != null) {
    result += line;
}
System.out.println(result);
response.body().close();
Linkboy answered 17/9, 2014 at 14:17 Comment(6)
The example you mentioned from documentation will work for a request with simple data or webpage but for downloading a binary file, wouldn't I need a bufferedinputstream? So that I can get the bytes in buffer and write them to Fileoutputstream to save in local storage?Distillation
Edited with solution for binaryLinkboy
OKHTTP response does not have getEntity() method like Apache HTTP Client. Made edits in the questionDistillation
Edited with ultimate answer ;)Linkboy
Do not use BufferedReader with binary data; it will be corrupted by the unnecessary byte-to-character decoding.Weekday
Why do you close() the response body?Theoretician
T
218

For what it's worth, I would recommend response.body().source() from okio (since OkHttp is already supporting it natively) in order to enjoy an easier way to manipulate a large quantity of data that can come when downloading a file.

@Override
public void onResponse(Call call, Response response) throws IOException {
    File downloadedFile = new File(context.getCacheDir(), filename);
    BufferedSink sink = Okio.buffer(Okio.sink(downloadedFile));
    sink.writeAll(response.body().source());
    sink.close();
}

A couple of advantages taken from the documentation in comparison with InputStream:

This interface is functionally equivalent to InputStream. InputStream requires multiple layers when consumed data is heterogeneous: a DataInputStream for primitive values, a BufferedInputStream for buffering, and InputStreamReader for strings. This class uses BufferedSource for all of the above. Source avoids the impossible-to-implement available() method. Instead callers specify how many bytes they require.

Source omits the unsafe-to-compose mark and reset state that's tracked by InputStream; callers instead just buffer what they need.

When implementing a source, you need not worry about the single-byte read method that is awkward to implement efficiently and that returns one of 257 possible values.

And source has a stronger skip method: BufferedSource.skip(long) won't return prematurely.

Testee answered 12/3, 2015 at 14:51 Comment(8)
This approach is also the fastest, since there won't be any unnecessary copies of the response data.Weekday
How handle progress with this approach?Borreri
@Borreri simply use write(Source source, long byteCount) and a loop. Of course, you need to get contentLength from the body to ensure that you read the correct number of bytes. Fire up an event at each loop.Testee
Thanks. But Is there a way without knowing contentLength? Something of while ((source.read(fileSink, 2048)) != -1). In this code i have an issue with casting: Buffer needs, but BufferedSink found.Borreri
Solved! Working code: while ((source.read(fileSink.buffer(), 2048)) != -1)Borreri
Solutions we find ourselves are the one we remember the best. Happy you found a solution.Testee
@Borreri can you please provide full code with progress bar...i am trying the same but cant achieveFilefish
@Theoretician it does, but you have to implement a Zip deflater to get all the files you need down the line if you expect to get access to the files inside the archive.Testee
L
41

Getting ByteStream from OKHTTP

I've been digging around in the Documentation of OkHttp you need to go this way

use this method :

response.body().byteStream() wich will return an InputStream

so you can simply use a BufferedReader or any other alternative

OkHttpClient client = new OkHttpClient();
request = new Request.Builder().url("URL string here")
                     .addHeader("X-CSRFToken", csrftoken)
                     .addHeader("Content-Type", "application/json")
                     .build();
response = getClient().newCall(request).execute();

InputStream in = response.body().byteStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
String result, line = reader.readLine();
result = line;
while((line = reader.readLine()) != null) {
    result += line;
}
System.out.println(result);
response.body().close();
Linkboy answered 17/9, 2014 at 14:17 Comment(6)
The example you mentioned from documentation will work for a request with simple data or webpage but for downloading a binary file, wouldn't I need a bufferedinputstream? So that I can get the bytes in buffer and write them to Fileoutputstream to save in local storage?Distillation
Edited with solution for binaryLinkboy
OKHTTP response does not have getEntity() method like Apache HTTP Client. Made edits in the questionDistillation
Edited with ultimate answer ;)Linkboy
Do not use BufferedReader with binary data; it will be corrupted by the unnecessary byte-to-character decoding.Weekday
Why do you close() the response body?Theoretician
D
14

The best option to download (based on source code "okio")

private void download(@NonNull String url, @NonNull File destFile) throws IOException {
    Request request = new Request.Builder().url(url).build();
    Response response = okHttpClient.newCall(request).execute();
    ResponseBody body = response.body();
    long contentLength = body.contentLength();
    BufferedSource source = body.source();

    BufferedSink sink = Okio.buffer(Okio.sink(destFile));
    Buffer sinkBuffer = sink.buffer();

    long totalBytesRead = 0;
    int bufferSize = 8 * 1024;
    for (long bytesRead; (bytesRead = source.read(sinkBuffer, bufferSize)) != -1; ) {
        sink.emit();
        totalBytesRead += bytesRead;
        int progress = (int) ((totalBytesRead * 100) / contentLength);
        publishProgress(progress);
    }
    sink.flush();
    sink.close();
    source.close();
}
Deanadeanda answered 13/4, 2016 at 9:23 Comment(4)
write downloadFile methodLenity
i cant get body.contentLength()...its always -1Filefish
Not positive but you'll probably want to put the flush() and close() methods in a finally block to ensure that any exceptions will still flush and close them.Lois
NOTE: sink.buffer() is now deprecated in favour of sink.getBuffer() (Java) or sink.buffer (Kotlin): square.github.io/okio/changelog/#version-210Lois
C
10

This is how I use Okhttp + Okio libraries while publishing download progress after every chunk download:

public static final int DOWNLOAD_CHUNK_SIZE = 2048; //Same as Okio Segment.SIZE

try {
        Request request = new Request.Builder().url(uri.toString()).build();

        Response response = client.newCall(request).execute();
        ResponseBody body = response.body();
        long contentLength = body.contentLength();
        BufferedSource source = body.source();

        File file = new File(getDownloadPathFrom(uri));
        BufferedSink sink = Okio.buffer(Okio.sink(file));

        long totalRead = 0;
        long read = 0;
        while ((read = source.read(sink.buffer(), DOWNLOAD_CHUNK_SIZE)) != -1) {
            totalRead += read;
            int progress = (int) ((totalRead * 100) / contentLength);
            publishProgress(progress);
        }
        sink.writeAll(source);
        sink.flush();
        sink.close();
        publishProgress(FileInfo.FULL);
} catch (IOException e) {
        publishProgress(FileInfo.CODE_DOWNLOAD_ERROR);
        Logger.reportException(e);
}
Coerce answered 17/12, 2015 at 21:59 Comment(7)
what getDownloadPathFrom do ?Lamia
write missing methods plsLenity
i cant get body.contentLength()...its always -1Filefish
while (read = (source.read(sink.buffer(), DOWNLOAD_CHUNK_SIZE)) != -1) { should be while ((read = source.read(sink.buffer(), DOWNLOAD_CHUNK_SIZE)) != -1) {Yearling
If you don't call sink.emit(); in the while cycle you'll use a huge amount of memory.Diamagnetic
why no source.close() ?Khalkha
Doesn't sink.flush() write all the data to the file here. Why do we need to do sink.writeAll?Zenithal
B
6

Better solution is to use OkHttpClient as:

OkHttpClient client = new OkHttpClient();

            Request request = new Request.Builder()
                    .url("http://publicobject.com/helloworld.txt")
                    .build();



            client.newCall(request).enqueue(new Callback() {
                @Override
                public void onFailure(Call call, IOException e) {
                    e.printStackTrace();
                }

                @Override
                public void onResponse(Call call, Response response) throws IOException {

                    if (!response.isSuccessful()) throw new IOException("Unexpected code " + response);

//                    Headers responseHeaders = response.headers();
//                    for (int i = 0; i < responseHeaders.size(); i++) {
//                        System.out.println(responseHeaders.name(i) + ": " + responseHeaders.value(i));
//                    }
//                    System.out.println(response.body().string());

                    InputStream in = response.body().byteStream();
                    BufferedReader reader = new BufferedReader(new InputStreamReader(in));
                    String result, line = reader.readLine();
                    result = line;
                    while((line = reader.readLine()) != null) {
                        result += line;
                    }
                    System.out.println(result);


                }
            });
Bobbery answered 17/2, 2016 at 10:49 Comment(3)
i cant get body.contentLength()...its always -1Filefish
@HRaval that is not an okhttp issue, that simply means the server didn't send a "Content-Length" headerAttainder
@Joolah how to store that file in internal storage ?Cuckoopint
B
5

Kotlin version based on kiddouk answer

 val request = Request.Builder().url(url).build()
 val response = OkHttpClient().newCall(request).execute()
 val downloadedFile = File(cacheDir, filename)
 val sink: BufferedSink = downloadedFile.sink().buffer()
 sink.writeAll(response.body!!.source())
 sink.close()
Bullis answered 23/9, 2020 at 4:7 Comment(0)
C
4

Update in Kotlin to match KiddoUK's answer from 2015 (https://mcmap.net/q/235802/-download-binary-file-from-okhttp):

val sourceBytes = response.body().source()
val sink: BufferedSink = File(downloadLocationFilePath).sink().buffer()
sink.writeAll(sourceBytes)
sink.close()

and to add progress monitoring:

val sourceBytes = response.body().source()
val sink: BufferedSink = File(downloadLocationFilePath).sink().buffer()

var totalRead: Long = 0
var lastRead: Long
while (sourceBytes
          .read(sink.buffer, 8L * 1024)
          .also { lastRead = it } != -1L 
) {
    totalRead += lastRead
    sink.emitCompleteSegments()
    // Call your listener/callback here with the totalRead value        
}

sink.writeAll(sourceBytes)
sink.close()
Cootch answered 13/8, 2021 at 12:32 Comment(1)
Great answer. It goes without saying that totalRead on itself is not an indicator of progress, it needs to be compared with body.contentLength().Garvin
E
3

Here is a quick method to do it.

  • client - OkHttpClient
  • url - the URL you wish to download
  • destination - where do you wish to save it
    fun download(url: String, destination: File) = destination.outputStream().use { output ->
        client.newCall(Request.Builder().url(url).build())
            .execute().body!!.byteStream().use { input ->
                input.copyTo(output)
            }
    }
Escarpment answered 4/4, 2022 at 21:27 Comment(0)
L
1

If you are trying to write downloaded bytes into Shared Storage on latest Android, you should have Uri on your hand instead of File instance. Here is how to convert Uri to OutputStream:

fun Uri?.toOutputStream(context: Context)
        : OutputStream? {
    if (this == null) {
        return null
    }

    fun createAssetFileDescriptor() = try {
        context.contentResolver.openAssetFileDescriptor(this, "w")
    } catch (e: FileNotFoundException) {
        null
    }

    fun createParcelFileDescriptor() = try {
        context.contentResolver.openFileDescriptor(this, "w")
    } catch (e: FileNotFoundException) {
        null
    }

    /** scheme://<authority>/<path>/<id> */
    if (scheme.equals(ContentResolver.SCHEME_FILE)) {
        /** - If AssetFileDescriptor is used, it always writes 0B.
         * - (FileOutputStream | ParcelFileDescriptor.AutoCloseOutputStream) works both for app-specific + shared storage
         * - If throws "FileNotFoundException: open failed: EACCES (Permission denied)" on Android 10+, use android:requestLegacyExternalStorage="true" on manifest and turnOff/turnOn "write_external_storage" permission on phone settings. Better use Content Uri on Android 10+ */
        return try {
            FileOutputStream(toFile())
        } catch (e: Throwable) {
            null
        }

    } else if (scheme.equals(ContentResolver.SCHEME_ANDROID_RESOURCE)) {
        // i think you can't write to asset inside apk
        return null

    } else {
        // content URI (MediaStore)
        if (authority == android.provider.MediaStore.AUTHORITY) {
            return try {
                context.contentResolver.openOutputStream(this, "w")
            } catch (e: Throwable) {
                null
            }
        } else {
            // content URI (provider), not tested
            return try {
                val assetFileDescriptor = createAssetFileDescriptor()
                if (assetFileDescriptor != null) {
                    AssetFileDescriptor.AutoCloseOutputStream(assetFileDescriptor)
                } else {
                    val parcelFileDescriptor = createParcelFileDescriptor()
                    if (parcelFileDescriptor != null) {
                        ParcelFileDescriptor.AutoCloseOutputStream(parcelFileDescriptor)
                    } else {
                        null
                    }
                }
            } catch (e: Throwable) {
                null
            }
        }
    }
}

Once you have OutputStream, rest is similar to other answers. More about sink/source and emit/flush:

// create Request
val request = Request.Builder()
            .method("GET", null)
            .url(url)
            .build()

// API call function
fun apiCall(): Response? {
    return try {
        client.newCall(request).execute()
    } catch (error: Throwable) {
        null
    }
}

// execute API call
var response: Response? = apiCall()
// your retry logic if request failed (response==null)

// if failed, return
if (response == null || !response.isSuccessful) {
    return
}

// response.body
val body: ResponseBody? = response!!.body
if (body == null) {
    response.closeQuietly()
    return
}

// outputStream
val outputStream = destinationUri.toOutputStream(appContext)
if (outputStream == null) {
    response.closeQuietly() // calls body.close
    return
}
val bufferedSink: BufferedSink = outputStream!!.sink().buffer()
val outputBuffer: Buffer = bufferedSink.buffer

// inputStream
val bufferedSource = body!!.source()
val contentLength = body.contentLength()

// write
var totalBytesRead: Long = 0
var toBeFlushedBytesRead: Long = 0
val BUFFER_SIZE = 8 * 1024L // KB
val FLUSH_THRESHOLD = 200 * 1024L // KB
var bytesRead: Long = bufferedSource.read(outputBuffer, BUFFER_SIZE)
var lastProgress: Int = -1
while (bytesRead != -1L) {
    // emit/flush
    totalBytesRead += bytesRead
    toBeFlushedBytesRead += bytesRead
    bufferedSink.emitCompleteSegments()
    if (toBeFlushedBytesRead >= FLUSH_THRESHOLD) {
        toBeFlushedBytesRead = 0L
        bufferedSink.flush()
    }

    // write
    bytesRead = bufferedSource.read(outputBuffer, BUFFER_SIZE)

    // progress
    if (contentLength != -1L) {
        val progress = (totalBytesRead * 100 / contentLength).toInt()
        if (progress != lastProgress) {
            lastProgress = progress
            // update UI (using Flow/Observable/Callback)
        }
    }
}

bufferedSink.flush()

// close resources
outputStream.closeQuietly()
bufferedSink.closeQuietly()
bufferedSource.closeQuietly()
body.closeQuietly()
Lenity answered 25/4, 2021 at 10:40 Comment(0)
S
1

I append my solution here:

fun downloadFile(url: String, file: File) : Completable {
    val request = Request.Builder().url(url).build()
    return Completable.fromPublisher<Boolean> { p ->
        OkHttpClient.Builder().build().newCall(request).enqueue(object: Callback {
            override fun onFailure(call: Call, e: IOException) {
                p.onError(e)
            }

            override fun onResponse(call: Call, response: Response) {
                val sink = Okio.buffer(Okio.sink(file))
                sink.writeAll(response.body()!!.source())
                sink.close()
                p.onComplete()
            }

        })
    }
}

Rebuild the client is important if you use various middlewares or you encoding the data in json, or find a solution that not encode the response in other format.

Svetlana answered 28/2, 2022 at 9:42 Comment(0)
H
0
    val request = Request.Builder().url(URL).build()
    val response = OkHttpClient().newCall(request).execute()

    val `is`: InputStream = response.body!!.byteStream()

    val PATH = (Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS)
        .toString() + "/Folder")
    val file = File(PATH)

    if (!file.exists()) file.mkdir()  
  
        val outputFile = File(file, getFileName(URL))

        val input = BufferedInputStream(`is`)
        val output: OutputStream = FileOutputStream(outputFile)

        val data = ByteArray(1024)

        var total: Long = 0

         while (`is`.read(data).also { total = it.toLong() } != -1) {
             output.write(data, 0, total.toInt())
         }

        output.flush()
        output.close()
        input.close()
Hirsute answered 15/3, 2022 at 11:48 Comment(1)
As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.Leanoraleant

© 2022 - 2024 — McMap. All rights reserved.