Resume http file download in java
Asked Answered
K

8

61
URL url = new URL("http://download.thinkbroadband.com/20MB.zip");

URLConnection connection = url.openConnection();
File fileThatExists = new File(path); 
OutputStream output = new FileOutputStream(path, true);
connection.setRequestProperty("Range", "bytes=" + fileThatExists.length() + "-");

connection.connect();

int lenghtOfFile = connection.getContentLength();

InputStream input = new BufferedInputStream(url.openStream());
byte data[] = new byte[1024];

long total = 0;

while ((count = input.read(data)) != -1) {
    total += count;

    output.write(data, 0 , count);
}

in this code I try to resume download. Target file is 20MB. But when I stop download on 10mb, then contunue, I get file with filesize 30MB. It seems that it continue writing to file, but cant partly download from server. Wget -c works great with this file. How can I resume file download?

Kathrinkathrine answered 4/6, 2011 at 13:45 Comment(1)
possible duplicate of how to resume an interrupted download - part 2Innermost
K
61
 HttpURLConnection connection = (HttpURLConnection) url.openConnection();
    if(ISSUE_DOWNLOAD_STATUS.intValue()==ECMConstant.ECM_DOWNLOADING){
        File file=new File(DESTINATION_PATH);
        if(file.exists()){
             downloaded = (int) file.length();
             connection.setRequestProperty("Range", "bytes="+(file.length())+"-");
        }
    }else{
        connection.setRequestProperty("Range", "bytes=" + downloaded + "-");
    }
    connection.setDoInput(true);
    connection.setDoOutput(true);
    progressBar.setMax(connection.getContentLength());
     in = new BufferedInputStream(connection.getInputStream());
     fos=(downloaded==0)? new FileOutputStream(DESTINATION_PATH): new FileOutputStream(DESTINATION_PATH,true);
     bout = new BufferedOutputStream(fos, 1024);
    byte[] data = new byte[1024];
    int x = 0;
    while ((x = in.read(data, 0, 1024)) >= 0) {
        bout.write(data, 0, x);
         downloaded += x;
         progressBar.setProgress(downloaded);
    }

This is not my code, but it works.

Kathrinkathrine answered 12/6, 2011 at 16:21 Comment(10)
Hello , 'ISSUE_DOWNLOAD_STATUS.intValue() ' i am not understand about this value please give me the reference for this.Crispa
I anticipate that this code will not work well. If the condition of if(ISSUE_DOWNLOAD_STATUS.intValue()==ECMConstant.ECM_DOWNLOADING) fails, than the else{} statement will be executed where "downloaded" variable is undefined.Multinuclear
@Multinuclear Let's assume we all can code a little bit, this code gives you a hint on how to solve the problem not a total fix.Fultz
Hey there @Ceetn, if you can't give any constructive help, then zip it. Most people here try hard to to provide functional solutions. This one looks like it originated from somewhere where it might be working, but there are missing variable declarations. I tried to used it, and I pretty much spent several minutes trying to understand it. What are ISSUE_DOWNLOAD_STATUS & ECMConstant? It may be clear for you. But you are totally wrong to assume that people can code when they are seeking help here. You would have reacted differently if you teachers had acted like that in your programming classes.Multinuclear
@Multinuclear The condition in "if" statement just checks the state of the particular file (if it was paused before, and resume download required). This is a custom condition, so you should replace it with your own logic.Hendiadys
This snippet gives us the perfect idea how to go about resuming the downloads with few tweaks. Kudos!!Desulphurize
@Kathrinkathrine you just pasted the code and ran away.. pls explain lil bit broFaction
@Kathrinkathrine i get NotFoundException for valid url and download linksBritain
I don't get how this answer has more upvotes and also accepted. Lot of not explained things, outside references. Constants does not have value. downloaded property's value set in if branch, used in else branch. If and else branch is the same. Why setDoOutput is set to true? Any POST passed? Same buffer size used at 3 places, no constant used. No error handling at all.Snath
Please provide complete code if it is for resume then please mention pause function as well ...Thank youDamnation
S
19

I guess the problem you are facing is calling url.openStream() after url.openConnection().

url.openStream() is equivalent to url.openConnection().getInputStream(). Hence, you are requesting the url twice. Particularly the second time, it is not specifying the range property. Therefore download always starts at the beginning.

You should replace url.openStream() with connection.getInputStream().

Sultry answered 14/6, 2012 at 10:28 Comment(2)
Thanks man very much. You really saved me! I encountered a similar problem and your solution was perfect. Thank you!Bertram
I was searching for my problem for like an hour, and you sir, you made my day and solved it! Thanks a lot!Zoogeography
D
7

This is what I am using to download the file in chunk Updating the UI with progress.

/*
 * @param callback = To update the UI with appropriate action
 * @param fileName = Name of the file by which downloaded file will be saved.
 * @param downloadURL = File downloading URL
 * @param filePath = Path where file will be saved
 * @param object = Any object you want in return after download is completed to do certain operations like insert in DB or show toast
 */

public void startDownload(final IDownloadCallback callback, String fileName, String downloadURL, String filePath, Object object) {
    callback.onPreExecute(); // Callback to tell that the downloading is going to start
    int count = 0;
    File outputFile = null; // Path where file will be downloaded
    try {
        File file = new File(filePath);
        file.mkdirs();
        long range = 0;
        outputFile = new File(file, fileName);
        /**
         * Check whether the file exists or not
         * If file doesn't exists then create the new file and range will be zero.
         * But if file exists then get the length of file which will be the starting range,
         * from where the file will be downloaded
         */
        if (!outputFile.exists()) {
            outputFile.createNewFile();
            range = 0;
        } else {
            range = outputFile.length();
        }
        //Open the Connection
        URL url = new URL(downloadURL);
        URLConnection con = url.openConnection();
        // Set the range parameter in header and give the range from where you want to start the downloading
        con.setRequestProperty("Range", "bytes=" + range + "-");
        /**
         * The total length of file will be the total content length given by the server + range.
         * Example: Suppose you have a file whose size is 1MB and you had already downloaded 500KB of it.
         * Then you will pass in Header as "Range":"bytes=500000".
         * Now the con.getContentLength() will be 500KB and range will be 500KB.
         * So by adding the two you will get the total length of file which will be 1 MB
         */
        final long lenghtOfFile = (int) con.getContentLength() + range;

        FileOutputStream fileOutputStream = new FileOutputStream(outputFile, true);
        InputStream inputStream = con.getInputStream();

        byte[] buffer = new byte[1024];

        long total = range;
        /**
         * Download the save the content into file
         */
        while ((count = inputStream.read(buffer)) != -1) {
            total += count;
            int progress = (int) (total * 100 / lenghtOfFile);
            EntityDownloadProgress entityDownloadProgress = new EntityDownloadProgress();
            entityDownloadProgress.setProgress(progress);
            entityDownloadProgress.setDownloadedSize(total);
            entityDownloadProgress.setFileSize(lenghtOfFile);
            callback.showProgress(entityDownloadProgress);
            fileOutputStream.write(buffer, 0, count);
        }
        //Close the outputstream
        fileOutputStream.close();
        // Disconnect the Connection
        if (con instanceof HttpsURLConnection) {
            ((HttpsURLConnection) con).disconnect();
        } else if (con instanceof HttpURLConnection) {
            ((HttpURLConnection) con).disconnect();
        }
        inputStream.close();
        /**
         * If file size is equal then return callback as success with downlaoded filepath and the object
         * else return failure
         */
        if (lenghtOfFile == outputFile.length()) {
            callback.onSuccess(outputFile.getAbsolutePath(), object);
        } else {
            callback.onFailure(object);
        }
    } catch (Exception e) {
        e.printStackTrace();
        callback.onFailure(object);
    }
}

interface IDownloadCallback {

    void onPreExecute(); // Callback to tell that the downloading is going to start
    void onFailure(Object o); // Failed to download file
    void onSuccess(String path, Object o); // Downloaded file successfully with downloaded path
    void showProgress(EntityDownloadProgress entityDownloadProgress); // Show progress
}

public class EntityDownloadProgress {

    int progress; // range from 1-100
    long fileSize;// Total size of file to be downlaoded
    long downloadedSize; // Size of the downlaoded file

    public void setProgress(int progress) {this.progress = progress;}

    public void setFileSize(long fileSize) {this.fileSize = fileSize;}

    public void setDownloadedSize(long downloadedSize) {this.downloadedSize = downloadedSize;}
}
Demonography answered 5/6, 2018 at 5:21 Comment(1)
This should be the accepted answer as it perfectly takes care of progress bar changes.Ancon
O
3

Check out this thread which has a problem similar to yours. If wget is working, then the server clearly supports resuming downloads. It looks like you're not setting the If-Range header as mentioned in the accepted answer of the above link. ie. add:

// Initial download.
String lastModified = connection.getHeaderField("Last-Modified");

// ...

// Resume download.
connection.setRequestProperty("If-Range", lastModified); 
Olander answered 4/6, 2011 at 13:56 Comment(1)
With your code my app doesn't download anything. lastmodified string contains valid date (14 may 2008). What can it be?Kathrinkathrine
E
3

I have a way for your code to work.

  1. First, check if the file exits or not
  2. If the file exists, set the connection:

    connection.setRequestProperty("Range", "bytes=" + bytedownloaded + "-");
    
  3. If file does not exist, do the same download in a new file.

Extremist answered 12/11, 2014 at 13:37 Comment(0)
C
3

How about this?

public static void download(DownloadObject object) throws IOException{
    String downloadUrl = object.getDownloadUrl();
    String downloadPath = object.getDownloadPath();
    long downloadedLength = 0;

    File file = new File(downloadPath);
    URL url = new URL(downloadUrl);

    BufferedInputStream inputStream = null;
    BufferedOutputStream outputStream = null;

    URLConnection connection = url.openConnection();

    if(file.exists()){
        downloadedLength = file.length();
        connection.setRequestProperty("Range", "bytes=" + downloadedLength + "-");
        outputStream = new BufferedOutputStream(new FileOutputStream(file, true));

    }else{
        outputStream = new BufferedOutputStream(new FileOutputStream(file));

    }

    connection.connect();

    inputStream = new BufferedInputStream(connection.getInputStream());


    byte[] buffer = new byte[1024*8];
    int byteCount;

    while ((byteCount = inputStream.read(buffer)) != -1){
        outputStream.write(buffer, 0, byteCount);
        break;

    }

    inputStream.close();
    outputStream.flush();
    outputStream.close();

}

Used break; to test the code.. ;)

Crabber answered 26/8, 2016 at 18:37 Comment(3)
In addition to providing some code, please add some text explaining why your code worked.Weathertight
@Weathertight It is standard code to download files, except connect.setRequestProperty method and append property for FileOutputStream set to true.Halfpint
Check the request in the postman, you may need to add a check that the server responds in parts or in full. I had to add: connection.setRequestProperty("Accept-Encoding", "")Instanter
B
1

Since the question is tagged with Android: Have you tried using DownloadManager. It handles all this stuff nicely for you.

Bellicose answered 5/10, 2012 at 9:25 Comment(2)
Thanks, I'll switch to it, downloads should be much more stable now!Kathrinkathrine
Remember that it needs API level 9 (Android 2.3). I don't know if that's a problem. 2.2 is pretty much on the way out anyway.Bellicose
D
0

How I did it with OkHttp + Okio; Pause, Resume, Progress Update

    implementation("com.squareup.okhttp3:okhttp:4.12.0")

Saved totalBytes downloaded in variable: downloadedSize;. To publish download progress, save the totalSize of file with first request because the length() property of File can give inconsistent result. Also add tag to the request so you can cancel it to pause your file download, I used the file path.

        Request.Builder requestBuilder = new Request.Builder();
        if(isResume) requestBuilder.addHeader("Range","bytes="+downloadedSize+"-");
        okHttpClient.newCall(requestBuilder.url(downloadUrl).tag(filePath).build()).enqueue(new Callback() {
            @Override
            public void onFailure(@NonNull Call call, @NonNull IOException e) {e.printStackTrace();}
            @Override
            public void onResponse(@NonNull Call call, @NonNull Response response) {
                if(response.isSuccessful()){
                    BufferedSink sink=null; BufferedSource source=null;
                    try {
                        ResponseBody responseBody = response.body();assert responseBody != null;
                        if(responseBody.contentLength() != -1 && totalSize<=0)) totalSize = responseBody.contentLength(); 
                        source = responseBody.source(); 
                        File downloadFile = new File(filePath);
                        sink = Okio.buffer(downloadFile.exists() ? Okio.appendingSink(downloadFile) : Okio.sink(downloadFile)); //append if resume
                        int bufferSize = 8 * 1024;
                        for (long bytesRead; (bytesRead = source.read(sink.getBuffer(), bufferSize)) != -1; ) {
                            sink.emit(); downloadedSize+=bytesRead; publishProgress(downloadedSize*100/totalSize);
                        }
                        Log.d(TAG, "File successfully downloaded and saved to storage.");
                    }
                    catch (IOException e){e.printStackTrace();}
                    finally {
                        try {if(sink!=null) {sink.flush();sink.close();} if(source != null)source.close();}catch (IOException e){e.printStackTrace();}
                    }
                }
            }
        });

Note: if you want to update the UI, do it on main thread:

Handler handler = new Handler(Looper.getMainLooper());
handler.post(()-> {
   publishProgress(); //do UI stuff
});

To pause download, cancel the OkHttp call request using the saved tag:

    private void cancel(String tag){
        for(Call call: okHttpClient.dispatcher().queuedCalls()){
            if(tag.equals(call.request().tag())) call.cancel();
        }
        for(Call call: okHttpClient.dispatcher().runningCalls()){
            if(tag.equals(call.request().tag())) call.cancel();
        }
    }
Derose answered 21/5 at 21:10 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.