How can I download and save a file from the Internet using Java?

Question

There is an online file (such as http://www.example.com/information.asp) I need to grab and save to a directory. I know there are several methods for grabbing and reading online files (URLs) line-by-line, but is there a way to just download and save the file using Java?

Nodular

related #8325362 – Helbona

docs.oracle.com/javase/7/docs/api/java/nio/file/… – Cabernet

Interesting article: baeldung.com/java-download-file – Gesualdo

Answer 1

Give Java NIO a try:

URL website = new URL("http://www.website.com/information.asp");
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream("information.html");
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);

Using transferFrom() is potentially much more efficient than a simple loop that reads from the source channel and writes to this channel. Many operating systems can transfer bytes directly from the source channel into the filesystem cache without actually copying them.

Check more about it here.

Note: The third parameter in transferFrom is the maximum number of bytes to transfer. Integer.MAX_VALUE will transfer at most 2^31 bytes, Long.MAX_VALUE will allow at most 2^63 bytes (larger than any file in existence).

Abound

hey what is that shifty bit? – Powder

@willcodejavaforfood: it is just an alias for 16777216. An arbitrary long block size. A magic costant (beware it!) – Abound

@will, the shift is just another way of representing 2^24 (the amount he is reading from the channel) – Gavage

What if that file is bigger than 16777216 bytes? – Kelt

sorry to sound like a total noob, but should i be closing fos? regardless of that, thanks, it works perfectly! – Nodular

Would it still be wise to close the FileOutputStream? Or does the transferFrom method do something different that it isn't required anymore? – Openhearth

1 << 24 just seems so arbitrary, would something like Long.MAX_VALUE make more sense? – Beal

Close all three with Java 7 try-with-resource: try (InputStream inputStream = website.openStream(); ReadableByteChannel readableByteChannel = Channels.newChannel(inputStream); FileOutputStream fileOutputStream = new FileOutputStream(outputFileName)) { fileOutputStream.getChannel().transferFrom(readableByteChannel, 0, 1 << 24); } – Staphyloplasty

This will only download the first 16MB of a file: https://mcmap.net/q/42277/-downloading-files-with-java – Panic

Just use Integer.MAX_VALUE if your file can be bigger than 16 MB (or even Long.MAX_VALUE if it can be bigger than 2 GB). – Melamie

@kirdie and if I want more than 8388608 TB? – Hellion

Is there any way to track the progress of the download with this method? – Originate

A single call isn't adequate. transferFrom() isnt' specified to complete the entire transfer in a single call. That's why it returns a count. You have to loop. – Barb

@dfa, would you mind including also the slighly more modern solution mentioned here? – Distillery

Why was this answer even accepted? URL::openStream()returns just a regular stream, meaning the entire traffic is still being copied through Java byte[] arrays instead of remaining in native buffers. Only fos.getChannel()is actually a native channel, so the overhead remains in full. That's zero gains from using NIO in this case. Apart from being broken, as EJP and Ben MacCann correctly noticed. – Worden

If you're using NIO, be consistent and also use FileChannel.open(Paths.get("information.html"), StandardOpenOption.CREATE, StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.WRITE) instead of FileOutputStream – Waltner

@Waltner that's an NIO/2 addition of Java 7, which came out two years after this question was originally answered. That being said, it might make sense to update the answer with the new method. – Cyclograph

I tried transferFrom in android. With Long.MAX_VALUE give an "invalid parameter" value. Then I changed it in Integer.MAX_VALUE and it gives an OutOfMemory due to trying to allocate 2^31 bytes. So it does not work for every java.nio implementations. – Bunni

@EJP How can this use of transferFrom() fail to complete the entire transfer (ignoring exceptions)? The stream, opened from the url, blocks its read methods until input is available so the transferFrom() method only stops if its source channel has fewer than the requested number of bytes remaining. – Forfar

How to set encoding utf-8 – Cuneo

docs.oracle.com/javase/7/docs/api/java/nio/file/… – Cabernet

that's for normal http get parameter.... what about combining with post for downloading a same call? – Idyll

@Kröw Because there is nothing in the Javadoc that says it will transfer the entire stream. That's why it returns a count. You have to loop. Blocking has nothing to do with it. – Barb

@Barb "Blocking has nothing to do with it" The documentation for transferFrom() points out the cases in which the call can fail, and describes them with regard to whether or not the source stream is blocking. So no, blocking has everything to do with it. – Forfar

@Kröw Those sections in the JavaDoc are specific examples of how such a phenomenon might occur. It is not an exhaustive list of cases. – Cyclograph

Answer 2

Use Apache Commons IO. It is just one line of code:

FileUtils.copyURLToFile(URL, File)

Cienfuegos

Nice! Just what I'm looking for! I knew Apache libraries would already cover this. BTW, it's recommended to use the overloaded version with timeout parameters! – Cementum

...and when using that overloaded version, remember that the timeouts are specified in milliseconds, not seconds. – Colemancolemanite

Take note that copyURLToFile with timeout parameter is only available since version 2.0 of Commons IO library. See Java docs – Disoperation

@卢声远 how can we add cookies to the request? – Adobe

"The method copyURLToFile(URL, File) in the type FileUtils is not applicable for the arguments (String, String)" – Hills

Agreed. Mark this as the accepted answer. Why re-invent the wheel? – Forbiddance

what if basic authentication header has to be added to the request? is there a workaround? – Containerize

Although this is "short", it's actually very slow. – Evenings

A native Java solution is better than an external library, IMO. – Scharf

so in Java 8, this is not a thing anymore? Link is dad, and Eclipse doesn't know FileUtils. – Excess

@Blauhirn commons-io is an external library, available from the Apache Commons website. I have just fixed the broken link to the Javadoc in the answer. – Aleksandropol

How can I use this method, when I'm behind a proxy? – Hinton

Answer 3

Simpler non-blocking I/O usage:

URL website = new URL("http://www.website.com/information.asp");
try (InputStream in = website.openStream()) {
    Files.copy(in, target, StandardCopyOption.REPLACE_EXISTING);
}

Corky

Unfortunately this silently fails (downloads 0 bytes) in case there is a redirect such as "302 Found". – Plum

@AlexanderK But why would you blindly download such a resource anyway? – Corky

Despite the fact this is an elegant solution, behind the scenes this approach could silently betray you. Files.copy( InputStream, Paths, FileOption) delegates the copy process to the Files.copy( InputStream, OutputStream ). This last method does not check for the end of stream (-1) but checks for no byte read (0). It means that, if your network had a little pause, it could read 0 bytes and end the copy process, even if the stream isn't finished to be downloaded by the OS. – Engenia

@Engenia It is impossible for InputStream.read() to return zero unless you provided a zero length buffer or count, 'little pause' or otherwise. It will block until at least one byte has been transferred or end of stream or an error occurs. Your claim about the internals of Files.copy() is baseless. – Barb

I have an unit test that reads a binary file with 2.6TiB. Using Files.copy it always fails on my HDD storage server (XFS) but it fails only a few times my SSH one. Looking at JDK 8 the code of File.copy I've identified that it checks for '> 0' to leave the 'while' loop. I just copied the exactly same code with the -1 and both unit tests never stopped again. Once InputStream can represent Network and local file descriptors, and both IO operations are subject to OS context switching, I cant see why my claim is baseless. One may claim it be working by luck, but it gave no headaches any more. – Engenia

Is there a way to discover the extension before downloading? – Spireme

@CardinalSystem I believe you can Path path = Paths.get("https://docs.oracle.com/javase/7/docs/api/java/nio/file/Path.html"); String fileName = path.getFileName().toString(); String ext = fileName.substring(fileName.lastIndexOf("."), fileName.length()); But there are libraries out there that will do the job (Apache Commons/Guava) – Corky

@CardinalSystem Of course you cannot guarantee that extension will really correspond to its binary contents:) So, perhaps, you'll need to double-check after downloading – Corky

What would be path where file is stored? – Dijon

It will save it to the target path – Corky

Answer 4

public void saveUrl(final String filename, final String urlString)
        throws MalformedURLException, IOException {
    BufferedInputStream in = null;
    FileOutputStream fout = null;
    try {
        in = new BufferedInputStream(new URL(urlString).openStream());
        fout = new FileOutputStream(filename);

        final byte data[] = new byte[1024];
        int count;
        while ((count = in.read(data, 0, 1024)) != -1) {
            fout.write(data, 0, count);
        }
    } finally {
        if (in != null) {
            in.close();
        }
        if (fout != null) {
            fout.close();
        }
    }
}

You'll need to handle exceptions, probably external to this method.

Visualize

How to download very faster? Like download accelerator? – Thoracoplasty

If in.close throws an exception, fout.close is not called. – Cantus

@ComFreek That is simply untrue. Using a BufferedInputStream has precisely zero effect on socket timeouts. I had already refuted that as 'urban myth' in my comments to the 'background details' you cited. Three years earlier. – Barb

@EJP Thank you for the correction! I removed my comment (for the archive: I linked to this answer stating that BufferedInputStream "can cause unpredictable failures"). – Girardi

+1 My only objection to this answer (and others here) is that the caller cannot distinguish the event "not found" from some connection error (upon which you might want to retry). – Neldanelia

@Neldanelia That's not correct. 'Not found' becomes a FileNotFoundException. Nothing else does. Ergo the caller can distinguish. – Barb

MalformedURLException extends IOException, so you only have to throw the IOException – Supergalaxy

Answer 5

Here is a concise, readable, JDK-only solution with properly closed resources:

static long download(String url, String fileName) throws IOException {
    try (InputStream in = URI.create(url).toURL().openStream()) {
        return Files.copy(in, Paths.get(fileName));
    }
}

Two lines of code and no dependencies.

Here's a complete file downloader example program with output, error checking, and command line argument checks:

package so.downloader;

import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Paths;

public class Application {
    public static void main(String[] args) throws IOException {
        if (2 != args.length) {
            System.out.println("USAGE: java -jar so-downloader.jar <source-URL> <target-filename>");
            System.exit(1);
        }

        String sourceUrl = args[0];
        String targetFilename = args[1];

        long bytesDownloaded = download(sourceUrl, targetFilename);

        System.out.println(String.format("Downloaded %d bytes from %s to %s.", bytesDownloaded, sourceUrl, targetFilename));
    }

    static long download(String url, String fileName) throws IOException {
        try (InputStream in = URI.create(url).toURL().openStream()) {
            return Files.copy(in, Paths.get(fileName));
        }
    }    
}

As noted in the so-downloader repository README:

To run file download program:

java -jar so-downloader.jar <source-URL> <target-filename>

For example:

java -jar so-downloader.jar https://github.com/JanStureNielsen/so-downloader/archive/main.zip so-downloader-source.zip

Capablanca

this works for me thanks. I have URL to PDF file and it downloads the exact PDF without using any library for PDF. I appreciate it very much – Tailorbird

Answer 6

Downloading a file requires you to read it. Either way, you will have to go through the file in some way. Instead of line by line, you can just read it by bytes from the stream:

BufferedInputStream in = new BufferedInputStream(new URL("http://www.website.com/information.asp").openStream())
byte data[] = new byte[1024];
int count;
while((count = in.read(data, 0, 1024)) != -1)
{
    out.write(data, 0, count);
}

Gavage

Answer 7

When using Java 7+, use the following method to download a file from the Internet and save it to some directory:

private static Path download(String sourceURL, String targetDirectory) throws IOException
{
    URL url = new URL(sourceURL);
    String fileName = sourceURL.substring(sourceURL.lastIndexOf('/') + 1, sourceURL.length());
    Path targetPath = new File(targetDirectory + File.separator + fileName).toPath();
    Files.copy(url.openStream(), targetPath, StandardCopyOption.REPLACE_EXISTING);

    return targetPath;
}

Documentation is here.

Violaceous

I wish there was a StandardCopyOption to rename the file instead of replacing existing. – Lindeman

Answer 8

This answer is almost exactly like the selected answer, but with two enhancements: it's a method and it closes out the FileOutputStream object:

    public static void downloadFileFromURL(String urlString, File destination) {
        try {
            URL website = new URL(urlString);
            ReadableByteChannel rbc;
            rbc = Channels.newChannel(website.openStream());
            FileOutputStream fos = new FileOutputStream(destination);
            fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
            fos.close();
            rbc.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

Aer

A single call isn't adequate. transferFrom() isnt' specified to complete the entire transfer in a single call. That's why it returns a count. You have to loop. – Barb

And your code doesn't close anything if there is an exception. – Barb

Answer 9

import java.io.*;
import java.net.*;

public class filedown {
    public static void download(String address, String localFileName) {
        OutputStream out = null;
        URLConnection conn = null;
        InputStream in = null;

        try {
            URL url = new URL(address);
            out = new BufferedOutputStream(new FileOutputStream(localFileName));
            conn = url.openConnection();
            in = conn.getInputStream();
            byte[] buffer = new byte[1024];

            int numRead;
            long numWritten = 0;

            while ((numRead = in.read(buffer)) != -1) {
                out.write(buffer, 0, numRead);
                numWritten += numRead;
            }

            System.out.println(localFileName + "\t" + numWritten);
        } 
        catch (Exception exception) { 
            exception.printStackTrace();
        } 
        finally {
            try {
                if (in != null) {
                    in.close();
                }
                if (out != null) {
                    out.close();
                }
            } 
            catch (IOException ioe) {
            }
        }
    }

    public static void download(String address) {
        int lastSlashIndex = address.lastIndexOf('/');
        if (lastSlashIndex >= 0 &&
        lastSlashIndex < address.length() - 1) {
            download(address, (new URL(address)).getFile());
        } 
        else {
            System.err.println("Could not figure out local file name for "+address);
        }
    }

    public static void main(String[] args) {
        for (int i = 0; i < args.length; i++) {
            download(args[i]);
        }
    }
}

Eclogite

If in.close throws an exception, out.close is not called. – Cantus

Answer 10

Personally, I've found Apache's HttpClient to be more than capable of everything I've needed to do with regards to this. Here is a great tutorial on using HttpClient

Peon

also commons-io is a great library – Abound

Answer 11

This is another Java 7 variant based on Brian Risk's answer with usage of a try-with statement:

public static void downloadFileFromURL(String urlString, File destination) throws Throwable {

    URL website = new URL(urlString);
    try(
        ReadableByteChannel rbc = Channels.newChannel(website.openStream());
        FileOutputStream fos = new FileOutputStream(destination);
       ) {

        fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
    }
}

Metallo

A single call isn't adequate. transferFrom() isnt' specified to complete the entire transfer in a single call. That's why it returns a count. You have to loop. – Barb

I don't know why you're addressing that stupid question to me. It doesn't have anything to do with what I said, and I really decline to have words put into my mouth. – Barb

Answer 12

One liner with the built-in Java HTTP Client added in Java 11:

URI url = ...;
Path path = ...; // output file path
HttpClient.newHttpClient().send(HttpRequest.newBuilder(url).build(), HttpResponse.BodyHandlers.ofFile(path));

If you're making multiple requests, you can reuse the HttpClient and the body handler.

You can also customize the request by adding parameters to the HttpRequest and customize writing/creating the file by adding parameters to the HttpResponse.BodyHandlers.ofFile method.

No need to worry about closing resources.

Didier

Answer 13

There are many elegant and efficient answers here. But the conciseness can make us lose some useful information. In particular, one often does not want to consider a connection error an Exception, and one might want to treat differently some kind of network-related errors - for example, to decide if we should retry the download.

Here's a method that does not throw Exceptions for network errors (only for truly exceptional problems, as malformed url or problems writing to the file)

/**
 * Downloads from a (http/https) URL and saves to a file. 
 * Does not consider a connection error an Exception. Instead it returns:
 *  
 *    0=ok  
 *    1=connection interrupted, timeout (but something was read)
 *    2=not found (FileNotFoundException) (404) 
 *    3=server error (500...) 
 *    4=could not connect: connection timeout (no internet?) java.net.SocketTimeoutException
 *    5=could not connect: (server down?) java.net.ConnectException
 *    6=could not resolve host (bad host, or no internet - no dns)
 * 
 * @param file File to write. Parent directory will be created if necessary
 * @param url  http/https url to connect
 * @param secsConnectTimeout Seconds to wait for connection establishment
 * @param secsReadTimeout Read timeout in seconds - trasmission will abort if it freezes more than this 
 * @return See above
 * @throws IOException Only if URL is malformed or if could not create the file
 */
public static int saveUrl(final Path file, final URL url, 
  int secsConnectTimeout, int secsReadTimeout) throws IOException {
    Files.createDirectories(file.getParent()); // make sure parent dir exists , this can throw exception
    URLConnection conn = url.openConnection(); // can throw exception if bad url
    if( secsConnectTimeout > 0 ) conn.setConnectTimeout(secsConnectTimeout * 1000);
    if( secsReadTimeout > 0 ) conn.setReadTimeout(secsReadTimeout * 1000);
    int ret = 0;
    boolean somethingRead = false;
    try (InputStream is = conn.getInputStream()) {
        try (BufferedInputStream in = new BufferedInputStream(is); OutputStream fout = Files
                .newOutputStream(file)) {
            final byte data[] = new byte[8192];
            int count;
            while((count = in.read(data)) > 0) {
                somethingRead = true;
                fout.write(data, 0, count);
            }
        }
    } catch(java.io.IOException e) { 
        int httpcode = 999;
        try {
            httpcode = ((HttpURLConnection) conn).getResponseCode();
        } catch(Exception ee) {}
        if( somethingRead && e instanceof java.net.SocketTimeoutException ) ret = 1;
        else if( e instanceof FileNotFoundException && httpcode >= 400 && httpcode < 500 ) ret = 2; 
        else if( httpcode >= 400 && httpcode < 600 ) ret = 3; 
        else if( e instanceof java.net.SocketTimeoutException ) ret = 4; 
        else if( e instanceof java.net.ConnectException ) ret = 5; 
        else if( e instanceof java.net.UnknownHostException ) ret = 6;  
        else throw e;
    }
    return ret;
}

Neldanelia

Answer 14

It's possible to download the file with with Apache's HttpComponents instead of Commons IO. This code allows you to download a file in Java according to its URL and save it at the specific destination.

public static boolean saveFile(URL fileURL, String fileSavePath) {

    boolean isSucceed = true;

    CloseableHttpClient httpClient = HttpClients.createDefault();

    HttpGet httpGet = new HttpGet(fileURL.toString());
    httpGet.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0");
    httpGet.addHeader("Referer", "https://www.google.com");

    try {
        CloseableHttpResponse httpResponse = httpClient.execute(httpGet);
        HttpEntity fileEntity = httpResponse.getEntity();

        if (fileEntity != null) {
            FileUtils.copyInputStreamToFile(fileEntity.getContent(), new File(fileSavePath));
        }

    } catch (IOException e) {
        isSucceed = false;
    }

    httpGet.releaseConnection();

    return isSucceed;
}

In contrast to the single line of code:

FileUtils.copyURLToFile(fileURL, new File(fileSavePath),
                        URLS_FETCH_TIMEOUT, URLS_FETCH_TIMEOUT);

This code will give you more control over a process and let you specify not only time-outs, but User-Agent and Referer values, which are critical for many websites.

Affirmative

Answer 15

To summarize (and somehow polish and update) previous answers. The three following methods are practically equivalent. (I added explicit timeouts, because I think they are a must. Nobody wants a download to freeze forever when the connection is lost.)

public static void saveUrl1(final Path file, final URL url,
    int secsConnectTimeout, int secsReadTimeout))
    throws MalformedURLException, IOException {

    // Files.createDirectories(file.getParent()); // Optional, make sure parent directory exists
    try (BufferedInputStream in = new BufferedInputStream(
         streamFromUrl(url, secsConnectTimeout,secsReadTimeout));
         OutputStream fout = Files.newOutputStream(file)) {

            final byte data[] = new byte[8192];
            int count;
            while((count = in.read(data)) > 0)
                fout.write(data, 0, count);
        }
}

public static void saveUrl2(final Path file, final URL url,
    int secsConnectTimeout, int secsReadTimeout))
    throws MalformedURLException, IOException {

    // Files.createDirectories(file.getParent()); // Optional, make sure parent directory exists
    try (ReadableByteChannel rbc = Channels.newChannel(
             streamFromUrl(url, secsConnectTimeout, secsReadTimeout)
        );
        FileChannel channel = FileChannel.open(file,
             StandardOpenOption.CREATE,
             StandardOpenOption.TRUNCATE_EXISTING,
             StandardOpenOption.WRITE)
        ) {

        channel.transferFrom(rbc, 0, Long.MAX_VALUE);
    }
}

public static void saveUrl3(final Path file, final URL url,
    int secsConnectTimeout, int secsReadTimeout))
    throws MalformedURLException, IOException {

    // Files.createDirectories(file.getParent()); // Optional, make sure parent directory exists
    try (InputStream in = streamFromUrl(url, secsConnectTimeout,secsReadTimeout) ) {
        Files.copy(in, file, StandardCopyOption.REPLACE_EXISTING);
    }
}

public static InputStream streamFromUrl(URL url,int secsConnectTimeout,int secsReadTimeout) throws IOException {
    URLConnection conn = url.openConnection();
    if(secsConnectTimeout>0)
        conn.setConnectTimeout(secsConnectTimeout*1000);
    if(secsReadTimeout>0)
        conn.setReadTimeout(secsReadTimeout*1000);
    return conn.getInputStream();
}

I don't find significant differences, and all seem right to me. They are safe and efficient. (Differences in speed seem hardly relevant - I write 180 MB from the local server to a SSD disk in times that fluctuate around 1.2 to 1.5 secs). They don't require external libraries. All work with arbitrary sizes and (to my experience) HTTP redirections.

Additionally, all throw FileNotFoundException if the resource is not found (error 404, typically), and java.net.UnknownHostException if the DNS resolution failed; other IOException correspond to errors during transmission.

Neldanelia

Answer 16

There is a method, U.fetch(url), in the underscore-java library.

File pom.xml:

<dependency>
  <groupId>com.github.javadev</groupId>
  <artifactId>underscore</artifactId>
  <version>1.84</version>
</dependency>

Code example:

import com.github.underscore.U;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;

public class Download {
    public static void main(String[] args) throws IOException {
        Files.write(Paths.get("data.bin"),
            U.fetch("https://stackoverflow.com/questions"
                + "/921262/how-to-download-and-save-a-file-from-internet-using-java").blob());
    }
}

Rimbaud

how useful is this answer, when the link becomes invalid? Please look at How to Answer – Margarite

You code won't compiles. Question ask for solution in Java, but your answer look like JavaScript – Aho

@Aho I added pom.xml section and improved code example. – Rimbaud

The question is about saving data to a file in a directory, not reading it all into a string, which is invalid in the case of binary data. – Barb

@Barb I added example with saving binary file. Files.write(Paths.get("data.bin"), U.fetch(url).blob()). – Rimbaud

What is U here? – Petigny

Answer 17

Below is the sample code to download a movie from the Internet with Java code:

URL url = new
URL("http://103.66.178.220/ftp/HDD2/Hindi%20Movies/2018/Hichki%202018.mkv");
    BufferedInputStream bufferedInputStream = new  BufferedInputStream(url.openStream());
    FileOutputStream stream = new FileOutputStream("/home/sachin/Desktop/test.mkv");

    int count = 0;
    byte[] b1 = new byte[100];

    while((count = bufferedInputStream.read(b1)) != -1) {
        System.out.println("b1:" + b1 + ">>" + count + ">> KB downloaded:" + new File("/home/sachin/Desktop/test.mkv").length()/1024);
        stream.write(b1, 0, count);
    }

Drucie

Generally, answers are much more helpful if they include an explanation of what the code is intended to do, and why that solves the problem without introducing others. – Maclaine

This code never closes anything, and uses a ridiculously small buffer, – Barb

Answer 18

Solution on java.net.http.HttpClient using Authorization:

HttpClient client = HttpClient.newHttpClient();

HttpRequest request = HttpRequest.newBuilder()
        .GET()
        .header("Accept", "application/json")
        // .header("Authorization", "Basic ci5raG9kemhhZXY6NDdiYdfjlmNUM=") if you need
        .uri(URI.create("https://jira.google.ru/secure/attachment/234096/screenshot-1.png"))
        .build();

HttpResponse<InputStream> response = client.send(request, HttpResponse.BodyHandlers.ofInputStream());

try (InputStream in = response.body()) {
    Files.copy(in, Paths.get(target + "screenshot-1.png"), StandardCopyOption.REPLACE_EXISTING);
}

Pharmacology

"Authorization" is commented out. What is its role (in light of the first sentence's "...using Authorization")? – Jewess

This doesn't even compile (also indicated by the syntax highligting) - near "screenshot-1.png"". – Jewess

@PeterMortensen If resource is free, you don't need Authorization. Syntax error has been fixed. – Pharmacology

Answer 19

There is an issue with simple usage of:

org.apache.commons.io.FileUtils.copyURLToFile(URL, File)

if you need to download and save very large files, or in general if you need automatic retries in case connection is dropped.

I suggest Apache HttpClient in such cases, along with org.apache.commons.io.FileUtils. For example:

GetMethod method = new GetMethod(resource_url);
try {
    int statusCode = client.executeMethod(method);
    if (statusCode != HttpStatus.SC_OK) {
        logger.error("Get method failed: " + method.getStatusLine());
    }
    org.apache.commons.io.FileUtils.copyInputStreamToFile(
        method.getResponseBodyAsStream(), new File(resource_file));
    } catch (HttpException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
    method.releaseConnection();
}

Osric

This code doesn't handle retries any better than any other answer or comment here: that is to say, it doesn't do them at all. – Barb

Answer 20

You can do this in one line using netloader for Java:

new NetFile(new File("my/zips/1.zip"), "https://example.com/example.zip", -1).load(); // Returns true if succeed, otherwise false.

Longhair

Answer 21

This can read a file on the Internet and write it into a file.

import java.net.URL;
import java.io.FileOutputStream;
import java.io.File;

public class Download {
    public static void main(String[] args) throws Exception {
         URL url = new URL("https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png");  // Input URL
         FileOutputStream out = new FileOutputStream(new File("out.png"));  // Output file
         out.write(url.openStream().readAllBytes());
         out.close();
    }
}

Ricardoricca

Answer 22

If you are behind a proxy, you can set the proxies in the Java program as below:

Properties systemSettings = System.getProperties();
systemSettings.put("proxySet", "true");
systemSettings.put("https.proxyHost", "HTTPS proxy of your org");
systemSettings.put("https.proxyPort", "8080");

If you are not behind a proxy, don't include the lines above in your code. Full working code to download a file when you are behind a proxy.

public static void main(String[] args) throws IOException {
    String url = "https://raw.githubusercontent.com/bpjoshi/fxservice/master/src/test/java/com/bpjoshi/fxservice/api/TradeControllerTest.java";
    OutputStream outStream = null;
    URLConnection connection = null;
    InputStream is = null;
    File targetFile = null;
    URL server = null;

    // Setting up proxies
    Properties systemSettings = System.getProperties();
        systemSettings.put("proxySet", "true");
        systemSettings.put("https.proxyHost", "HTTPS proxy of my organisation");
        systemSettings.put("https.proxyPort", "8080");
        // The same way we could also set proxy for HTTP
        System.setProperty("java.net.useSystemProxies", "true");
        // Code to fetch file
    try {
        server = new URL(url);
        connection = server.openConnection();
        is = connection.getInputStream();
        byte[] buffer = new byte[is.available()];
        is.read(buffer);

        targetFile = new File("src/main/resources/targetFile.java");
        outStream = new FileOutputStream(targetFile);
        outStream.write(buffer);
    } catch (MalformedURLException e) {
        System.out.println("THE URL IS NOT CORRECT ");
        e.printStackTrace();
    } catch (IOException e) {
        System.out.println("I/O exception");
        e.printStackTrace();
    }
    finally{
        if(outStream != null)
            outStream.close();
    }
}

Lucas

systemSettings.put("proxySet", "true"); is an urban myth deriving from the HotJava bean that became defunct in 1998. In any Sun or Oracle JDK it does exactly nothing. Proof: set it to false in any situation when you need the other proxy settings and watch it continue to work. – Barb

hmm interesting, gonna check more about it. thanks – Lucas

Answer 23

First method using the new channel

ReadableByteChannel aq = Channels.newChannel(new url("https//asd/abc.txt").openStream());
FileOutputStream fileOS = new FileOutputStream("C:Users/local/abc.txt")
FileChannel writech = fileOS.getChannel();

Second method using FileUtils

FileUtils.copyURLToFile(new url("https//asd/abc.txt", new local file on system("C":/Users/system/abc.txt"));

Third method using

InputStream xy = new ("https//asd/abc.txt").openStream();

This is how we can download file by using basic Java code and other third-party libraries. These are just for quick reference. Please google with the above keywords to get detailed information and other options.

Argeliaargent

3rd method does not compile. Neither the 1st nor the 3rd woudl actually copy anything. – Barb

Answer 24

public class DownloadManager {

    static String urls = "[WEBSITE NAME]";

    public static void main(String[] args) throws IOException{
        URL url = verify(urls);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        InputStream in = null;
        String filename = url.getFile();
        filename = filename.substring(filename.lastIndexOf('/') + 1);
        FileOutputStream out = new FileOutputStream("C:\\Java2_programiranje/Network/DownloadTest1/Project/Output" + File.separator + filename);
        in = connection.getInputStream();
        int read = -1;
        byte[] buffer = new byte[4096];
        while((read = in.read(buffer)) != -1){
            out.write(buffer, 0, read);
            System.out.println("[SYSTEM/INFO]: Downloading file...");
        }
        in.close();
        out.close();
        System.out.println("[SYSTEM/INFO]: File Downloaded!");
    }
    private static URL verify(String url){
        if(!url.toLowerCase().startsWith("http://")) {
            return null;
        }
        URL verifyUrl = null;

        try{
            verifyUrl = new URL(url);
        }catch(Exception e){
            e.printStackTrace();
        }
        return verifyUrl;
    }
}

Burnham

You can improve your answer by providing information how does your code work instead of just dumping it. – Ministrant

and by fixing it to close resources if there is an exception. – Barb