File Upload with Java (with progress bar)

Question

I'm extremely new to Java, and have mostly just been teaching myself as I go, so I've started building an applet. I'd like to make one that can select a file from the local disk and upload it as a multipart/form-data POST request but with a progress bar. Obviously the user has to grant permission to the Java applet to access the hard drive. Now I've already got the first part working: the user can select a file using a JFileChooser object, which conveniently returns a File object. But I'm wondering what comes next. I know that File.length() will give me the total size in bytes of the file, but how do I send the selected File to the web, and how do I monitor how many bytes have been sent? Thanks in advance.

Conscript

This is just a side comment, I hate it when companies use applets to provide a better upload interface and the one in html. I feel this is a web browser issue. You can't blindly trust applets that ask for permission, but most users give it anyways. Sad but true. – Cummerbund

@Cummerbund I agree, but unfortunately, the browser support for uploading files is still horrible. If you add in the need / desire to do any client-side processing on those files, you don't have any option other than to use an applet or flash. – Tonry

Answer 1

I ended up stumbling across an open source Java uploader applet and found everything I needed to know within its code. Here are links to a blog post describing it as well as the source:

Article
Source Code

Conscript

The Source Code link is dead. – Unguent

Not surprising as my question here is 12 years old. The landscape has changed dramatically since then, so I definitely wouldn't be using a Java applet to handle file uploads in 2020 if I were you. First, I think those are now generally blocked by major browsers anyway, and second, the functionality I was shooting for at the time is now fully supported in HTML5. No need to involve Java at all. What was necessary in 2008 is bad practice today. – Conscript

That's not what I meant, haha. It was just a comment over using links without the content on S.O. because of dead links, but I didn't made it clear in the first comment. But indeed I'm researching ways to do upload feedback, just not with some sort of gui like applets, swing, javafx or even HTML. Regards! – Unguent

Answer 2

To check progress using HttpClient, wrap the MultipartRequestEntity around one that counts the bytes being sent. Wrapper is below:

import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import org.apache.commons.httpclient.methods.RequestEntity;

public class CountingMultipartRequestEntity implements RequestEntity {
    private final RequestEntity delegate;

    private final ProgressListener listener;

    public CountingMultipartRequestEntity(final RequestEntity entity,
            final ProgressListener listener) {
        super();
        this.delegate = entity;
        this.listener = listener;
    }

    public long getContentLength() {
        return this.delegate.getContentLength();
    }

    public String getContentType() {
        return this.delegate.getContentType();
    }

    public boolean isRepeatable() {
        return this.delegate.isRepeatable();
    }

    public void writeRequest(final OutputStream out) throws IOException {
        this.delegate.writeRequest(new CountingOutputStream(out, this.listener));
    }

    public static interface ProgressListener {
        void transferred(long num);
    }

    public static class CountingOutputStream extends FilterOutputStream {

        private final ProgressListener listener;

        private long transferred;

        public CountingOutputStream(final OutputStream out,
                final ProgressListener listener) {
            super(out);
            this.listener = listener;
            this.transferred = 0;
        }

        public void write(byte[] b, int off, int len) throws IOException {
            out.write(b, off, len);
            this.transferred += len;
            this.listener.transferred(this.transferred);
        }

        public void write(int b) throws IOException {
            out.write(b);
            this.transferred++;
            this.listener.transferred(this.transferred);
        }
    }
}

Then implements a ProgressListener which updates a progress bar.
Remember that the progress bar update must not run on the Event Dispatch Thread.

Intubate

Could you please give an example of ProgressListener which will update Swing components – Rocambole

why CountingMultipartRequestEntity should be wrapped around MultipartRequestEntity? – Rocambole

Guys, your answer is incorrect. For many cases, this will double-count all bytes sent. FilterOutputStream#write(byte[],int,int) simply calls FOS.write(byte) in a loop, so you are double counting all bytes. Also, FOS docs recommend changing this behavior as it's not efficient. See my answer, where I save the underlying stream and call it's write(byte[],int,int) method. – Assumption

The more interesting ? is how to know which method to 'count' the bytes being sent in, the FOS#write(byte[],int,int) or FOS#write(b). I choose the first, b/c it works for me, but I asked this question over at #3163631 – Assumption

@Harny I answered that question and I've also fixed this answer to do the correct thing. – Rustie

In case anyone is having the same problem, the .jar you want to use this specific example is commons-httpclient-3.0.1 , which is stored as an archival copy on the Apache HttpClient project page. Newer versions don't seem to have the interface this example uses. – Poult

Awesome answer. If you're looking for the calling side for the above code I found this tutorial which helped me understand both sides. toolongdidntread.com/android/… – Anorthic

Answer 3

A simpler countingEntity would not depend on a specific entity type but rather extend HttpEntityWrapped:

package gr.phaistos.android.util;

import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.OutputStream;

import org.apache.http.HttpEntity;
import org.apache.http.entity.HttpEntityWrapper;

public class CountingHttpEntity extends HttpEntityWrapper {

    public static interface ProgressListener {
        void transferred(long transferedBytes);
    }


    static class CountingOutputStream extends FilterOutputStream {

        private final ProgressListener listener;
        private long transferred;

        CountingOutputStream(final OutputStream out, final ProgressListener listener) {
            super(out);
            this.listener = listener;
            this.transferred = 0;
        }

        @Override
        public void write(final byte[] b, final int off, final int len) throws IOException {
            //// NO, double-counting, as super.write(byte[], int, int) delegates to write(int).
            //super.write(b, off, len);
            out.write(b, off, len);
            this.transferred += len;
            this.listener.transferred(this.transferred);
        }

        @Override
        public void write(final int b) throws IOException {
            out.write(b);
            this.transferred++;
            this.listener.transferred(this.transferred);
        }

    }


    private final ProgressListener listener;

    public CountingHttpEntity(final HttpEntity entity, final ProgressListener listener) {
        super(entity);
        this.listener = listener;
    }

    @Override
    public void writeTo(final OutputStream out) throws IOException {
        this.wrappedEntity.writeTo(out instanceof CountingOutputStream? out: new CountingOutputStream(out, this.listener));
    }
}

Amalgam

how would i implement this ? could you tag me if you answer :) – Credulous

like this ProgressListener listener = new ProgressListener() { @Override public void transferred(long num) { parentTask.publicPublishProgress(num); } }; StringEntity se = new StringEntity(XML); CountingHttpEntity ce = new CountingHttpEntity(se, listener); httppost.setEntity(ce); – Filipino

Answer 4

The amount of bytes returned by the listener is different from the original file size. So, instead of having transferred++, I modified it so that transferred=len; that is the length of the actual amount of bytes being written to the output stream. And when I compute the addition of the total bytes transferred it is equal to the actual ContentLength returned by CountingMultiPartEntity.this.getContentLength();

public void write(byte[] b, int off, int len) throws IOException {
    wrappedOutputStream_.write(b,off,len);
    transferred=len;
    listener_.transferred(transferred);
}

Prig

Answer 5

Keep in mind that the progress bar might be misleading when an intermediate component in the network (e.g., an ISP's HTTP proxy, or a reverse HTTP proxy in front of the server) consumes your upload faster than the server does.

Ifill

Answer 6

As noted by the article Vincent posted, you can use Apache commons to do this.

Little snipped

DiskFileUpload upload = new DiskFileUpload();
upload.setHeaderEncoding(ConsoleConstants.UTF8_ENCODING);

upload.setSizeMax(1000000);
upload.setSizeThreshold(1000000);

Iterator it = upload.parseRequest((HttpServletRequest) request).iterator();
FileItem item;
while(it.hasNext()){
    item = (FileItem) it.next();
    if (item.getFieldName("UPLOAD FIELD"){
       String fileName = item.getString(ConsoleConstants.UTF8_ENCODING);
       byte[] fileBytes = item.get();
    }
}

Concerned

Answer 7

Just my 2c worth:

This is based off of tuler's answer(has a bug at time of writing). I modified it slightly, so here is my version of tuler and mmyers answer (I can't seem to edit their answer). I wanted to attempt to make this a bit cleaner and faster. Besides the bug(which I discuss in comments on their answer), the big issue I have with their version is that it creates a new CountingOutputStream with every write. This can get very expensive in terms of memory - tons of allocations and garbage collections. Smaller issue is that is uses a delegate when it could just expand the MultipartEntity. Not sure why they chose that, so I did it in a manner I was more familiar with. If anyone knows pros/cons of the two approaches that would be great. Finally, the FilterOutputStream#write(byte[], int,int) method just calls the FilterOutputStream#write(byte) in a loop. The FOS documentation recommends subclasses overriding this behavior and making this more efficient. The best way to do that here is to let the underlying OutputStream handle the writing request.

import java.io.FilterOutputStream;
import java.io.IOException;
import java.io.OutputStream;

import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntity;

public class CountingMultiPartEntity extends MultipartEntity {

    private UploadProgressListener listener_;
    private CountingOutputStream outputStream_;
    private OutputStream lastOutputStream_;

    // the parameter is the same as the ProgressListener class in tuler's answer
    public CountingMultiPartEntity(UploadProgressListener listener) {
        super(HttpMultipartMode.BROWSER_COMPATIBLE);
        listener_ = listener;
    }

    @Override
    public void writeTo(OutputStream out) throws IOException {
        // If we have yet to create the CountingOutputStream, or the
        // OutputStream being passed in is different from the OutputStream used
        // to create the current CountingOutputStream
        if ((lastOutputStream_ == null) || (lastOutputStream_ != out)) {
            lastOutputStream_ = out;
            outputStream_ = new CountingOutputStream(out);
        }

        super.writeTo(outputStream_);
    }

    private class CountingOutputStream extends FilterOutputStream {

        private long transferred = 0;
            private OutputStream wrappedOutputStream_;

        public CountingOutputStream(final OutputStream out) {
            super(out);
                    wrappedOutputStream_ = out;
        }

        public void write(byte[] b, int off, int len) throws IOException {
                    wrappedOutputStream_.write(b,off,len);
                    ++transferred;
            listener_.transferred(transferred);
        }

        public void write(int b) throws IOException {
            super.write(b);
        }
    }
}

Assumption

This has a lot more issues than the small one the other had. – Rustie

Answer 8

Look into HTTP Client for uploadign the file to the web. It should be able to to do that. I am unsure how to get the progress bar, but it would involve querying that API somehow.

Chaldea

Answer 9

Apache common is very good option. Apache common allows you to configure following things.

1. Configure(xml file) the maximum file size/ upload file size
2. Destination path (where to save the uploaded file)
3. Set the temp. folder to swap the file , so that file upload would be fast.

Epact

Which one of the Apache Commons components? – Dalmatic

Answer 10

From the other answers you can just override the AbstractHttpEntity class children or implementations public void writeTo(OutputStream outstream) method you are using if do not want to create a class.

An example using a FileEntity instance:

FileEntity fileEntity = new FileEntity(new File("img.jpg")){
    @Override
    public void writeTo(OutputStream outstream) throws IOException {
        super.writeTo(new BufferedOutputStream(outstream){
            int writedBytes = 0;

            @Override
            public synchronized void write(byte[] b, int off, int len) throws IOException {
                super.write(b, off, len);

                writedBytes+=len;
                System.out.println("wrote: "+writedBytes+"/"+getContentLength()); //Or anything you want [using other threads]
            }
        });
    }

};

Lalia