Returning huge file from a remote EJB call

Question

I have an EJB client that needs to retrieve a large file from an EJB server (JBoss).

The evident way to implement this is the server to offer a EJB facade with a method like this one:

public byte[] getFile(String fileName);

That means, loading the whole file in memory, in a byte array, and then sending this byte array on the wire.

The problem is that this approach loads the whole file in memory, and since the file is huge, it would overflow it.

Is there any option to overcome this problem?

Diagram

Answer 1

HTTP would be a better choice, but that said, try this serialization trick:

import java.io.*;

public class FileContent implements Serializable {

    private transient File file;

    public FileContent() {
    }

    public FileContent(File file) {
        this.file = file;
    }

    private void writeObject(ObjectOutputStream out) throws IOException {
        // 1. write the file name
        out.writeUTF(file.getAbsolutePath());

        // 2. write the length
        out.writeLong(file.length());

        // 3. write the content
        final InputStream in = new BufferedInputStream(new FileInputStream(file));
        final byte[] buffer = new byte[1024];

        int length;
        while ((length = in.read(buffer)) != -1) {
            out.write(buffer, 0, length);
        }
        out.flush();
        in.close();
    }

    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
        // 1. read the file name
        final String path = in.readUTF();

        // 2. read the length
        long remaining = in.readLong();

        // 3. read the content
        file = new File(path);
        final OutputStream out = new BufferedOutputStream(new FileOutputStream(file));

        final byte[] buffer = new byte[1024];

        while (true) {
            int length = in.read(buffer, 0, (int) Math.min(remaining, buffer.length));
            if (length == -1) break;

            out.write(buffer, 0, length);

            remaining -= length;
            if (remaining <= 0) break;
        }
        out.flush();
        out.close();
    }
}

Nunci

agreed that HTTP would be a better option where possible, but the Externalizable solution you have seems pretty good and I've seen it working well in practice. The contents of the file are never held in memory though as your note implies. The code above will efficiently stream arbitrary sized files without causing memory issues. It does lack try/finally to ensure the IO streams are closed though. – Attica

as i mentioned in my answer, the rmiio library has a more mature implementation of this concept (which is generalized to any InputStream, not just a File, and includes on-the-fly compression support). another problem with this example is that it assumes the same file name is valid on the client and server. – Trespass

@Trespass Thanks for the notes. I wasn't aware of RMIIO, sounds like a fantastic library. Looks like my Externalizable note is outdated. This is the way it worked way back in JDK 1.2 days. Looks like they removed that at some point, which is fantastic. – Nunci

Answer 2

The RMIIO library was built for exactly this situation. It even includes an implementation of @DavidBlevins solution, the DirectRemoteInputStream.

Trespass

Answer 3

You may use ZipOutputStream, which wrapped in BytearrayOutputStream. This action will allow you to return compressed byte array, using you method declaration in the first place, in order to decrease overhead of RMI transmition.

Goldbrick

Answer 4

RMI protocol is a totally bad solution for sending large files. A possible but quiet inefficient solution is splitting your file in small chunks, send them from EJB and reassemble file on the client-side. Something like this:

public class FileMetadata {
    ....
    private long chunkCount;
    private long chunkSize;
    ....
}

...
public FileMetadata getFileMetadata(String fileName) {...}
public byte[] getFileChunk(String fileName, long chunkNumber) {...}
...

Monetmoneta

Answer 5

I would reconsider you architecture, IMHO you should keep your ejb calls quick, what about returning the location of the file from the ejb call and then handling the downlod in another procedure, have a look here for more on how to download the large file How to download and save a file from Internet using Java?

Jaela

Answer 6

The approach I took (mind you only works for @LocalBeans) is to write to a File.createTempFile from the EJB and return a File to retrieve the data and delete it from the client. Path may be used as well, but note that it is not Serializable.

However, you can also make it write to a file hosted by an HTTP server and retrieve it from there later. Then send an HTTP DELETE request to delete the file afterwards. Your EJB would instead return a java.net.URI.

Charmine