Constructing a DataSource from an InputStream or Byte array

Question

I am writing a small file upload utility thing as part of a larger project. Originally I was handling this from a servlet using the Apache commons File utility classes. Here is a snippet from a quick test client I wrote for the service:

public static void main(String[] args) {

  JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();

 factory.getInInterceptors().add(new LoggingInInterceptor());
 factory.getOutInterceptors().add(new LoggingOutInterceptor());
 factory.setServiceClass(FileUploadService.class);
 factory.setAddress("http://localhost:8080/FileUploadService/FileUploadService");
 FileUploadService client = (FileUploadService) factory.create();

 FileType file = new FileType();
 file.setName("statemo_1256144312279");
 file.setType("xls");

 DataSource source = new FileDataSource(new File("c:/development/statemo_1256144312279.xls"));
 file.setHandler(new DataHandler(source));
 Boolean ret = client.uploadFile(file);
 System.out.println (ret);
 System.exit(0);

}

This works absolutely fine. Now the problem comes when I am trying to replace the Apache commons utilities. In the above code I am creating a DataSource from a File with an absolute path name. In my servlet, I can't get an absolute path name however and the file I am sending over the wire is empty.

Here is the servlet code:

 @SuppressWarnings("unchecked")
    protected void doPost (final HttpServletRequest request, final HttpServletResponse response) 
        throws ServletException, IOException {

    // form should have enctype="multipart/form-data" as an attribute
 if (!ServletFileUpload.isMultipartContent (request)) {
  LOG.info("Invalid form attribute");
  return;
 }

 //DataInputStream in = new DataInputStream(request.getInputStream());

 final DiskFileItemFactory factory = new DiskFileItemFactory ();
 factory.setSizeThreshold(FILE_THRESHOLD_SIZE);

 final ServletFileUpload sfu = new ServletFileUpload (factory);
 sfu.setSizeMax(MAX_FILE_SIZE);

 final HttpSession session = request.getSession();

 final List<FileItem> files = new ArrayList<FileItem>();

 final List<String> filesToProcess = new ArrayList<String>();

 try {
        final List<FileItem> items = sfu.parseRequest(request);

        for (final FileItem f : items) {
            if (!f.isFormField())
                files.add(f);
        }

        /*for (final FileItem f : files) {
         final String absoluteFileName = UPLOAD_DESTINATION + FilenameUtils.getName(f.getName());

            //f.write(new File (absoluteFileName));
            filesToProcess.add(absoluteFileName);
        }*/

        FileItem f = files.get(0);

        LOG.info("File: " + FilenameUtils.getName(f.getName()));
        LOG.info("FileBaseName: " + FilenameUtils.getBaseName(f.getName()));
        LOG.info("FileExtension: " + FilenameUtils.getExtension(f.getName()));

        FileUploadServiceClient client = new FileUploadServiceClient();

        DataSource source = new FileDataSource(new File(f.getName()));

        FileType file = new FileType();
        file.setHandler(new DataHandler(source));
        file.setName(FilenameUtils.getBaseName(f.getName()));
        file.setType(FilenameUtils.getExtension(f.getName()));

        Boolean ret = client.uploadFile(file);

        LOG.info("File uploaded - " + ret);

        filesToProcess.add(UPLOAD_DESTINATION + FilenameUtils.getName(f.getName()));
        session.setAttribute("filesToProcess", filesToProcess);

  final RequestDispatcher dispatcher = request.getRequestDispatcher("Validate");
        if (null != dispatcher) {
         dispatcher.forward(request, response);
        }
    } catch (FileUploadException e) {
        LOG.info("Exception " + e.getMessage());
        e.printStackTrace();
    } catch (Exception e) {
        LOG.info("Exception " + e.getMessage());
        e.printStackTrace();
    }

}

I've been working on this for the better part of this morning and am not getting anywhere. Even if I get rid of the Apache commons file stuff completely and handle the parsing of the request myself, I still can't construct the DataSource appropriately.

Thanks!

Trichomonad

Answer 1

This was rather simple actually, I just copied over the bytes from the InputStream to the DataSource:

FileItem f = files.get(0);

// there is a problem here where the file being created is empty, since we only have a
// partial path:
DataSource source = new FileDataSource(new File(f.getName()));

// because of the above problem, we are going to copy over the data ourselves:
byte[] sourceBytes = f.get();
OutputStream sourceOS = source.getOutputStream();
sourceOS.write(sourceBytes);

Trichomonad

Exactly. You need to send the file contents, not just the file handle ;) – Dorkas

Answer 2

• This is the code of commons-email ByteArrayDataSource
• it sounds odd to try to replace apache commons - don't, unless you have a really good reason
• you can get absolute paths in a servlet. You can call getServletContext().getRealPath("/") which will return the absolute path of your application, and then you can get files relative to it.

Aperture

Yes, I didn't want to replace apache commons. I didn't even think about just getting the path from the context and getting the file that way. The other part of this project would make that difficult though. – Trichomonad

I will take a look at the ByteArrayDataSource code tomorrow and may just use that in my solution. Thanks! – Trichomonad

Answer 3

In our application there are objects that have properties InputStream and Name. We are using next class to construct DataSource with those properties.

public class InputStreamDataSource implements DataSource {

    ByteArrayOutputStream buffer = new ByteArrayOutputStream();
    private final String name;

    public InputStreamDataSource(InputStream inputStream, String name) {
        this.name = name;
        try {
            int nRead;
            byte[] data = new byte[16384];
            while ((nRead = inputStream.read(data, 0, data.length)) != -1) {
              buffer.write(data, 0, nRead);
            }
            inputStream.close();
            buffer.flush();
        } catch (IOException e) {
            e.printStackTrace();
        }

    }

    @Override
    public String getContentType() {            
        return new MimetypesFileTypeMap().getContentType(name);
    }

    @Override
    public InputStream getInputStream() throws IOException {
            return new ByteArrayInputStream(buffer.toByteArray());
    }

    @Override
    public String getName() {
       return name;
    }

    @Override
    public OutputStream getOutputStream() throws IOException {
        throw new IOException("Read-only data");
    }

}

Torment

I think this still loads the entire stream into memory, or am I wrong? (this is also the case with the other answers, or they load a file while I only have an InputStream). – Interdict

Answer 4

Most of the solutions shown here require that the InpustStream be closed (read into memory). It is possible to wrap the InputStream in a DataSource object without closing the InputStream though:

  private record PipedDataSource(InputStream in, String contentType, String encoding) 
    implements DataSource, EncodingAware {

    public String getContentType() {
      return contentType;
    }

    public InputStream getInputStream() {
      return in;
    }

    public String getName() {
      return "PipedDataSource";
    }

    public OutputStream getOutputStream() throws IOException {
      throw new IOException("No OutputStream");
    }

    @Override
    public String getEncoding() {
      return encoding;
    }
  }

The example above also implements EncodingAware. This can prevent the InputStream from being closed by third part libraries (for example java.mail.internet.MimeUtility) when they get the data source encoding.

Viscus