Java in-memory file structure?

Question

I need to do a lot of things with resources on the fly: parsing xsd/xml docs, building and compiling java classes, package them into jars ans wars, persist in DB, deploy them as OSGi, etc.

Most of the libraries/API's, which I use, allow to perform all these intermediate tasks in memory, but there are some "special" libraries operating with java.io.File only. And there's nothing left for me but using real temporary files and directories which is not good in Java EE environment.

I believe there must be a library/solution for in-memory file structure having nodes extending java.io.File (as I see it). Please drop in a link to known/similar libraries. Any comments are welcome.

Thanks!

Passade

I believe this is more or less the same question as #578805 – Paucker

@Chris Thompson, I've read it before posting. This is really not the same imho. – Passade

fair enough, I can see that side of things for sure. Either way, your question is a good one and I'd be shocked if there wasn't something out there to accomplish this... – Paucker

@Chris Thompson, i'm already shocked. Honestly, I was expecting 3-5 answers within 10 mins pointing to apache-zzz-lib or google-xxx-lib. I really don't want to reinvent the wheel and kill couple of hours coding my own implementation. – Passade

Answer 1

I do not believe you are going to find what you are looking for. The java.io.File API was not written with the intention of providing a file system abstraction that can be implemented in a variety of ways. While it does expose method for some FS operations (such as delete and mkdir), it doesn't handle the basic read/write I/O. That is left to other classes, such as FileInputStream. This means that from API standpoint, a File object is no more than a path. Nothing is abstracted. You are stuck.

Decisive

Good shot. Is it for sure that java.io.File does not perform basic I/O? Then I have no chances :( – Passade

Well, here is the javadoc: download.oracle.com/javase/1.4.2/docs/api/java/io/File.html. If File class was abstracting I/O operations you would expect to see methods like getInputStream(), getOutputStream(), etc., which of course aren't there. Since introduction of Java, others have written true file system APIs that can be implemented in variety of ways, but that doesn't help your usecase. – Decisive

Just looked through File and FileInputStream sources. You're right, I'm stuck. – Passade

@Osw - indeed, you have no chance of getting your original idea to work. You either have to change those "special" libraries, or use file system objects. – Nanaam

Answer 2

One option is to use a RAM disk. Your program will think its using the disk with java.io.File, but it will really be using main memory.

Lapboard

+1 for pointing out existing tooling. However, this still doesn't necessarily "fit well" into a "Java EE" environment ... whatever that is :) – Longsuffering

@pst, Why doesn't it "fit well" into a "Java EE" environment? Its just an extra FS the host system provides. He has to use some path to read and write his files anyway; it may as well be on the in-memory FS. – Lapboard

@Jay, thanks for idea, but it's a bit hardware/system stuff while I was thinking about software abstraction layer. – Passade

@Osw - instead of dismissing this idea out of hand as being "not what you were thinking of", why don't you think again ... and see if you can make use of it? The idea you were thinking of simply won't work. – Nanaam

@Stephen C, because it's the same file i/o abstraction, requiring extra attention/configuration, which is OS specific, etc, etc. And, finally, being not welcome in java ee to the very same extent. – Passade

@Osw - none of that is relevant. The standard Java class libraries are not implemented like that, and there's no point wishing that they were. Look at what is possible instead ... – Nanaam

Answer 3

There is a fine alternative available: https://github.com/google/jimfs

This supports java(7+) in memory filesystem handling and is very easy to use too.

Amal

Is there a way to get Jimfs to work with memory mapped byte buffers rather than heap allocated byte buffers? Thanks. -- Jon – Lamelli