java embedded database w/ ability to store as one file

Question

I need to create a storage file format for some simple data in a tabular format, was trying to use HDF5 but have just about given up due to some issues, and I'd like to reexamine the use of embedded databases to see if they are fast enough for my application.

Is there a reputable embedded Java database out there that has the option to store data in one file? The only one I'm aware of is SQLite (Java bindings available). I tried H2 and HSQLDB but out of the box they seem to create several files, and it is highly desirable for me to have a database in one file.

edit: reasonably fast performance is important. Object storage is not; for performance concerns I only need to store integers and BLOBs. (+ some strings but nothing performance critical)

edit 2: storage data efficiency is important for larger datasets, so XML is out.

Waterspout

Have you tried Apache Derby/JavaDB? I have no idea how it stores data behind the scenes, but it's one other thing you can look at. – Mcquade

Firebird definitely meets the reasonably fast requirement. :) – Writeoff

Try github.com/jankotek/MapDB – Ory

Answer 1

I think for now I'm just going to continue to use HDF5 for the persistent data storage, in conjunction with H2 or some other database for in-memory indexing. I can't get SQLite to use BLOBs with the Java driver I have, and I can't get embedded Firebird up and running, and I don't trust H2 with PAGE_STORE yet.

Waterspout

Answer 2

Nitrite Database http://www.dizitart.org/nitrite-database.html

NOsql Object (NO2 a.k.a Nitrite) database is an open source nosql embedded document store written in Java with MongoDB like API. It supports both in-memory and single file based persistent store.

Thousandfold

Answer 3

H2 uses only one file, if you use the latest H2 build with the PAGE_STORE option. It's a new feature, so it might not be solid.

Kiefer

Answer 4

If you only need read access then H2 is able to read the database files from a zip file.

Likewise if you don't need persistence it's possible to have an in-memory only version of H2.

If you need both read/write access and persistence, then you may be out of luck with standard SQL-type databases, as these pretty much all uniformly maintain the index and data files separately.

Montague

Answer 5

Once i used an object database that saved its data to a file. It has a Java and a .NET interface. You might want to check it out. It's called db4o.

Immobile

+1: for mentioning db4o, since I would have posted it if you hadn't – Ritornello

Answer 6

Chronicle Map is an embedded pure Java database.

• It stores data in one file, i. e.

  ChronicleMap<Integer, String> map = ChronicleMap
      .of(Integer.class, String.class)
      .averageValue("my-value")
      .entries(10_000)
      .createPersistedTo(databaseFile);
  

• Chronicle Map is mature (no severe storage bugs reported for months now, while it's in active use).

• Idependent benchmarks show that Chronicle Map is the fastest and the most memory efficient key-value store for Java.

The major disadvantage for your use case is that Chronicle Map supports only a simple key-value model, however more complex solution could be build on top of it.

Disclaimer: I'm the developer of Chronicle Map.

Aspectual

Answer 7

If you are looking for a small and fast database to maybe ship with another program I would check Apache Derby I don't know how you would define embedded-database but I used this in some projects as a debugging database that can be checked in with the source and is available on every developer machine instantaneous.

Conchology

Apache Derby is now the Java DB. As such it's well documented and supported, and probably came with your IDE. – Cuneiform

Answer 8

This isn't an SQL engine, but If you use Prevayler with XStream, you can easily create a single XML file with all your data. (Prevayler calls it a snapshot file.)

Although it isn't SQL-based, and so requires a little elbow grease, its self-contained nature makes development (and especially good testing) much easier. Plus, it's incredibly fast and reliable.

Eleventh

Answer 9

You may want to check out jdbm - we use it on several projects, and it is quite fast. It does use 2 files (a database file and a log file) if you are using it for ACID type apps, but you can drop directly to direct database access (no log file) if you don't need solid ACID.

JDBM will easily support integers and blobs (anything you want), and is quite fast. It isn't really designed for concurrency, so you have to manage the locking yourself if you have multiple threads, but if you are looking for a simple, solid embedded database, it's a good option.

Latency

A bit of fun history: Jan Kotek - one of the later jdbm contributors, but definitely not the original author - forked the project into mapdb and has done some amazing work with it. mapdb is no longer even remotely like jdbm - it's better in every way conceivable, and Jan continues to impress. Worth checking out. – Latency

Answer 10

Since you mentioned sqlite, I assume that you don't mind a native db (as long as good java bindings are available). Firebird works well with java, and does single file storage by default.

Both H2 and HSQLDB would be excellent choices, if you didn't have the single file requirement.

Writeoff

are you using the Jaybird driver? – Waterspout