Is there a dbunit-like framework that doesn't suck for java/scala?

Question

I was thinking of making a new, light-weight database population framework. I absolutely hate dbunit. Before I do, I want to know if someone already did it.

Things i dislike about dbunit:

1) The simplest format to write and get started is deprecated. They want you to use formats that are bloated. Some even require xml schemas. Yeah, whatever.

2) They populate rows not in the order you write them, but in the order tables are defined in the xml file. This is really bad because you can't order your data in such a way that foreign key constraints won't cause problems. This just forces you to go through the hassle of turning them off altogether.

This also wastes time and bloats up your junit base classes to include code to disable the foreign key constraints. You will probably have to test for the database type (hsqldb, etc.) and disable them in database-specific ways. This is way bad.

It could be better if dbunit helped in disabling foreign key constraints as part of their framework automatically, but they don't do this. They do keep track of dialects... so why not use them for this? Ultimately, all of this does is force the programmer to waste time and not get up and testing quickly.

3) XML is a pain to write. I don't need to say more about this. They also offer so many ways to do it, that I think it just complicates matters. Just offer one really solid way and be done with it.

4) When your data gets large, keeping track of the ids and their consistent/correct relationships is a royal pain.

Also, if you don't work on a project for a month, how are you to remember that user_id 1 was an admin, user_id 2 was a business user, user_id 3 was an engineer and user_id 4 was something else? Going back to check this is wasting more time. There should be a meaningful way to retrieve it other than an arbitrary number.

5) It's slow. I've found that unless hsqldb is used, it is painfully slow. It doesn't have to be. There are also numerous ways to mess up its configuration as it is not easy to do "out of the box". There is a hump that you must go through to get it working right. All this does is encourage people to not use it, or be pissed of when they do start to use it.

6) Some values tend to repeat a lot, likes dates. It'd be nice to specify defaults, or even have the framework put defaults in automatically, even without you telling it to put defaults in there. That way you can create objects just with the values you want, and leave the rest off. This sure beats specifying every nook and cranny of a column if it's not required.

7) Probably the most annoying thing is that the first entry must include ALL the values - even null placeholders - or future rows won't pick the columns that you actually specified.

DBunit doesn't have a sensible default for translating [NULL] to a real null value either. You have to manually add it. Tell me, who hasn't done this with dbunit? Everyone has. It shouldn't be like this!

What this means is that if you have a polymorphic object, you must declare all the foreign keys to the joining tables of each subclass in the first row, even though they are null. If you do a table for all subclasses pattern, you still have to specify all the fields on the first row. This is just awful.

Anything out there to satisfy me, or should I become the next framework developer of a much better database testing framework?

Scarlatina

Instead of writing a new just fix what's broken... – Exoergic

I was thinking of just doing it in scala with backwards compatibility in mind, since that's pretty much the way of the future anyway. – Scarlatina

Answer 1

I'm not aware of any real alternative to DbUnit and none of the tools mentioned by @Joe are in my eyes:

• Incanto: not DB agnostic
• SQLUnit: a regression and unit testing harness for testing database stored procedures (that's not what DbUnit is about)
• Cactus: a tool for In-container testing (I fail to see where it helps with databases)
• Liquibase: a database migration tool (doesn't load/verify data)
• ORMUnit: can initialize a database but that's all
• JMock: doesn't compete with DbUnit at all

That being said, I've personally used DbUnit successfully several times, on small and huge projects, and I find it pretty usable, especially when using Unitils and its DbUnit module. This doesn't mean it's perfect and can't be improved but with decent tooling (either custom made or something like Unitils), using it has been a decent experience.

So let me answer some of your points:

1. The simplest format to write and get started is deprecated. They want you to use formats that are bloated. Some even require xml schemas. Yeah, whatever.

DbUnit supports flat or structured XML, XLS, CSV. What revolutionary format would you like to use? By the way, a DTD or schema is not mandatory when using XML. But it gives you nice things like validation and auto-completion, how is that bad? And Unitils can generate it easily for you, see Generate an XSD or DTD of the database structure.

It could be better if dbunit helped in disabling foreign key constraints as part of their framework automatically, but they don't do this. They do keep track of dialects... so why not use them for this? Ultimately, all of this does is force the programmer to waste time and not get up and testing quickly.

They are waiting for your patch.

Meanwhile, Unitils provides support to handle constraints transparently, see Disabling constraints and updating sequences.

3. XML is a pain to write. I don't need to say more about this. They also offer so many ways to do it, that I think it just complicates matters. Just offer one really solid way and be done with it.

I guess pain is subjective but I don't find it painful, especially when using a schema and autocompletion. What is the silver bullet you're suggesting?

4. When your data gets large, keeping track of the ids and their consistent/correct relationships is a royal pain.

Keep them small, that's a know best practice. You're going against a known best practice and then complain...

Also, if you don't work on a project for a month, how are you to remember that user_id 1 was an admin, user_id 2 was a business user, user_id 3 was an engineer and user_id 4 was something else? Going back to check this is wasting more time. There should be a meaningful way to retrieve it other than an arbitrary number.

Yes, task switching is counter productive. But since you're working with low level data, you have to know how they are represented, there is no magic solution unless you use a higher level API of course (but that's not the purpose of DbUnit).

5. It's slow. I've found that unless hsqldb is used, it is painfully slow. It doesn't have to be. There are also numerous ways to mess up its configuration as it is not easy to do "out of the box". There is a hump that you must go through to get it working right. All this does is encourage people to not use it, or be pissed of when they do start to use it.

That's inherent to databases and JDBC, not DbUnit. Use a fast database like H2 if you want things to be as fast as possible (if you have a better agnostic way to do things, I'd be glad to learn about it).

6. Probably the most annoying thing is that the first entry must include ALL the values - even null placeholders - or future rows won't pick the columns that you actually specified.

Not when using Unitils as mentioned in presentations like Unitils - Home - JavaPolis 2008 or Unit testing: unitils & dbmaintain.

Anything out there to satisfy me, or should I become the next framework developer of a much better database testing framework?

If you think you can make things better, maybe contribute to existing solutions. If that's not possible and if you think you can create the killer database testing framework, what can I say, do it. But don't forget, ranting is easy, coming up with solutions using your own solutions is less so.

Peppercorn

Good wisdom there at the last paragraph. – Like

Just because it's not easy doesn't mean it cannot be done or that I cannot do. I'm quite smart and hard working. Thanks for the post. You deserve the +rep for all the stuff you wrote ;) – Scarlatina

Oh, I never had problems with auto-completion without a schema actually. IntelliJ Idea auto-completes just fine given 1 example of the tag without a schema. Schemas just ended up being bloated as I had to modify the schema at the same time as my domain model and hibernate configuration. Adding "just one more thing" to this process is bad, and getting a tool to disguise it is not a real solution :/ – Scarlatina

I just wanted to say that ranting was not all that I did. Visit here for scaladbtest which fixes everything I "ranted" about: github.com/egervari/scaladbtest – Scarlatina

@Scarlatina Thanks for your work and for sharing it Ken, I respect that. Good job. Competition is good and I'll give scaladbtest a try to see how it feels :) – Peppercorn

My answer is buried below but have a look at github.com/eclecticlogic/pedal-loader – Cummerbund

Answer 2

As a DbUnit developer I'm grateful for criticism and I must partially agree with you. We are currently starting the design of the next DbUnit major release and I wish to invite you to participate both in the discussion and development.

I'm not going to answer your points as your question is not really related to DbUnit, but to DbUnit alternatives. Anyway, I just want to highlight your point 7 is completely false: you do not need to specify all the columns on first row any more, the feature is called column sensing. I'm not going to tell you why it's not enabled by default as you are surely smart enough to understand it by yourself.

I'll give scaladbtest a deep examination in the hope we can integrate their ideas.

Drusi

I've dug into your scaladbtest project and I have to admit that it is a good extension but you strived too much on comparing it to dbUnit. Actually you only support one file format and you haven't dealt with many issues dbUnit is capable to solve automatically. Can I suggest to follow the dbUnit evolution? Next dbUnit version will be very interesting for you. Adn we love competition so, please, continue to push us to our best, we'll do the same ;) – Drusi

Answer 3

Faced with similar concerns using DBUnit I have found this : http://dbsetup.ninja-squad.com/index.html which may address concerns. Such as instead of representing test data in separate files all DB content is contained within the java class itself.

Abroms

This is awesome. I'm thinking of using this from now on instead of DbUnit. – Foretime

+1, I find DbSetup the most convenient to work with. – Bicipital

Answer 4

If you use the Spring Framework (or don’t mind using it at least for testing), then Spring DBUnit is currently the best (maintained) alternative to plain DBUnit that I know and use. Quoting their website:

Spring DBUnit provides integration between the Spring testing framework and the popular DBUnit project. It allows you to setup and teardown database tables using simple annotations as well as checking expected table contents once a test completes.

Spring DBUnit appears to be the ‘somewhat official’ Spring solution for DB unit testing (with DBUnit); at least the author/maintainer of the library, Phil Webb, is working at SpringSource/Pivotal.

Caughey

I was disappointed that @DatabaseSetup can't be inherited from parent class. Looks like the project is not maintained well now. – Umlaut

Answer 5

I use DBUnit, with a few wrappers to smooth over the rough edges. A nice tool that can either complement or overlap the functionality is Jailer. It can extract subsets of data from a reference database, and store this as either DBUnit compatible XML files, or as "topologically sorted DML files", which respect the foreign key constraints.

Betatron

Answer 6

I just released a library called JDBDT (Java Database Delta Testing) that you may use for database setup and validation in software tests.

Have a look at http://jdbdt.org

Best, Eduardo

Orren

just checked on your library JDBDT. Good job man !! I like it. It has many upsides compared with DBUnit, and I think it will be become a thing soon – Sascha

Thanks, glad you liked it. Have a go at it and let me know of any issues through the GitHub issue tracker. There is a tutorial available - jdbdt.org/Tutorial.html and related code at GitHub. – Orren

Answer 7

Another vote for wrapping DBUnit with a modern library to improve usability and conciseness. My choice is database-rider, which makes DBUnit a breeze to use and even supports JUnit 5 as demonstrated in the following example:

@RunWith(JUnitPlatform.class)
@ExtendWith(DBUnitExtension.class)
@DBUnit(cacheConnection = true, cacheTableNames = true)
class TestInstrumentQueryService {

    private ConnectionHolder connHolder = () -> EntityManagerProvider.instance("my-jta-unit").connection();

    @DBRider
    @DataSet("datasets/instrumentIds.yml")
    void testFindInstrumentById() {

        InstrumentQueryService iqs = new InstrumentQueryService(EntityManagerProvider.em());

        Instrument instr = iqs.findInstrumentById(InstrumentIdType.TICKER_BBG, "AAPL");
        assertEquals(100, instr.getId());
    }
}

Notice how this allows leveraging (concise) YAML testing data sets seamlessly (YAML not XML, though I'm lead to believe that DBUnit actually supports those natively).

Sheik

Answer 8

You're making excellent point.

I've been working for a lot of web portals over the last years, mostly with PHP, but also some Java now and then.
And like you I don't get that after all these years framework and unittesting developers don't seem to realize how much storage handling has changed in the last decade. It's not enough to just send create/insert/truncate statements to some database! If you're operating at large scale you end up employing all sorts of storage backends, organized in layers to push hot content out fast. Plus on the Database front there's the issue of data partitioning. If you don't have a proper foreign key abstraction provided you will certainly go nuts when your storage setup changes. And while we're at it: fixture ordering by foreign key precedence has many pitfalls and I have yet to see a real solution for that with DBUnit.

Anyway, the point is having just a basic database storage in place for unittesting is not enough for complex storage setups, since they often fail to reproduce problems in the live environment and are a pain in the ass to maintain.

Without wanting to sound like a fanboy: one place where things are okay is ruby on rails. That has a persistent model concept that people seem to have actually put some thought into. If you're dealing in PHP, Symfony is the place to go. It is limited via the default inclusion of Doctrine, with is also quite DB-centric, but it has clean interfaces and great extensibility and copied the rails fixture system completely. Professionally I need to stick to homebrew solutions for now, but they work okay.

Adequacy

Answer 9

Here's a short list of a few tools in this vein (besides DBunit) that I particularly like, or find interesting. At the very least they may offer some inspiration:

• Incanto
• SQLunit
• Cactus
• Liquibase
• ORMUnit
• JMock

Note that none of these are really competitors to DBunit in terms of scope or feature sets. However, there are some interesting ideas there that might be worth taking a look at. Good luck!

Embrangle

Answer 10

We are writing Daleq as a wrapper around DbUnit to address some of the mentioned concerns. It allows populating a DB just within your unit test rather than relying on editing XML files.

Aplacental

Answer 11

I too had similar issues with DBUnit. Especially for using it to populate local development data and exporting data from a real database. I ran into several cases where it would export a dataset that it couldn't then import.

This inspired me to write a new library for it: https://github.com/jeffskj/phonydata

This uses a groovy DSL to define the datasets which makes for a very compact representation of the data and makes it possible to do cool things like generate random data since it's just groovy code.

Conjugated

Answer 12

The situation of DBUnit is indeed sometimes frustrating. Some of the problem are solved from Marc Philipp with dbunit-datasetbuilder, specially if you combine it with the validator, which is in a very early stage. You can see it in action at SZE.

Disclaimer: All referenced github-resources are maintained by me.

Stunsail

Answer 13

An alternative using Spring configuration and Specs2 testing can be found here

Walt

Answer 14

I just released a groovy DSL based framework called pedal-loader available via github. Documentation here.

It allows you to work with JPA entity level abstraction directly. Since it is a groovy script, you can use all of the groovy constructs.

To insert rows into a table backed by a JPA entity called Student, with fields (not database columns, but mapped fields) called id, name and grade, you would do something like this:

allStudents = table(Student, ['id', 'name', 'grade']) {
    row 1, 'Joe', Grade.A
    rowOfInterest = row 2, 'John', Grade.B
}

Grade is an enum in the Student class that is mapped to the database column (perhaps using JPA 2.1 @Convert annotation). allStudents is a list that will hold the rows and rowOfInterest is a reference to a particular row. These properties (allStudents and rowOfInterest) become available to your unit test.

Sideline