Suggestions to avoid DB deps when using an in-memory sqlite DB to speed up unit tests

Question

I've been using sqlite::memory: for unit tests: it is quick, and makes test clean-up automatic. But I'm concerned my tests could be missing bugs due to SQL server assumptions. (This particular code is supposed to be DB-neutral, but pragmatically it must work with MySQL.)

As a specific example, I think date string inputs to sqlite get stored as strings (*), so they get given back to me in exactly the same format. Whereas in MySQL they appear to get parsed, normalized, and returned in a consistent format. My bigger concern is that there are differences I'm not so conscious of.

What is the best way to handle this? E.g. repeat all unit tests for mysql too (slow - is there a way to make an in-memory MySQL DB?). Just do mysql tests for some key areas of concern? Rely on functional tests that use MySQL (this is my current approach, but the coverage is much spottier). Something else I've not thought of?

*: If I give "2012-12-25 09:12:34" I get that exact string back. If I give "2012-12-25 09:12:34 UTC" I get back that exact string (i.e. with the timezone this time). If I give "2012-12-25 09:12:34 hello world" I get back "2012-12-25 09:12:34 hello world" :-(

---

In this particular case I'm using PHP, PDO and PHPUnit. I'm after language-independent strategies, but it is worth noting that PDO is a quite thin abstraction, more to do with abstracting the connection string and low-level API calls. It doesn't try to abstract data type differences between the databases.

Phono

Unit testing with a database is an oxymoron. – Prepotent

Answer 1

If you want to test MySQL-specific SQL details, you just have to test on MySQL.

MySQL has the MEMORY storage engine, but this is not fully compatible with other engines. You should put the database on a RAM disk (tmpfs).

SQLite does not have dedicated data types for dates/times.

Craps

Answer 2

What we typically do where I work is, split the tests in:

• Unit tests: very fast, using an in-memory database (H2 in my case).

• Integration test: slower, use the 'real' database, plus additional (longer running) tests

During development, we just run the unit tests. The automated build runs the integration tests as well (but you can run them yourself if you want to).

Liqueur

Thomas, that is exactly what I do (and advise others to do). But do you ever get conflicts and how do you deal with them? E.g. the integration test fails because a unit test passed that ought to have failed. Does it happen rarely enough that you just live with the pain? – Phono

I don't have a generic solution. It also depends on the project: one of the projects I work on has concurrency issues that cause the integration tests to fail sometimes, which ius a pain. Of course I tried to add unit tests that are faster and uncover the same problem, but they are typically less 'real life'. – Liqueur