What's the "best" database for embedded? [closed]

Question

I'm an embedded guy, not a database guy. I've been asked to redesign an existing system which has bottlenecks in several places.

The embedded device is based around an ARM 9 processor running at 220mHz.

There should be a database of 50k entries (may increase to 250k) each with 1k of data (max 8 filed). That's approximate - I can try to get more precise figures if necessary.

They are currently using SqlLite 2 and planning to move to SqlLite 3.

Without starting a flame war - I am a complete d/b newbie just seeking advice - is that the "best" decision? I realize that this might be a "how long is a piece of string?" question, but any pointers woudl be greatly welcomed. I don't mind doing a lot of reading & research, but just hoped that you could get me off to a flying start. Thanks.

p.s Again, a total rewrite, might not even stick with embedded Linux, but switch to eCos, don't worry too much about one time conversion between d/b formats. Oh, and accesses should be infrequent, at most one every few seconds.

---

edit: ok, it seems they have 30k entries (may reach 100k or more) of only 5 or 6 fields each, but at least 3 of them can be a search key for a record. They are toying with "having no d/b at all, since the data are so simple", but it seems to me that with multiple keys, we couldn't use fancy stuff like a quicksort() type search (recursive, binary search). Any thoughts on "no d/b", just data-structures?

Btw, one key is 800k - not sure how well SqlLite handles that (maybe with "no d/b" I have to hash that 800k to something smaller?)

Maximomaximum

sqllite seems like a good choice to stick with – Folium

yeah, i'd recommend sqllite too. – Roe

Is the database even your bottleneck? By your numbers, it should be fine unless your storage is really slow. Remember that 10 years ago that would have been considered a fairly fast machine. – Superpower

Wow, that's a sslllooowwww ARM9 processor, running at 0.220 cycles / sec! ;-) (By the way, I like the questions you've asked recently) – Clow

220mHz used to be a powerful PC only a few years ago ;-) But the task that the device does won't strain that. Thanks for the compliment on the questions - I'll go check out your answers now ;-) – Maximomaximum

Mawg: What Dan is coming at, is that you've written mHz instead of MHz. m = milli, M = mega. Semantics :) – Pyroxylin

Answer 1

Also SQLite is the Database chosen by virtually all mobile operating systems. Android, Iphone OS and Symbian ship with SQLite which makes me think that manpower was spent to optimize it for the processor in those phones (nearly always ARM).

Ilario

Answer 2

I would stick with SQLite, it's widely supported and pretty rich in features.

Soundproof

Answer 3

• Firebird (previously Interbase) claims to work well embedded.

• HypersonicQL (HQL) is small and fast and also claims to be suitable for embedded use.

Alas, I have no personal experience to back up either claim.

Tweedy

Answer 4

SQLite is probably a pretty safe bet. However, if performance is really important for your application and you do not need a relational database, I would suggest you take a look at Berkeley DB link text . Berkeley DB is not a relational database though. In other words, if your data is grouped in different tables and you constantly need to query result sets that require relating data from more than one table, you probably need a relational database. Berkeley DB is better suited for something like look up tables (i.e., the data is organized in a few tables and you don't need to query data from more than one of them in order to produce the result sets you want). Berkeley DB is very fast but it will require more work on your end in order to get the most out of it.

Closure

Just a single table, 5 or 6 fields, but three of them can be used as a unique key... and there are a lot of records - 30k, might become 100k, possibly even 256k – Maximomaximum

In this case, I'd say Berkeley DB is absolutely perfect for your application. If performance is of utmost importance for your application, I think you should look into Berkeley DB. – Closure

Here's a performance comparison of sqlite and Berkeley DB: sqlite.org/cvstrac/wiki?p=KeyValueDatabase for key-value data storing – Dedrick

Answer 5

if you want an alternative, then berkeleydb is worth looking at. it used to be owned by sleepycat software, but is now available from oracle. it's a barebones database engine; is directly programmable (rather than a sql) frontend. it's used as part of the core engine in many major databases, and as the database in many embedded devices - it used to be particularly popular for managing routing tables in routers. it tends to get overlooked these days for more fashionable setups, but i've found it to be decent, solid and for the numbers you are talking about it can be lightning fast.

Awhile

Answer 6

SQLite is ok, but don't plan to use if you plan to insert, update and delete data that involves more that 6 millon rows(All at the same time, or any partial part). The thing is that the VACCUM keyword has to be done everynow and then and it becomes a very severe bottleneck for performance, even when it's automatic.

Sectarianize

Answer 7

8 Years late, but as an update: I've had pretty good experience using Raima Database Manager. If you are looking for a small footprint db, they can get down to 40k. One of the reasons I like RDM is the platform independence, it is portable across 32-bit and 64-bit machines and between big-endian and little-endian architectures as well as support for most operating systems, meaning you can use it on Embedded Linux and eCos as mentioned in the first post. And it's performance gets better as you add better hardware and users as opposed to SQLite

Pico

Welcome aboard, Frederick. Someone (not me) downvoted you with not explanation. I would guess it was because you posted a "link only answer" which is frowned upon. We prefer explanations on our site, in case the link dies. – Maximomaximum

Btw, can you give us an idea of how much it costs? – Maximomaximum

Thanks, Mawg. I've added a better explanation in my original post. I don't think I'm allowed to share the details of what we pay, but it is fairly cheap compared to many of the commercial embedded databases we evaluated. – Pico

Answer 8

I will suggest sqlite3 too. It is used by many famous application.

Luthuli

Answer 9

i am not familiar with embed system, but iphone use arm9, and sqlite as DB

Withershins

Answer 10

The 01-11-10 Embedded.com Newsletter does a nice job of covering this topic. The newsletter can be found at Embedded.com: Embedded.com Tech Focus Newsletter (1-11-10): Embedding Databases.

Michigan