Time series database for java?

Question

I want to store millions of time series entries (long time, double value) with Java. (Our monitoring system is currently storing every entry in a large mySQL table but performance is very bad.)

Are there time series databases implemented in java out there?

Lice

Performance is bad in what sense? INSERTing? SELECTing? Unless you provide some use cases (i.e. problematic queries) nobody will ever be able to give a meaningful answer... – Ward

Can you please what do you really mean by "Performance is really bad". Millions of inserts should not be a problem for mysql. What are you really trying to do with the data is that table? – Satterlee

Answer 1

• checkout http://opentsdb.net/ as used by StumbleUpon?
• checkout http://square.github.com/cube/ as used by square

I hope to see additional suggestions in this thread.

Dur

Answer 2

The performance was bad because of wrong database design. I am using mysql and the table had this layout:

+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+
| Field       | Type                                 | Null | Key | Default           | Extra                       |
+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+
| fk_category | smallint(6)                          | NO   | PRI | NULL              |                             |
| method      | enum('min','max','avg','sum','none') | NO   | PRI | none              |                             |
| time        | timestamp                            | NO   | PRI | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| value       | float                                | NO   |     | NULL              |                             |
| accuracy    | tinyint(1)                           | NO   |     | 0                 |                             |
+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+

My fault was an inapproriate index. After adding a multi column primary key all my queries are lightning fast:

+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| job   |          0 | PRIMARY  |            1 | fk_category | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| job   |          0 | PRIMARY  |            2 | method      | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| job   |          0 | PRIMARY  |            3 | time        | A         |   452509710 |     NULL | NULL   |      | BTREE      |         |               |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

Thanks for all you answers!

Lice

Answer 3

You can take a look at KDB. It's primarily used by financial companies to fetch market time series data.

Countermark

Answer 4

What do you need to do with the data and when?

If you are just saving the values for later, a plain text file might do nicely, and then later upload to a database.

Day