Too Little CPU Utilization in Java

Question

Hey stackoverflow community!

I'm having an issue where a highly involved algorithmic program is using TOO LITTLE cpu utilization: somewhere between 3 and 4%. It is taking very long to return results, and I believe it's just not working hard enough.

Do any of you geniuses have any ideas why this would occur - if anything I would expect 100% utilization. One additional detail is that the program makes inserts into a sqlite3 database, and thus yes, there are a lot of JNI calls via the sqlite3jdbc library I believe. (Note that I wanted to defer these inserts with a PreparedQuery batch earlier, but this caused major memory problems - there's a lot of data).

Thanks in advance

UPDATE: Fixed. Yeah, I was just being a doofus, but I didn't expect that sqlite would start a new transaction and do so much overhead.

I now use a PreparedStatement and queue 32768 entries before insert - seemed like a good enough number to me.

Rishi

Answer 1

Obviously the database calls are causing delays. Isn't it an option to create smaller batches and test if that helps?? Maybe you could parallelize the algorithm as well to have a queue somewhere taking results and another thread cleaning out that queue?

edit:

There are also some other problem areas:

• Database optimalization (model)
• Database server configuration
• Disk speed

All these factors should be taken into account

Tunis

Answer 2

I would never recommend that someone use a JDBC driver with JNI if a type IV, 100% Java version is available. Google found this one.

With that aside, I can't tell anything without more info. Are the app and the database running on the same hardware?

What is so "intensive" about INSERTs?

I'd recommend profiling and getting some real data rather than guessing. Faith-based computing never works for me.

Tymon

Answer 3

If you're writing a lot of data, then it sounds like you may be disk bound. Take a look at your disk io stats on the machine, and if that's actually the bottleneck, either find hardware with better io, or figure out how to do less writes.

Calyces

Answer 4

The disk is slowing down your app. INSERTS use the disk, disk is slow, and the OS needs to wait for the write operations to finish.

Can't you use 2 threads, one for the algorithm, and another for the inserts? If you only make inserts, you may also write then to a text file, and execute them at a later time

Cybele

Multithreaded communication is slower than queueing the updates and flushing them in the same thread, and also more complicated to get bugfree for programmers which aren't "born thread-safe". – Sammysamoan