~1s latency control app: is this suitable for Java?

Question

At my work we recently finished the system architecture for a control application which has a maximum latency of roughly one to two seconds. It is distributed on small ARM on-chip boxes communicating via an IP LAN.

We initially foresee that we would use C or C++, since it is a classical control system language. After discussing how to implement the application we now realize that C++ has a quite limited amount of libraries, lacks introspection, and has some other properties which might slow down development. My colleague then suggested that Java might be up for the job.

I'm really scared about the latency of running a GC for a control app, and I'm also reluctant to drop RAII, since the app will use a lot of external resources (sockets, file handles, handles from external libs etc).

The pro/con list is currently as follows:

C++

+ RAII - Easy resource management - it will be a complex system
+ System language - speed if we cant't find a JIT VM for our ARM
+ No GC - no big worst case latencies from the GC
+ Easy to integrate with some shared mem libs that we have to interface with
- Fewer free as in beer libs 
- Lacks introspection - Mapping classes to DB and external data formats (XML)    
  would benefit from this (ORM /JAXB) approach
- Easy to shoot one self in the foot - hard and expensive to find programmers 
  which don't make big mistakes
- Memory fragmentation - needs tuning and workarounds

Java

+ Huge amount of libs
+ Introspection - serialization becomes a breeze (see C++ section)
+ Easier to find 'good enough' programmers
- No RAII - Client has to remember finally or you leak 
   resources. IMO Java programmers tend to ignore this 
   problem unless they have server app background.
- No System Language - possibly slower although ARMj could alleviate this
- GC - latency might go up (don't know if parallel GC will work - seems that
     you might get fragmentation, see note below).
- Need to write JNI for the shared mem libs that we interface with
- Maybe ORACLE will eat us

The memory fragmentation with parallel GC was mentioned in this AMD article

I would love to use Java if GC latency wasn't a problem and we could get RAII. Therefore I've also looked into other langs which have RAII and could serve as good alternatives, and so far I've found that D,Ada,VB,Perl,Python(C),PHP,tcl and Lua seem to have some sort of out-of-scope callback. My spontainous reaction that perhaps D, Python, and ADA could be suitable for a control app. D and ADA are my favourites.

So, my question is: do you have any recommendations on this? Is Java a viable option, and if you could choose any language, what would it be?

Peculation

@disown: The GC isn't your only ennemy. The JIT kicking in can introduce a latency-spike too, (which was visible at a Google I/O 2010 speak btw, where the new Android JIT device would perform slower at first than the non-JIT one, then faster). I think you may be interested in javolution.org where there are quite a few interesting articles linked. – Fearless

@Webinator: thanks for the link, looking into it now... – Peculation

Answer 1

The GC is only used for reclaiming memory from discarded objects. Discard very little resources and you will get very little, shorter GCs.

You might use lots of sockets, file handles, handles from external libs, but how fast do you discard them?

A full GC is designed to remove fragmentation. It does this by copying all the memory so it is used continously. This is why its is expensive, so you want to minimise these if latency is important to you. That said, if your full GCs are taking more than 100 ms, you have a serious performance issue. It shouldn't be that high.

IMHO, I short you should be able to develop a control system with a latency which is well under 10 ms, 99%+ of the time.

Redeploy

Even if your program is very conservative with resources, how do you guarantee that the libraries that you're calling aren't churning through objects at epic rate? – Vascular

I don't think we can do much pooling, sockets will be to different clients, so we'll be building up and tearing down quite a lot. Maybe caching the connections could help, and only re-init when connections time out. You say 'under 10ms 99% of the time'. What I wonder is if there are any 'worst case' peaks which are much higher. I don't mean in theory, but if you run for a couple of months, what would be your worst case latency, and how does that compare to what you would have in C++. I know its a really hard question to answer, I'm just looking for practical pointers. – Peculation

The only way to guarentee low GC time is to use small numbers of temporary objects. i.e. use a small young gneration size. However this is very difficult to comtrol below 10 ms reliably. Using the new G1 collector is supposed to help control the latency of a full collection but it is still under development. In my experience, you see much high variability in the network than in a carefully written Java application which is why in my role I spend more time looking at network performance than Java performance. – Redeploy

Thank you. That is valuable advice. – Peculation

Answer 2

Probably you need a prove of concept if you wish to use java, I would suggest to write a simple prototye and a stress test, to examine what latency time you have under high load during garbage collection. Then checkout the different collector types i.g. low pause collector .

The RAII argument I don't really understand, in c++ it's IMHO easier to create memory leaks than in java.

Ober

Memory leaks yes, but RAII works also for other resources like DB connections and file handles, where in Jave the only thing you have is try/finally, which is definitely inferior to RAII (which AFAIK requires stack-allocated objects, thus impossible to implement on the JVM). – Navelwort

The low pause collector can apparently cause memory fragmentation, as per the article that I linked in my question. I don't know how severe this is though, and if C++ will be any better. RAII definitely helps not leaking resources, which is why I was looking for a RAII lang. – Peculation

Answer 3

I would stay away from GC (certainly as implemented in any JVM I have seen) in such an environment. You will forever be banging your head against the wall on the problem.

However, I just wanted to point out that the RAII argument seems very weak - you need code reviews and other such training to ensure that such issues are well understood by your team. It is a very bad reason to exclude a language that is otherwise appropriate, as every language has gotchas that an inexperienced/less than stellar programmer can miss.

I felt from your list that you were missing C# (I'm thinking of with Mono) where you have a lot more control when you need it. I don't know if it is appropriate for your environment, but if you list VB on your laundry list of languages, it was an obvious oversight.

Diastase

The laundry list only consisted of languages which could support RAII and have out-of-scope callbacks. C# has 'using', but this is just syntactic sugar for try..finally, which you also have in Java. I don't mean that RAII is a silver bullet, it is just a very handy feature to help programmers to not leak resources. I'm worried that the code will become harder to read if we don't have RAII, that's all. – Peculation

@disown, I was more thinking of C# having most of the Java advantage (with the exception of the libs) with easier integration, and I do think that using is much easier to statically check for to catch the failure to use try/finally. If your issue is readability, then using is also more readable IMO. I'm not really avocating for it, though, just saying that it seems like it deserves consideration. D and ADA may indeed be better choices. – Diastase