Is a garbage collector (.net/java) an issue for real-time systems?

Question

When building a system which needs to respond very consistently and fast, is having a garbage collector a potential problem?

I remember horror stories from years ago where the typical example always was an action game where your character would stop for a few seconds in mid-jump, when the garbage collector would do its cleanup.

We are some years further, but I'm wondering if this is still an issue. I read about the new garbage collector in .Net 4, but it still seems a lot like a big black box, and you just have to trust everything will be fine.

If you have a system which always has to be quick to respond, is having a garbage collector too big of a problem and is it better to chose for a more hardcore, control it yourself language like c++? I would hate it that if it turns out to be a problem, that there is basically almost nothing you can do about it, other than waiting for a new version of the runtime or doing very weird things to try and influence the collector.

EDIT

thanks for all the great resources. However, it seems that most articles/custom gc's/solutions pertain to the Java environment. Does .Net also have tuning capabilities or options for a custom GC?

Sartain

See ibm.com/developerworks/java/library/j-rtj4/index.html – Detergent

Heck, in the early days of .NET (1 & 1.1) just throwing an exception could cause a detectable pause. That's much better now too. – Rump

Note that "real-time" has a very specific meaning. "Real-time" does not mean "fast"; it means that the system must guarantee to react within a certain time limit. Games are not examples of "real-time" applications. Control systems for robots are examples of real-time systems. You need a special real-time operating system. – Virulence

@Virulence There's subcategory in realtime stuff. Hard-realtime and soft-reatime. What you're talking is hard one, and games are falling into soft one. The difference is — failure on hard one makes people hurt or die, and failure on soft one makes people gets angry or upset. Soft realtime doesn't mean timing control failure is usually acceptable. – Collop

Answer 1

To be precise, garbage collectors are a problem for real-time systems. To be even more precise, it is possible to write real-time software in languages that have automatic memory management.

More details can be found in the Real Time Specification for Java on one of the approaches for achieving real-time behavior using Java. The idea behind RTSJ is very simple - do not use a heap. RTSJ provides for new varieties of Runnable objects that ensure threads do not access heap memory of any kind. Threads can either access scoped memory (nothing unusual here; values are destroyed when the scope is closed) or immortal memory (that exists throughout the application lifetime). Variables in the immortal memory are written over, time and again with new values.

Through the use of immortal memory, RTSJ ensures that threads do not access the heap, and more importantly, the system does not have a garbage collector that preempts execution of the program by the threads.

More details are available in the paper "Project Golden Gate: Towards Real-Time Java in Space Missions" published by JPL and Sun.

Masonmasonic

Those first 2 sentences seem contradictory. Did you miss a 'not' in one of them? – Petta

@Henk, I've stated so intentionally. Several systems cannot be considered applicable for real-time usage merely due to the presence of a garbage collector in them. Not having one, increases chances of being considered as a real-time system. – Masonmasonic

So what other GC based language, or at least, memory managed languages would have such a design? Does something like OCaml or Go fit the bill? – Chandler

Answer 2

I've written games in Java and .NET and never found this to be a big problem. I expect your "horror stories" are based on the garbage collectors of many years ago - the technology really has moved a long way since then.

The only thing I would hesitate to use Java/.NET for on the the basis of garbage collection would be something like embedded programming with hard real time constraints (e.g. motion controllers).

However you do need to be aware of GC pauses and all of the following can be helpful in minimising the risk of GC pauses:

• Minimise new object allocations - while object allocations are extremely fast in modern GC systems, they do contribute to future pauses so should be minimised. You can use techniques like pre-allocating arrays of objects, keeping object pools or using unboxed primitives.
• Use specialized low-latency libraries such as Javalution for heavily used functions and data types. These are designed specifically for real-time / low latency application
• Make sure you are using the best GC algorithm when there are multiple versions available. I've heard good things about the Sun G1 Collector for low latency applications. The best GC systems do most of their collections concurrently so that garbage collections do not have to "stop the world" for very long if at all.
• Tune the GC parameters appropriately. Usually there is a trade-off between overall performance and pause times, you may want to improve the latter at the expense of the former.

If you're very rich, you can of course buy machines with hardware GC support. :-)

Rearmost

Be cautious about object pooling. Java and .Net use generational GC, and pooled objects will end up in the eldest generation. This can force more frequent full collections (as opposed to the shorter 'youngest generation only' collections). – Bade

Answer 3

Yes, garbage must be handled in a deterministic manner in real-time systems.

One approach is to schedule a certain amount of garbage collection time during each memory allocation. This is called "work-based garbage collection." The idea is that in the absence of leaks, allocation and collection should be proportional.

Another simple approach ("time-based garbage collection") is to schedule a certain proportion of time for periodic garbage collection, whether it is needed or not.

In either case, it is possible that a program will run out of usable memory because it is not allowed to spend enough time to do a full garbage collection. This is in contrast to a non-realtime system, which is permitted to pause as long as it needs to in order to collect garbage.

Nymphomania

Answer 4

On a theoretical point of view, garbage collectors are not a problem but a solution. Real-time systems are hard, when there is dynamic memory allocation. In particular, the usual C functions malloc() and free() do not offer real-time guarantees (they are normally fast but have, at least theoretically, "worst cases" where they use inordinate amounts of time).

It so happens that it is possible to build a dynamic memory allocator which offers real-time guarantees, but this requires the allocator to do some heavy stuff, in particular moving some objects in RAM. Object moving implies adjusting pointers (transparently, from the application code point of view), and at that point the allocator is just one small step away from being a garbage collector.

Usual Java or .NET implementations do not offer real-time garbage collection, in the sense of guaranteed response times, but their GC are still heavily optimized and have very short response times most of the time. Under normal conditions, very short average response times are better than guaranteed response times ("guaranteed" does not mean "fast").

Also, note that usual Java or .NET implementations run on operating systems which are not real-time either (the OS can decide to schedule other threads, or may aggressively send some data to a swap file, and so on), and neither is the underlying hardware (e.g. a typical hard disk may make "recalibration pauses" on time to time). If you are ready to tolerate the occasional timing glitch due to the hardware, then you should be fine with a (carefully tuned) JVM garbage collector. Even for games.

Bearden

Answer 5

It is a potential problem, BUT...

Your character might also freeze in the middle of your C++ program while the OS retrieves a page of memory from an overtaxed hard disk. If you are not using a real-time OS on hardware designed to provide concrete performance guarantees, you are never guaranteed performance.

To get a more specific answer, you'd have to ask about a specific implementation of a specific virtual machine. You can use a garbage-collected virtual machine for real-time systems if it provides suitable performance guarantees about garbage collection.

Ramrod

true, yet you can still do something about this. with Garbage collectors it's more tricky to solve. However, there's a lot of new info for me that apparently GC's do offer some choices and tuning options – Sartain

Answer 6

You bet it is a problem. If you are writing low-latency applications you cannot afford the stop-the-world pauses that most garbage collectors impose. Since Java does not allow you to turn off the GC, your only option is to produce no garbage. That can be done and has been done through object pooling and bootstrapping. I wrote a blog article where I talk about this in detail.

Christa

Answer 7

Our company is employing a large .Net-based software application that amongst other things monitors binary sensors over fieldbus networks. In some situations, the sensors activate only for a short amount of time (300 ms) but our software still needs to capture those events as the controlled system will immediately fail when an event is missed. We recently observed increased problems at our customer sites due to the garbage collector running for long timespans (up to 1 second). We are still trying to figure out how to enforce a time limit on the garbage collector. In conclusion of this short story, i would say the garbage collector is a handicap in time critical applications.

Pivotal

1second GCs with a concurrent modern GC? Either .NET doesn't have any useful GCs or you're doing something horribly wrong (code style or configuration). In the java world several JVMs/GCs have much better times there. Azul gets milliseconds GC pauses even for large heaps in the hundreds of GBs. And IBM has some even lower targets for smaller heaps (search for "metronome") – Ruggles