.NET runtime vs. Java Hotspot: Is .NET one generation behind?

Question

According to the information I could gather on .NET and Java execution environment, the current state of affairs is follows:

• Modern Java VM are capable of performing continuous recompilation, which combined with profiling can yield great performance improvements. Older JVMs employed JIT. More information in this article: http://www.ibm.com/developerworks/library/j-jtp12214/ and especially: Java theory and practice: Dynamic compilation and performance measurement

• .NET uses JIT or NGEN to generate native code, but once the native code is generated, no further (runtime) optimizations are performed.

Benchmarks aside and with no intention to escalate holy wars, does this mean that Java Hotspot VM is one generation ahead of .Net. Will these technologies employed at Java VM eventually find its way into .NET runtime?

Hopscotch

The linked article suggests that dynamic recompilation might be a good thing, but that it is difficult to know for sure, since the VM can choose to re-optimize without warning. Without a sensible code profiling, it seems like an article of faith, since there is no proof that "great performance improvements" actually occur. – Moncrief

It seems to be a efficient technique to improve polymorphic dispatch overhead. See also: en.wikipedia.org/wiki/Adaptive_optimization and citeseerx.ist.psu.edu/viewdoc/… – Hopscotch

If that technology was such an improvement over the JIT/NJEN model I'm sure Microsoft would have implemented it. That dissertation is from 1994 and the IBM article is from 2004. – Incorporation

@Matthew, that means basically that you trust more Ms than Sun, who have implemented it. I am looking for some more reliable data, maybe a statement from someone from Ms team (for example they tried it and decided it is not worth it etc.) or maybe an info on current effort from Ms to implement it etc. – Hopscotch

I do trust Microsoft more than Sun. They were smart enough to higher Anders Hejlsberg => en.wikipedia.org/wiki/Anders_Hejlsberg As for why they didn't implement it, go post a question on the CLR Team blog blogs.msdn.com/b/clrteam – Incorporation

Besides, why would I put my career in their hands if I didn't trust them? – Incorporation

Benchmarks are an effective way of measuring whether one platforms compilation method is better than another's. "Benchmarks aside" sounds perhaps a little like "Ignore whether dynamic compilation actually does anything, doesn't it sound better?" – Reine

@kbrimington, if you look into java vs .net benchmarks, you can find a benchmark to favor either side. I don't want this turning into the list of java vs .net benchmarks. If you think that dynamic recompilation has no value, than you are advocating "No" answer to the question. I do wish answers to be informative and argumentative whether they are "No" or "Yes", so if you can give me a benchmark that demonstrates that .NET with dynamic recompilation is slower, please do cite the benchmark. Java performance on the other hand is known to be constantly improving in part thanks to this feature. – Hopscotch

@Dan: Sorry, Dan, haven't got any, or I would've posted as an answer instead. The intent of my comment was to discourage the inference that one technology is ahead of another without regard to benchmarks. @Matthew-Whited's answer develops the idea more thoroughly. – Reine

Answer 1

I've never benchmarked the two to compare, and I'm more familiar with the Sun JVM, I can only speak in general terms about JITs.

There are always tradeoffs with optimizations, and not all optimizations work all the time. However, here are some modern JIT techniques. I think this can be the beginning of a good conversation if we stick to the technical stuff:

• escape analysis
• intrinsics
  • https://bugs.java.com/bugdatabase/view_bug?bug_id=6823354
  • http://weblog.ikvm.net/CommentView.aspx?guid=0404dd8a-88a8-4d62-9bcb-98324d57a2a9
• tail-call optimization
• on-stack replacement
• lock coarsening
• lock elision
• multi-threaded garbage collection
• low-pause garbage collection
• polymorphic method call removal
• fast heap allocation

There's also features that are helpful as far as good implementations of a VM go:

• being able to pick between GC
• implementations customization of each GC
• heap allocation parameters (such as growth)
• page locking

Based on these features and many more, we can compare VMs, and not just "Java" versus ".NET" but, say, Sun's JVM versus IBM's JVM versus .NET versus Mono.

For example, Sun's JVM doesn't do tail-call optimization, IIRC, but IBM's does.

Rennarennane

Answer 2

They follow two different strategies. I do not think one is better than the other.

• .NET does not interpret bytecode, so it has to JIT everything as is gets executed and therefore cannot optimise heavily due to time constraints. If you need heavy optimizations in some part of the code, you can always NGEN it manually, or do a fast but unsafe implementation. Furthermore, calling native code is easy. The approach here seems to be getting the runtime good enough and manually optimise bottlenecks.

• Modern JVMs will usually interpret most of the code, and then do an optimized compilation of the bottlenecks. This usually gets better results than straight JIT'ing, but if you need more, you don't have unsafe in Java, and calling native code is not nice. So the approach here is to do as much automatic optimising as possible, because the other options are not that good.

In reality Java applications tend to perform slightly better in time and worse in space when compared to .NET.

Kimbra

Best answer. Although I'm no Java expert, I think PInvoke's equivalent is the much simpler JNA (en.wikipedia.org/wiki/Java_Native_Access) and not JNI. – Jurdi

Answer 3

Apparently someone was working on something similar for Rotor. I don't have access to IEEE so I can't read the abstract.

Dynamic recompilation and profile-guided optimisations for a .NET JIT compiler

Quote from Summary...

An evaluation of the framework using a set of test programs shows that performance can improve by a maximum of 42.3% and by 9% on average. Our results also show that the overheads of collecting accurate profile information through instrumentation to an extent outweigh the benefits of profile-guided optimisations in our implementation, suggesting the need for implementing techniques that can reduce such overheads.

Incorporation

Interesting point. I wonder if this is why the -server switch seems to make so much of a difference in Java, perhaps they assume that if it doesn't have the -server switch that aggressive recompiling won't be a good trade-off, but with -server you'd assume you might be running for months so you might as well go ahead and make it run as well as possible... – Infusion

Answer 4

You may be interested in SPUR which is a Tracing JIT compiler. The focus is on javascript but it operates on CIL not the language itself. It is a research project based on Bartok not the standard .NET VM. The paper has some performance benchmarks showing 'it consistently performs faster than SPUR-CLR' which is the standard 3.5 CLR. There haven't been any announcements about it's future relating to the current VM however. Traces can cross method boundaries which is not something HotSpot does AFAIK, JVM tracing JITs are mentioned here.

I'd be hesitant to say the .NET VM is a generation behind especially when considering all the sub-systems, in particular generics. How the GC and DLR vs invokedynamic compare I'm unsure but there are lots of details about them at places like channel9.

Asmara