Why is it that bytecode might run faster than native code [closed]

Question

Java is slow.

That's more than an "urban legend", it seems to be a fact. You don't use it for live-coding because of latency and you don't use it for clusters/parallel computing. There are thousands of benchmarks out there, specially "Java vs C# vs C++".

http://benchmarksgame.alioth.debian.org/

According to the above site, not only is Java performance almost as good as C (far from the rest), but Scala and Clojure (both functional languages which runs on the JVM) both have a better performance that OCaml, Erlang.

And there are a lot of "Java is faster then X" out there, also (for instance, a question here on SO: Java Runtime Performance Vs Native C / C++ Code?).

So Java seems to be fast, for certain cases. Can someone explain why?

Why is it that bytecode might run faster then native code, in some cases, given dynamic code (Scala, Clojure) and garbage collection? How come if it is faster, there is still latency?

It seems to be a contradiction here, can someone shed light?

Fid

A proper GC is bound to be much faster than any non-trivial manual memory management. – Towrey

Java VM can perform optimizations with the information gathered during application runtime which is not available at compile time. – Combat

Create a manageable amount of objects and you won't get significant GC overhead, if this is a real worry but in 90%+ of cases its not. It is possible to build a system which doesn't GC at all during the day. – Livelihood

@Combat >>Java VM can perform optimizations with the information gathered during application runtime which is not available at compile time<< Note that GCC and the Intel compilers now allow some Profile Guided Optimization PGO – Monobasic

Answer 1

In the book masterminds for programming, James Gosling explained:

James: Exactly. These days we’re beating the really good C and C++ compilers pretty much always. When you go to the dynamic compiler, you get two advantages when the compiler’s running right at the last moment. One is you know exactly what chipset you’re running on. So many times when people are compiling a piece of C code, they have to compile it to run on kind of the generic x86 architecture. Almost none of the binaries you get are particularly well tuned for any of them. You download the latest copy of Mozilla,and it’ll run on pretty much any Intel architecture CPU. There’s pretty much one Linux binary. It’s pretty generic, and it’s compiled with GCC, which is not a very good C compiler.

When HotSpot runs, it knows exactly what chipset you’re running on. It knows exactly how the cache works. It knows exactly how the memory hierarchy works. It knows exactly how all the pipeline interlocks work in the CPU. It knows what instruction set extensions this chip has got. It optimizes for precisely what machine you’re on. Then the other half of it is that it actually sees the application as it’s running. It’s able to have statistics that know which things are important. It’s able to inline things that a C compiler could never do. The kind of stuff that gets inlined in the Java world is pretty amazing. Then you tack onto that the way the storage management works with the modern garbage collectors. With a modern garbage collector, storage allocation is extremely fast.

Revulsive

"we’re beating the really good C and C++ compilers pretty much always"... "GCC is not a very good C compiler"... "With a modern Garbage Collector, allocation is extremely fast"... this is just marketing and trolling. – Imputation

As the person above said - it's just marketing and trolling. Well said. The reason why it's not true – Cotangent

Java dosent even provide a way to use the stack - how can it ever be taken seriously for performance – Tortile

Even now Java performs approximately 1.5 to 2 times slower than C++, and up to 3 times slower than C. As compilers/interpreters mature, they become more efficient, so it's laughable to say a bytecode interpreter is faster than a compiler. Its a fair point that "knowing runtime information can guide an optimization further", but the counterpoint is that distributing a VM to hundreds of platforms requires code to run on a "generic x86 arch". The manpower to finetune for all of them doesn't exist. – Marlenemarler

@Marlenemarler Saying "laughable that a bytecode interpreter is faster than a compiler" indicate you may not have looked into how a modern JVM works. Also the ability to inline method calls automatically if "small enough" so you avoid jumping in and out of the pipeline is one of the reasons that modern JVMs can create very efficient machine code. Further improvements are coming, helping the JVM even more. Unfortunately it still takes a while, so Java is still best suited for long running jobs. – Flautist

@ThorbjørnRavnAndersen Likewise further improvements are coming to compilers, which also can inline quite well. The point I've made about manpower still stands against a bytecode interpreter's granularity; a few platforms may receive the fine tuning to out optimize a compiler, but not all. On those few platforms, it will outperform, but so can a specially tuned compiler. So in a very small subset of cases, a bytecode interpreter outperforms because it automatically applies tuning information (that must be coded into the vm for that platform). – Marlenemarler

@Marlenemarler No. The point is that there is information that is simply not available to the ahead-of-time compilers which the just-in-time compilers can utilize. Method inlining is the stock example because you cannot know for sure if a method is overloaded until you actually execute the code. For Java that might even change through out the lifetime of the JVM. – Flautist

@ThorbjørnRavnAndersen Sure, but you can't say "The potential benefits are much higher" to win an argument against the reality. The benchmark game seems to be well maintained, kept up to date, and uses a fairly credible measurement protocol. It says that you are incorrect. After a certain project scale, it is the source code's efficiency -- not the language -- which determines performance (assuming both languages are in the same general ballpark). – Marlenemarler

Wikipedia also provides support for my argument, noting that although Jake2 executes faster than C on some platforms, it is one of the rare programs that can, and that the benchmarking is not transparent enough to conclude its veracity. – Marlenemarler

@Marlenemarler I had a quick look at the programs in the benchmark game and it appears that they tend to be rather simple and primarily do recursive stuff - in other words: the really beneficial HotSpot optimizations do not apply. – Flautist

Answer 2

Fast JVMs use Just-In-Time (JIT) compilation. The bytecode gets translated into native code on the fly at run time. JIT provides many opportunities for optimization. See this Wikipedia article for more info on JIT.

Saucy

Great article, note especially the HPA-8000 in the History section at the end. I read the paper on this a while ago -- the HPA-8000 essentially emulated itself and was faster by doing so. – Have

Answer 3

JVM's come in many flavours!

The fastest ones compile byte code to native code on the fly, based on performance characteristics being collected. All this require extra memory, so they buy speed at the cost of memory. Also, top speed takes a while to reach, so the benefit does not show for short-lived programs.

Even so, the JamVM interpreter (which is tiny compared to the Oracle JVM) still reaches a top speed of a reasonable fraction of the compiled JVM's.

Regarding garbage collection, the speed again comes from having plenty of memory available. Also the true benefit come from the removal of responsibility from the code to keep track of when an object is not in use anymore. This result in simpler, less error prone programs.

Derickderide

Answer 4

Well this is an old argument. Almost all prevalent as Emacs and VI.(but definitely not as old). I have seen lot of c++ developers providing lots of arguments on why most of the performance benchmarks (especially mentioning how java is as fast as c++__) is skewed and to be honest they have a point.

I do not have enough knowledge or time to go deeper in to how Java could be as fast as C++, but the following are the reason why it could be...

1- When you ask two very capable developers to write code in Java and C++ for a real world problem (same problem), then I would be surprised if java performs faster than C++. C++ when well written uses a fraction of memory Java will use. ( Java objects are slightly more bloated).

2- Java is inherently meant to be a simpler language and it was written to make sure that sub- optimal code is hard to write. By abstracting memory management and also by handling low level optimization, its easy to write good code in Java than c++. (this I believe is the most important thing.... Its difficult to write bad code in Java).On the flip side a good C++ developer could handle memory management much better than automatic GC in java. ( Java stores everything in heap, so uses more memory... )

3- Java compiler has been improved consistently and ideas like hotspot has proved to be better than marketing term. (when JIT was introduced, it was just a marketing term.. according to me.. :))

4- Ergonomics or tailoring the settings based on the underlying operating system parameters makes java handle variation better. So in some environments, its not hard to imaging Java performing as good as C++.

5- Opening up high level concurrency and parallelism api's for java developers also is a reason. Java concurrency package is arguably the easiest way to write high performance code that could leverage today's multi processor environments.

6- As hardware as become more and more cheaper, developer competency has become a bigger factor and that's why I believe that lots of c++ code in the wild is probably slower than Java.

Rodrigues

generics don't remove any type casting at the byte code level, and have no effect on performance. – Turnedon

Yes you are correct. Corrected the answer. – Rodrigues

Answer 5

Java Byte code is much easier to optimize than most native opcode. Because the byte code is restricted, and you can't do some dangerous things, you can optimize more fully. Take pointer aliasing, for instance. http://en.wikipedia.org/wiki/Pointer_aliasing

In c/c++ you can do

char[] somebuffer = getABuffer();
char* ptr = &someBuffer[2];
memcpy(ptr, somebuffer, length);

this makes it difficult to optimize in some cases, because you can't be sure what is referring to what. In Java this kind of thing is not allowed.

in general the code mutations that you can perform on a higher level of abstraction are much more powerful than can be had on object code.

Turnedon