Why are JIT-ed languages still slower and less memory efficient than native C/C++?

Question

Interpreters do a lot of extra work, so it is understandable that they end up significantly slower than native machine code. But languages such as C# or Java have JIT compilers, which supposedly compile to platform native machine code.

And yet, according to benchmarks that seem legit enough, in most of the cases are still 2-4x times slower than C/C++? Of course, I mean compared to equally optimized C/C++ code. I am well aware of the optimization benefits of JIT compilation and their ability to produce code that is faster than poorly optimized C+C++.

And after all that noise about how good the Java memory allocation is, why such a horrendous memory usage? 2x to 50x, on average about 30x times more memory is being used across that particular benchmark suite, which is nothing to sneeze at...

NOTE that I don't want to start a WAR, I am asking about the technical details which define those performance and efficiency figures.

Distortion

Why did java have the reputation of being slow? – Dyslexia

While both JIT compilers and offline compilers have to balance compilation time vs execution speed, offline compilers can place the balance more on execution speed. – Reneareneau

Answer 1

Some reasons for differences;

• JIT compilers mostly compile quickly and skip some optimizations that take longer to find.

• VM's often enforce safety and this slows execution. E.g. Array access is always bounds checked in .Net unless guaranteed within the correct range

• Using SSE (great for performance if applicable) is easy from C++ and hard from current VM's

• Performance gets more priority in C++ over other aspects when compared to VM's

• VM's often keep unused memory a while before returning to the OS seeming to 'use' more memory.

• Some VM's make objects of value types like int/ulong.. adding object memory overhead

• Some VM's auto-Align data structures a lot wasting memory (for performance gains)

• Some VM's implement a boolean as int (4 bytes), showing little focus on memoryconservation.

Kucera

+1 That's a good answer for the speed part, care to share some insight on the memory side? I know that having a GC and creating objects for everything has nontrivial overhead. Every object has at least one word of overhead (GC metadata), more frequently two. – Resurrectionism

Answer 2

But languages such as C# or Java have JIT compilers, which supposedly compile to platform native machine code.

Interpreters also have to translate to machine code in the end. But JITters spend less effort to compile and optimize for the sake of better start up and execution time. Wasting time on compilation will make the perceived performance worse from the user's point of view so that's only possible when you do only once like in an AoT compiler.

They also have to monitor the compiled result to recompile and optimize the hot spots even more, or de-optimize the paths that are rarely used. And then they have to fire the GC once in a while. Those take more time than a normal compiled binary. Besides, the big memory usage of the JITter and the JITted program may mean less efficient cache usage which in turn also slows the performance down

For more information about memory usage you can reference here

Java memory usage is much heavier than C++'s memory usage because:

• There is an 8-byte overhead for each object and 12-byte for each array in Java (32-bit; twice as much in 64-bit java). If the size of an object is not a multiple of 8 bytes, it is rounded up to next multiple of 8. This means an object containing a single byte field occupies 16 bytes and requires a 4-byte reference. Please note that C++ also allocates a pointer (usually 4 or 8 bytes) for every object that declares virtual functions.
• Parts of the Java Library must be loaded prior to the program execution (at least the classes that are used "under the hood" by the program).[60] This leads to a significant memory overhead for small applications[citation needed].
• Both the Java binary and native recompilations will typically be in memory.
• The virtual machine itself consumes a significant amount of memory.
• In Java, a composite object (class A which uses instances of B and C) is created using references to allocated instances of B and C. In C++ the memory and performance cost of these types of references can be avoided when the instance of B and/or C exists within A.
• Lack of address arithmetic makes creating memory-efficient containers, such as tightly spaced structures and XOR linked lists, impossible.

In most cases a C++ application will consume less memory than the equivalent Java application due to the large overhead of Java's virtual machine, class loading and automatic memory resizing. For applications in which memory is a critical factor for choosing between languages and runtime environments, a cost/benefit analysis is required.

One should also keep in mind that a program that uses a garbage collector can need as much as five times the memory of a program that uses explicit memory management in order to reach the same performance.

Oodles

Answer 3

why such a horrendous memory usage? 2x to 50x, on average about 30x times more memory is being used across that particular benchmark suite, which is nothing to sneeze at...

See https://softwareengineering.stackexchange.com/a/189552

Dissonancy