Ngen vs RyuJIT - fastest x64 running code when (pre-)startup does not matter

Question

Are Ngen and RyuJIT two completely unrelated things under .NET 4.6 (especially with different optimization techniques and algorithms)?

What produces fastest (better optimized) x64 native code if we do not care about the cost of jitting itself and/or cold/warm startup time?

We are running a long running server app. The continuous running phase is very important performance-wise. The (pre-)startup phase is unimportant for us. Until now we have been on .NET 4.5 and always generated native images by Ngen. We are now in process of upgrading to .NET 4.6 and we want to be sure that this doesn't downgrade the performance of our continuous running phase. I have read some info that RyuJIT is great choice for improving the JITing time, but that the jited code can be less optimized compared to Ngen - see e.g. this github comment on one of the RyuJIT bugs.

Cardon

Answer 1

There is not enough difference between NGen and RyuJIT to make you happy. They do very different jobs, NGen jits ahead-of-time and RyuJIT jits in-time while the process is running. But NGen does not have its own jitter, it asks RyuJIT to get the job done. The generated machine code is not fundamentally different. There are a few optimizations that cannot be done up front, NGen-ed code is slightly slower.

Technically NGen could do a better job since the optimizer could spend more time analyzing the code and trying to find the best possible optimization. But Microsoft does not take advantage of this. It is not completely crystal why they don't but surely has something to do with their 1-800 support phone number. Code optimization is always the riskiest part of a code generator and the bugs in the existing jitters have always been optimization bugs. That this might change some day is not unthinkable.

You'd be ahead when you could take advantage of .NET Native. It generates code ahead-of-time with the back-end of the C++ compiler. But currently, and surely for quite a while to come, it is only supported for packaged apps. The kind that are delivered through the Windows Store, you'll have to target Store, Phone or Universal and use the Store as the deployment vehicle. The package is very important to make .NET Native work, only decent way that it can see what code needs to be translated. And it often still needs help to get it right, Reflection is a difficult problem to solve, the reason that you have it on your machine. Note that the same problem doesn't exist for NGen, it still relies on the jitter to get some code jitted in-time. Like Reflection target code and generics. That this might change some day is not unthinkable.

As noted, NGen code is slight slower. So if you don't care about warm-start delays then you don't want to use NGen.

Last but not least, RyuJIT does not generate faster code than its predecessor. Which already did a very decent job of optimizing. Too decent. The RyuJIT project was started to fix problems in the legacy x64 jitter, the kind that were pretty fundamental in the code base and could only be solved with a drastic rewrite. Optimization was one of them, it had no upper-bound on the amount of time it spent on it. Giving it very unreasonable jitting times on large methods. So if you want to squeeze the last ounce then intentionally disabling RyuJIT so it falls back to the legacy x64 jitter is something you ought to try.

Focalize

Hans, you praising the code quality of the .NET JITs breaks my heart. – Exercitation

@Hans Passant Thank you for a very detailed answer! 'NGen-ed code is slightly slower.' - you mean just the startup phase (due to additional files on disk that needs to be found and load into memory) or are there other reasons? Unfortunately we have some critical paths of code (order sending) that must perform quick even the first execution (JIT profile might be the way to go - but Ngen is still much more reliable in this) – Cardon

@Exercitation - you are reading my posts with gray colored glasses. Praise?? I did not skip .NET Native. – Focalize

@Cardon - no, the machine code is slightly slower. I avoided talking about cold and warm start since you indicated that you were not concerned. I have to say, it is rather odd that you are so concerned about speed but do not appear to have any idea whether your code is fast enough. Running perf tests with real datasets and using a profiler to find hotspots is mandatory. Nobody is going to do that for you. – Focalize

@Hans Passant: Valid point. However it's usually hard to give full relevant and concise context in Qs (so it might be bit misleading). We measured the critical path in 4.5 (with ~200us 1st execution time with Ngen vs some dozens ms without it). We have ongoing stats evaluation on other code performance. Those prevented us from upgrading to 4.5.1 & 4.5.2. However those evaluate rather overall performance. Any specific information topic can help focus newly added metrics (especially now when it's not only about deciding whether or not to upgrade as we need new GC features). – Cardon

Hmm, you are measuring the jitting overhead time. It is completely irrelevant on a "long running server". – Focalize

"NGen could do a better job since the optimizer could spend more time analyzing the code" or it could do a worse job as it cannot use runtime profiling data, a la HotSpot. My understanding is that Microsoft doesn't use that approach, though. – Amphisbaena