Debug vs. Release performance

Question

I've encountered the following paragraph:

“Debug vs. Release setting in the IDE when you compile your code in Visual Studio makes almost no difference to performance… the generated code is almost the same. The C# compiler doesn’t really do any optimization. The C# compiler just spits out IL… and at the runtime it’s the JITer that does all the optimization. The JITer does have a Debug/Release mode and that makes a huge difference to performance. But that doesn’t key off whether you run the Debug or Release configuration of your project, that keys off whether a debugger is attached.”

The source is here and the podcast is here.

Can someone direct me to a Microsoft article that can actually prove this?

Googling "C# debug vs release performance" mostly returns results saying "Debug has a lot of performance hit", "release is optimized", and "don't deploy debug to production".

Psychoneurotic

possible duplicate of Performance differences between debug and release builds – Spode

With .Net4 on Win7-x86, I have a CPU limited program that I wrote that runs nearly 2x faster in release than debug with no asserts/etc in the main loop. – Plaintiff

Also, if you care about memory use, there can be big differences. I've seen a case where a multi-threaded Windows service compiled in Debug mode used 700MB per thread, vs. 50MB per thread in the Release build. The Debug build quickly ran out of memory under typical usage conditions. – Quadric

@Plaintiff - did you verify that if you attach a debugger to the release build, it still runs 2x faster? Note that the quote above says that JIT optimization is affected by whether debugger is attached. – Crimea

Answer 1

Partially true. In debug mode, the compiler emits debug symbols for all variables and compiles the code as is. In release mode, some optimizations are included:

• unused variables do not get compiled at all
• some loop variables are taken out of the loop by the compiler if they are proven to be invariants
• code written under #debug directive is not included, etc.

The rest is up to the JIT.

Full list of optimizations here courtesy of Eric Lippert.

Butterandeggs

And don't forget about Debug.Asserts! In DEBUG build, if they fail, they will halt the thread and pop up a message box. In release they get not compiled at all. This applies for all methods that have [ConditionalAttribute]. – Utham

The C# compiler does not do tail call optimizations; the jitter does. If you want an accurate list of what the C# compiler does when the optimize switch is on, see blogs.msdn.com/ericlippert/archive/2009/06/11/… – Backstairs

Answer 2

There is no article which "proves" anything about a performance question. The way to prove an assertion about the performance impact of a change is to try it both ways and test it under realistic-but-controlled conditions.

You're asking a question about performance, so clearly you care about performance. If you care about performance then the right thing to do is to set some performance goals and then write yourself a test suite which tracks your progress against those goals. Once you have a such a test suite you can then easily use it to test for yourself the truth or falsity of statements like "the debug build is slower".

And furthermore, you'll be able to get meaningful results. "Slower" is meaningless because it is not clear whether it's one microsecond slower or twenty minutes slower. "10% slower under realistic conditions" is more meaningful.

Spend the time you would have spent researching this question online on building a device which answers the question. You'll get far more accurate results that way. Anything you read online is just a guess about what might happen. Reason from facts you gathered yourself, not from other people's guesses about how your program might behave.

Backstairs

I think you can care about performance yet still have a desire to use "debug". For example, if most of your time is waiting on dependencies, I don't think building in debug mode will make a big difference, yet you have the added benefit of getting line numbers in stack traces, which may help fix bugs faster and make happier users. The point is you have to weigh the pros and cons, and a general "running in debug is slower, but only if you're CPU bound" statement is enough to help with the decision. – Clomp

Answer 3

I can’t comment on the performance but the advice “don’t deploy debug to production” still holds simply because debug code usually does quite a few things differently in large products. For one thing, you might have debug switches active and for another there will probably be additional redundant sanity checks and debug outputs that don’t belong in production code.

Schaller

I agree with you on that issue, but this doesn't answer the main question – Psychoneurotic

@sagie: yes, I’m aware of that but I thought the point was still worth making. – Schaller

Answer 4

From msdn social

It is not well documented, here's what I know. The compiler emits an instance of the System.Diagnostics.DebuggableAttribute. In the debug version, the IsJitOptimizerEnabled property is True, in the release version it is False. You can see this attribute in the assembly manifest with ildasm.exe

The JIT compiler uses this attribute to disable optimizations that would make debugging difficult. The ones that move code around like loop-invariant hoisting. In selected cases, this can make a big difference in performance. Not usually though.

Mapping breakpoints to execution addresses is the job of the debugger. It uses the .pdb file and info generated by the JIT compiler that provides the IL instruction to code address mapping. If you would write your own debugger, you'd use ICorDebugCode::GetILToNativeMapping().

Basically debug deployment will be slower since the JIT compiler optimizations are disabled.

Branham

Answer 5

What you read is quite valid. Release is usually more lean due to JIT optimization, not including debug code (#IF DEBUG or [Conditional("DEBUG")]), minimal debug symbol loading and often not being considered is smaller assembly which will reduce loading time. Performance different is more obvious when running the code in VS because of more extensive PDB and symbols that are loaded, but if you run it independently, the performance differences may be less apparent. Certain code will optimize better than other and it is using the same optimizing heuristics just like in other languages.

Scott has a good explanation on inline method optimization here

See this article that give a brief explanation why it is different in ASP.NET environment for debug and release setting.

Slither

Answer 6

I recently run into a performance issue. The products full list was taking too much time, about 80 seconds. I tuned the DB, improved the queries and there wasn't any difference. I decided to create a TestProject and I found out that the same process was executed in 4 seconds. Then I realized the project was in Debug mode and the test project was in Release mode. I switched the main project to Release mode and the products full list only took 4 seconds to display all the results.

Summary: Debug mode is far more slower than run mode as it keeps debugging information. You should always deploy in Relase mode. You can still have debugging information if you include .PDB files. That way you can log errors with line numbers, for example.

Rhyne

By "run mode" you mean "Release"? – Sectarian

Yes, exactly. Release doesn't have all the debug overhead. – Rhyne

My upvote for being the only answer which provided some real comparisons. – Hearsh

Answer 7

In msdn site...

Release vs. Debug configurations

While you are still working on your project, you will typically build your application by using the debug configuration, because this configuration enables you to view the value of variables and control execution in the debugger. You can also create and test builds in the release configuration to ensure that you have not introduced any bugs that only manifest on one type of build or the other. In .NET Framework programming, such bugs are very rare, but they can occur.

When you are ready to distribute your application to end users, create a release build, which will be much smaller and will usually have much better performance than the corresponding debug configuration. You can set the build configuration in the Build pane of the Project Designer, or in the Build toolbar. For more information, see Build Configurations.

Terina

Answer 8

One thing you should note, regarding performance and whether the debugger is attached or not, something that took us by surprise.

We had a piece of code, involving many tight loops, that seemed to take forever to debug, yet ran quite well on its own. In other words, no customers or clients where experiencing problems, but when we were debugging it seemed to run like molasses.

The culprit was a Debug.WriteLine in one of the tight loops, which spit out thousands of log messages, left from a debug session a while back. It seems that when the debugger is attached and listens to such output, there's overhead involved that slows down the program. For this particular code, it was on the order of 0.2-0.3 seconds runtime on its own, and 30+ seconds when the debugger was attached.

Simple solution though, just remove the debug messages that was no longer needed.

Unkennel

Answer 9

To a large extent, that depends on whether your app is compute-bound, and it is not always easy to tell, as in Lasse's example. If I've got the slightest question about what it's doing, I pause it a few times and examine the stack. If there's something extra going on that I didn't really need, that spots it immediately.

Drambuie

Answer 10

Debug and Release modes have differences. There is a tool Fuzzlyn: it is a fuzzer which utilizes Roslyn to generate random C# programs. It runs these programs on .NET core and ensures that they give the same results when compiled in debug and release mode.

With this tool it was found and reported a lot of bugs.

Mealy

Answer 11

I just explain my own experience.

I work in health industry on several enterprise level applications. We did measure the performance of Debug VS Release (VS 2019 .NET 4.7) Majority of our applications are Web forms and C# based.

The difference between debug and release for various functions and pages in our case was only about a second(in fact less than 1 second.)

Now our business really do not care about this difference, but If I was handling an application for handling Market Stock transactions then even milliseconds were important and that exactly where Release mode becomes important.

Hearsh

Answer 12

Release is faster: I see about a 2.4 times performance increase in RAM data handling and simple but numerous math processes. I just tested this in a program I support for industrial process optimization. It searches an NP hard space for the best handling of situations, and will evaluate roughly a billion situations during a test, in blocks of 1-9 million at a shot. I ran it against a lot of scenarios and summed the time, so my sample was about one trillion flops & integer math operations. Most of my data was in List<> and array format and accessing the elements of these was a lot of the time. My total test time in each debug/release was about 50 minutes vs. 20 minutes. I'm not speaking of GUI performance, server access, or disk access, and instead crunching a ton of numbers in RAM. Of course if a program spends most of its time waiting for users or outside resources (servers) then it's not as big of a difference, but for RAM/array/math number crunching it makes a big difference.

Shaylyn