Performance differences between debug and release builds
Asked Answered
D

9

301

I must admit, that usually I haven't bothered switching between the Debug and Release configurations in my program, and I have usually opted to go for the Debug configuration, even when the programs are actually deployed at the customers place.

As far as I know, the only difference between these configurations if you don't change it manually is that Debug have the DEBUG constant defined, and Release have the Optimize code checked of.

So my questions is actually twofold:

  1. Are there much performance differences between these two configurations. Are there any specific type of code that will cause big differences in performance here, or is it actually not that important?

  2. Are there any type of code that will run fine under the Debug configuration that might fail under Release configuration, or can you be certain that code that is tested and working fine under the Debug configuration will also work fine under Release configuration.

Doublet answered 28/10, 2010 at 14:23 Comment(1)
Related: stackoverflow.com/questions/33871181/…Duckett
S
544

The C# compiler itself doesn't alter the emitted IL a great deal in the Release build. Notable is that it no longer emits the NOP opcodes that allow you to set a breakpoint on a curly brace. The big one is the optimizer that's built into the JIT compiler. I know it makes the following optimizations:

  • Method inlining. A method call is replaced by the injecting the code of the method. This is a big one, it makes property accessors essentially free.

  • CPU register allocation. Local variables and method arguments can stay stored in a CPU register without ever (or less frequently) being stored back to the stack frame. This is a big one, notable for making debugging optimized code so difficult. And giving the volatile keyword a meaning.

  • Array index checking elimination. An important optimization when working with arrays (all .NET collection classes use an array internally). When the JIT compiler can verify that a loop never indexes an array out of bounds then it will eliminate the index check. Big one.

  • Loop unrolling. Loops with small bodies are improved by repeating the code up to 4 times in the body and looping less. Reduces the branch cost and improves the processor's super-scalar execution options.

  • Dead code elimination. A statement like if (false) { /.../ } gets completely eliminated. This can occur due to constant folding and inlining. Other cases is where the JIT compiler can determine that the code has no possible side-effect. This optimization is what makes profiling code so tricky.

  • Code hoisting. Code inside a loop that is not affected by the loop can be moved out of the loop. The optimizer of a C compiler will spend a lot more time on finding opportunities to hoist. It is however an expensive optimization due to the required data flow analysis and the jitter can't afford the time so only hoists obvious cases. Forcing .NET programmers to write better source code and hoist themselves.

  • Common sub-expression elimination. x = y + 4; z = y + 4; becomes z = x; Pretty common in statements like dest[ix+1] = src[ix+1]; written for readability without introducing a helper variable. No need to compromise readability.

  • Constant folding. x = 1 + 2; becomes x = 3; This simple example is caught early by the compiler, but happens at JIT time when other optimizations make this possible.

  • Copy propagation. x = a; y = x; becomes y = a; This helps the register allocator make better decisions. It is a big deal in the x86 jitter because it has few registers to work with. Having it select the right ones is critical to perf.

These are very important optimizations that can make a great deal of difference when, for example, you profile the Debug build of your app and compare it to the Release build. That only really matters though when the code is on your critical path, the 5 to 10% of the code you write that actually affects the perf of your program. The JIT optimizer isn't smart enough to know up front what is critical, it can only apply the "turn it to eleven" dial for all the code.

The effective result of these optimizations on your program's execution time is often affected by code that runs elsewhere. Reading a file, executing a dbase query, etc. Making the work the JIT optimizer does completely invisible. It doesn't mind though :)

The JIT optimizer is pretty reliable code, mostly because it has been put to the test millions of times. It is extremely rare to have problems in the Release build version of your program. It does happen however. Both the x64 and the x86 jitters have had problems with structs. The x86 jitter has trouble with floating point consistency, producing subtly different results when the intermediates of a floating point calculation are kept in a FPU register at 80-bit precision instead of getting truncated when flushed to memory.

Sheaves answered 28/10, 2010 at 16:33 Comment(6)
I don't think all collections use array(s): LinkedList<T> doesn't, even though it's not used very often.Cultus
I think the CLR configures the FPU to 53-bit precision (matching 64-bit wide doubles), so there should be no 80-bit extended double calculations for Float64 values. However, Float32 computations might be computed at this 53-bit precision and only truncated when stored to memory.Gustin
The volatile keyword does not apply to local variables stored in a stack frame. From the documentation at msdn.microsoft.com/en-us/library/x13ttww7.aspx: "The volatile keyword can only be applied to fields of a class or struct. Local variables cannot be declared volatile."Pluton
as a humble amendment, I guess what really makes the difference between Debug and Release builds in this regard is the "optimize code" checkbox which is normally on for Release but off for Debug. It's just to make sure readers don't start to think that there are "magic", invisible differences between the two build configurations that go beyond what is found on the project property page in Visual Studio.Froth
Perhaps worth mentioning that virtually none of the methods on System.Diagnostics.Debug do anything in a debug build. Also variables don't get finalized quite so quickly see (https://mcmap.net/q/17275/-does-garbage-collection-run-during-debug).Countenance
@Froth - actually, according to what Hans is saying, and quotes elsewhere, what makes the biggest difference for C# isn't the "optimize code" checkbox, it is whether JIT is running in Debug or Release mode. That is determined by whether a debugger is attached, not by that checkbox nor anything done by C# compiler, nor even whether you are building in Debug or Release. If I understand correctly, if you attach a debugger to a Release process, you lose all the optimizations Hans mentions above.Fitly
S
23
  1. Yes, there are many performance differences and these really apply all over your code. Debug does very little performance optimization, and release mode very much;

  2. Only code that relies on the DEBUG constant may perform differently with a release build. Besides that, you should not see any problems.

An example of framework code that depends on the DEBUG constant is the Debug.Assert() method, which has the attribute [Conditional("DEBUG)"] defined. This means that it also depends on the DEBUG constant and this is not included in the release build.

Seifert answered 28/10, 2010 at 14:27 Comment(5)
This is all true, but could you ever measure a difference? Or notice a difference while using a program? Of course I don't want to encourage anyone to release their software in debug mode, but the question was if there is a huge performance difference and I can't see that.Illbred
Also worth noting is that debug versions correlate to the original source code to a much higher degree than release versions. If you think (however unlikely) that someone might try to reverse-engineer your executables, you don't want to make it easier on them by deploying debug versions.Valentine
@Illbred - Well, these days it's difficult. Processors have gotten that fast that the user hardly waits for a process to actually execute code because of a user action, so this is all relative. However, if you're actually doing some lengthy process, yes you will notice. The following code e.g. runs 40% slower under DEBUG: AppDomain.CurrentDomain.GetAssemblies().Sum(p => p.GetTypes().Sum(p1 => p1.GetProperties().Length)).Seifert
Also, if you are on asp.net and use debug instead of release some scripts might be added on you page, such as: MicrosoftAjax.debug.js that has about 7k lines.Vaticide
@Illbred Try for example a reasonably sized, CPU-heavy game (like RTS) made in Unity. When code in the Update() event is the limiter (for example because resolution is low), you easily achieve 10-20% more FPS in release mode.Gustave
L
15

This heavily depends on the nature of your application. If your application is UI-heavy, you probably won't notice any difference since the slowest component connected to a modern computer is the user. If you use some UI animations, you might want to test if you can perceive any noticeable lag when running in DEBUG build.

However, if you have many computation-heavy calculations, then you would notice differences (could be as high as 40% as @Pieter mentioned, though it would depend on the nature of the calculations).

It's basically a design tradeoff. If you're releasing under DEBUG build, then if the users experiences problems, you can get a more meaningful traceback and you can do much more flexible diagnostic. By releasing in DEBUG build, you also avoid the optimizer producing obscure Heisenbugs.

Laurinelaurita answered 28/10, 2010 at 15:2 Comment(0)
J
12
  • My experience has been that medium sized or larger applications are noticeably more responsive in a Release build. Give it a try with your application and see how it feels.

  • One thing that can bite you with Release builds is that Debug build code can sometimes suppress race conditions and other threading-related bugs. Optimized code can result in instruction reordering and faster execution can exacerbate certain race conditions.

Jemie answered 28/10, 2010 at 14:51 Comment(0)
G
11

You should never release a .NET Debug build into production. It may contain ugly code to support Edit-and-Continue or who knows what else. As far as I know, this happens only in VB not C# (note: the original post is tagged C#), but it should still give reason to pause as to what Microsoft thinks they are allowed to do with a Debug build. In fact, prior to .NET 4.0, VB code leaks memory proportional to the number of instances of objects with events that you construct in support of Edit-and-Continue. (Though this is reported to be fixed per https://connect.microsoft.com/VisualStudio/feedback/details/481671/vb-classes-with-events-are-not-garbage-collected-when-debugging, the generated code looks nasty, creating WeakReference objects and adding them to a static list while holding a lock) I certainly don't want any of this kind of debugging support in a production environment!

Galipot answered 28/10, 2010 at 23:5 Comment(1)
I have released Debug builds many times, and never seen a problem. The only difference perhaps, is that our server side application is not a web app supporting a lot of users. But it is a server side application with very high processing load. From my experience the difference between Debug and Release seems completely theoretical. I have never seen any practical difference with any of our apps.Silken
I
5

I would say that

  1. largely depends on your implementation. Usually, the difference is not that huge. I did lots of measurements and often I couldn't see a difference. If you use unmanaged code, lots of huge arrays and stuff like that, the performance difference is slightly bigger, but not a different world (like in C++).

  2. Usually in release code fewer errors are shown (higher tolerance), hence a switch should work fine.

Illbred answered 28/10, 2010 at 14:33 Comment(1)
For code that is IO bound, a release build could easily be no faster that debug.Vanadous
C
4

In my experience, the worst thing that has come out of Release mode are the obscure "release bugs". Since the IL (intermediate language) is optimized in Release mode, there exists the possibility of bugs that would not have manifested in Debug mode. There are other SO questions covering this problem: Common reasons for bugs in release version not present in debug mode

This has happened to me once or twice where a simple console app would run perfectly fine in Debug mode, but given the exact same input, would error out in Release mode. These bugs are EXTREMELY difficult to debug (by definition of Release mode, ironically).

Cioban answered 28/10, 2010 at 14:49 Comment(2)
To follow up, here's an article that gives an example of a Release Bug: codeproject.com/KB/trace/ReleaseBug.aspxCioban
Still its a problem if the application is tested and approved with the Debug settings, even if it suppress errors, if that causes the release build to fail during deployment.Circumcise
P
1
    **Debug Mode:**
    Developer use debug mode for debugging the web application on live/local server. Debug mode allow developers to break the execution of program using interrupt 3 and step through the code. Debug mode has below features:
   1) Less optimized code
   2) Some additional instructions are added to enable the developer to set a breakpoint on every source code line.
   3) More memory is used by the source code at runtime.
   4) Scripts & images downloaded by webresource.axd are not cached.
   5) It has big size, and runs slower.

    **Release Mode:**
    Developer use release mode for final deployment of source code on live server. Release mode dlls contain optimized code and it is for customers. Release mode has below features:
   1) More optimized code
   2) Some additional instructions are removed and developer can’t set a breakpoint on every source code line.
   3) Less memory is used by the source code at runtime.
   4) Scripts & images downloaded by webresource.axd are cached.
   5) It has small size, and runs fast.
Palembang answered 17/12, 2015 at 5:31 Comment(1)
it seems than in release mode sometimes the first elements of a list are not numbered correctly. Also some elements within list are duplicated. :)Stanton
C
0

I know that my answer is VERY late and my answer doesn't exactly what you want but, I thought some solid and simple example to play with would be good. Anyway, this piece of code results in a HUGE difference between Debug and Release. The code is written in C++ on Visual Studio 2019. The code is like this:

#include <iostream>

using namespace std;

unsigned long long fibonacci(int n)
{
    return n < 2 ? n : (fibonacci(n - 1) + fibonacci(n - 2));
}

int main()
{
    int x = 47;

    cout << "Calculating..." << endl;
    cout << "fib(" << x << ") = " << fibonacci(x) << endl;
}

EDIT: Performance Differences in Calculating Fibonacci Sequence

                       Debug        Release         
                C++ x86 C++ x64 C++ x86 C++ x64 C# Debug    C# Release
Time (mSeconds) 99384.9 27799.1 11066.0 11321.5 95233.7 24566.0
Time (Seconds)  99.4    27.8    11.1    11.3    95.2    24.6
Catlett answered 21/9, 2021 at 8:24 Comment(6)
It is a good thought to test such concrete programs. Why didn't you quantify the difference in your comment? How many seconds does it take with and without debugging?Prig
@Prig Actually, I did quantify the results but, I forgot to post them.Catlett
@Prig I ran the program and did the measurements again. What I really didn't get is why C++ x86 (Release) was kinda the fastest among the other?Catlett
Thanks for the numbers! x86 pointers are half the size of x64 pointers, so they occupy less cache, so more stuff fits in cache. That could be one reason why x86 is faster.Prig
@Prig Interesting reason. What about x86 and x64 in Debug mode?Catlett
A guess: They did not bother using the time to optimize x86 debug mode for speed. Generally speed is not the main purpose of debug mode, so you should not a priori expect speed differences between implementations to necessarily have any deeper meaning.Prig

© 2022 - 2024 — McMap. All rights reserved.