Why is my C# program faster in a profiler?

Question

I have a relatively large system (~25000 lines so far) for monitoring radio-related devices. It shows graphs and such using latest version of ZedGraph. The program is coded using C# on VS2010 with Win7. The problem is:

• when I run the program from within VS, it runs slow
• when I run the program from the built EXE, it runs slow
• when I run the program though Performance Wizard / CPU Profiler, it runs Blazing Fast.
• when I run the program from the built EXE, and then start VS and Attach a profiler to ANY OTHER PROCESS, my program speeds up!

I want the program to always run that fast!

Every project in the solution is set to RELEASE, Debug unmanaged code is DISABLED, Define DEBUG and TRACE constants is DISABLED, Optimize Code - I tried either, Warning Level - I tried either, Suppress JIT - I tried either, in short I tried all the solutions already proposed on StackOverflow - none worked. Program is slow outside profiler, fast in profiler. I don't think the problem is in my code, because it becomes fast if I attach the profiler to other, unrelated process as well!

Please help! I really need it to be that fast everywhere, because it's a business critical application and performance issues are not tolerated...

UPDATES 1 - 8 follow

--------------------Update1:--------------------

The problem seems to Not be ZedGraph related, because it still manifests after I replaced ZedGraph with my own basic drawing.

--------------------Update2:--------------------

Running the program in a Virtual machine, the program still runs slow, and running profiler from the Host machine doesn't make it fast.

--------------------Update3:--------------------

Starting screen capture to video also speeds the program up!

--------------------Update4:--------------------

If I open the Intel graphics driver settings window (this thing: http://www.intel.com/support/graphics/sb/img/resolution_new.jpg) and just constantly hover with the cursor over buttons, so they glow, etc, my program speeds up!. It doesn't speed up if I run GPUz or Kombustor though, so no downclocking on the GPU - it stays steady 850Mhz.

--------------------Update5:--------------------

Tests on different machines:

-On my Core i5-2400S with Intel HD2000, UI runs slow and CPU usage is ~15%.

-On a colleague's Core 2 Duo with Intel G41 Express, UI runs fast, but CPU usage is ~90% (which isn't normal either)

-On Core i5-2400S with dedicated Radeon X1650, UI runs blazing fast, CPU usage is ~50%.

--------------------Update6:--------------------

A snip of code showing how I update a single graph (graphFFT is an encapsulation of ZedGraphControl for ease of use):

public void LoopDataRefresh() //executes in a new thread
        {
            while (true)
            {
                while (!d.Connected)
                    Thread.Sleep(1000);
                if (IsDisposed)
                    return;
//... other graphs update here
                if (signalNewFFT && PanelFFT.Visible)
                {
                    signalNewFFT = false;
                    #region FFT
                    bool newRange = false;
                    if (graphFFT.MaxY != d.fftRangeYMax)
                    {
                        graphFFT.MaxY = d.fftRangeYMax;
                        newRange = true;
                    }
                    if (graphFFT.MinY != d.fftRangeYMin)
                    {
                        graphFFT.MinY = d.fftRangeYMin;
                        newRange = true;
                    }

                    List<PointF> points = new List<PointF>(2048);
                    int tempLength = 0;
                    short[] tempData = new short[2048];

                    int i = 0;

                    lock (d.fftDataLock)
                    {
                        tempLength = d.fftLength;
                        tempData = (short[])d.fftData.Clone();
                    }
                    foreach (short s in tempData)
                        points.Add(new PointF(i++, s));

                    graphFFT.SetLine("FFT", points);

                    if (newRange)
                        graphFFT.RefreshGraphComplete();
                    else if (PanelFFT.Visible)
                        graphFFT.RefreshGraph();

                    #endregion
                }
//... other graphs update here
                Thread.Sleep(5);
            }
        }

SetLine is:

public void SetLine(String lineTitle, List<PointF> values)
    {
        IPointListEdit ip = zgcGraph.GraphPane.CurveList[lineTitle].Points as IPointListEdit;
        int tmp = Math.Min(ip.Count, values.Count);
        int i = 0;
        while(i < tmp)
        {
            if (values[i].X > peakX)
                peakX = values[i].X;
            if (values[i].Y > peakY)
                peakY = values[i].Y;
            ip[i].X = values[i].X;
            ip[i].Y = values[i].Y;
            i++;
        }
        while(ip.Count < values.Count)
        {
            if (values[i].X > peakX)
                peakX = values[i].X;
            if (values[i].Y > peakY)
                peakY = values[i].Y;
            ip.Add(values[i].X, values[i].Y);
            i++;
        }
        while(values.Count > ip.Count)
        {
            ip.RemoveAt(ip.Count - 1);
        }
    }

RefreshGraph is:

public void RefreshGraph()
    {
        if (!explicidX && autoScrollFlag)
        {
            zgcGraph.GraphPane.XAxis.Scale.Max = Math.Max(peakX + grace.X, rangeX);
            zgcGraph.GraphPane.XAxis.Scale.Min = zgcGraph.GraphPane.XAxis.Scale.Max - rangeX;
        }
        if (!explicidY)
        {
            zgcGraph.GraphPane.YAxis.Scale.Max = Math.Max(peakY + grace.Y, maxY);
            zgcGraph.GraphPane.YAxis.Scale.Min = minY;
        }
        zgcGraph.Refresh();
    }

.

--------------------Update7:--------------------

Just ran it through the ANTS profiler. It tells me that the ZedGraph refresh counts when the program is fast are precisely two times higher compared to when it's slow. Here are the screenshots:

I find it VERY strange that, considering the small difference in the length of the sections, performance differs twice with mathematical precision.

Also, I updated the GPU driver, that didn't help.

--------------------Update8:--------------------

Unfortunately, for a few days now, I'm unable to reproduce the issue... I'm getting constant acceptable speed (which still appear a bit slower than what I had in the profiler two weeks ago) which isn't affected by any of the factors that used to affect it two weeks ago - profiler, video capturing or GPU driver window. I still have no explanation of what was causing it...

Luciennelucier

Does it actually work? My first guess would be that some aspect of the functionality isn't being performed at all when the profiler is attached so the increased speed isn't relevant as it isn't working. Also, some approximate times would help. Is it a 10x speedup, a 5% speedup, is it 5 seconds or 10ms that's the difference between fast and slow, what? Also, what's an acceptable performance level for the application? – International

In addition to all the points @International already mentioned: I am wondering what fast and slow actually refers to. UI updating? Some algorithm? Some sort of data throughput? – Ape

is the profiler in sampling or instrumentation mode ? – Subset

@International - Yes, every aspect of the program works. The performance difference is roughly in the lines of 10-50 times faster. It's Very fast under the profiler. The needed performance level is Realtime or as close to that as possible. Delays of up to 50ms between receiving and seeing the data are acceptable but the less - the better. – Luciennelucier

@Luciennelucier - UI refreshing is noticably slower. TCP communication is noticably slower. Performing decoding and other algorithmic tasks is almost the same (not slower). – Luciennelucier

@Subset - Sampling mode. – Luciennelucier

I don't understand why this happen except that the profiler keep CPU cache warm, pin accessed page in working set by periodically interrupt and access user mode stack, and code pages. as far as other process is conerned that also speed up, I guess that's because most applications share a large portion of system dlls, so other process's profiling cause these dlls kept in RAM, this indirectly decrease page fault of your process. – Subset

I know it's hard to decompose a complex application, but see if you can branch it and start removing features until it behaves the same in both modes. See if you can isolate the cause that way. – Blanketing

@Blanketing - I tried that. Out of 10 data diagrams that I show, I disabled 9 (didn't get their data through the TCP, didn't run any decoding functions, didn't try to refresh their graph controls). The one remaining did speed up, but it sped up under both scenarios, so the difference remained. Even if I disable all graphs (so the screen stays white), the poor performance remains. I ran timers against most functions, all give 0-50ms delay max. Yet the program is sluggish and UI refreshes once per second. – Luciennelucier

Have you tried reproducing this on a different computer, not just in a VM? It could be related to your graphics card, possibly. – Blanketing

@Blanketing - on my Core i5-2400S with Intel HD2000, UI runs slow and CPU usage is ~15%. On a colleague's Core 2 Duo with Intel G41 Express, UI runs fast, but CPU usage is ~90% (which isn't normal either). – Luciennelucier

@Blanketing - on Core i5-2400S with dedicated Radeon X1650, UI runs blazing fast, CPU usage is ~50%. – Luciennelucier

In your Update2 you are able to run a profiler even when it is running slow. Did you take a look at the numbers of the profiler to find the lack of performance? – Fretful

@Fretful - in Update2 the program is running in a VM, and the profiler is running on the host machine, attached not to my program, but to another random process, like StickyNotes, or Explorer. So I don't get the actual numbers for my program, but for StickyNotes. The aim of that update is to show that the weird relation where profiler speeds up my program even if attached to an unrelated process, doesn't hold true when my app is in a virtual machine (but still holds when both run on the host) – Luciennelucier

@Daniel: Okay, didn't read carefully enough. Next try: You are using a lock (d.fftDataLock). So how is the interference between the other resources that are using the same lock? What happens if you remove the lines about calling the graphFFT.Refresh/RefreshComplete method? – Fretful

@Daniel: Minor issue (maybe only in strip down code): You create an empty short[] tempData that will be replaced within the lock not filled up, so you can remove the = new short[2048]. Or better instantiate the buffer outside the while(true) once and instead of .Clone() use .CopyTo(). – Fretful

@Fretful - Clone is way faster than CopyTo, I tried CopyTo initially, but since it copies every element to a new memory location, and there are 2k elements, it can't manage to execute once every 1/10ms. But I think you're right about the "new short[2048]", correcting it now... – Luciennelucier

@Daniel: Never made a performance test about .CopyTo() but i though internally it would also make a memcpy and not iterating over each element. – Fretful

@Fretful - If I remove graphFFT.Refresh(), I get no graphs on the display, and can't tell if the performance is good or not - my eye can only catch the difference if the graphs are running. The only other thread that uses the lock is the one that receives the data through TCP and sets it in the buffer "d.fftData". Without the lock I get mixed readings - half of the graph is from one update, next half is from another. – Luciennelucier

@Fretful - Clone() doesn't do memcpy, it returns pointers. edit: my mistake, sorry, CopyTo also does shallow copy according to MSDN. My performance difference between the two must have come from something else... – Luciennelucier

@Daniel: Replace the graph displaying by a simple text message, written into a label (but don't use label.Text += myText, only label.Text = text) maybe with a time stamp etc. to see the flickering. I would bet the problem is the graphics driver and how the graph library invokes it to draw something (GDI, openGL, DirectX, etc.) and how good the driver of the graphic card is to act on these commands. – Fretful

How many graphs are there in the application? – Encyclopedic

@newStackExchangeInstance - There are 10 data vies, 6 of which are graphs, the others are also sort of visual representations. Only 4 of those are usually visible at a time. Those which are not visible are not processed at all (data is not decoded). I tried showing all 10 at the same time, and also showing only 1. The difference was minimal. – Luciennelucier

Perhaps some part of your code depends on the system-wide timer resolution? The profiler probably increases the timer frequency, so anything that uses that (for example, waiting on wait handles, Thread.Sleep, Timer etc.) will be that much more accurate. This would explain both the cases for speedup - WPF applications for example will increase the timer resolution when needed (such as when doing smooth animations) and then revert it back (when the animation stops). Profilers would obviously want more accurate timers as well. – Deceitful

Have you considered Power Saving settings in your OS? Sometimes the processor will be clocked up/down depending on usage... set your windows to 'High Performance' power plan and check any BIOS settings relating to this too? – Portingale

Oh, lord, @Milney, this was almost 2 years ago! And yes, I did check all that, no effect whatsoever. – Luciennelucier

Answer 1

Luaan posted the solution in the comments above, it's the system wide timer resolution. Default resolution is 15.6 ms, the profiler sets the resolution to 1ms.

I had the exact same problem, very slow execution that would speed up when the profiler was opened. The problem went away on my PC but popped back up on other PCs seemingly at random. We also noticed the problem disappeared when running a Join Me window in Chrome.

My application transmits a file over a CAN bus. The app loads a CAN message with eight bytes of data, transmits it and waits for an acknowledgment. With the timer set to 15.6ms each round trip took exactly 15.6ms and the entire file transfer would take about 14 minutes. With the timer set to 1ms round trip time varied but would be as low as 4ms and the entire transfer time would drop to less than two minutes.

You can verify your system timer resolution as well as find out which program increased the resolution by opening a command prompt as administrator and entering:

powercfg -energy duration 5

The output file will have the following in it somewhere:

Platform Timer Resolution:Platform Timer Resolution The default platform timer resolution is 15.6ms (15625000ns) and should be used whenever the system is idle. If the timer resolution is increased, processor power management technologies may not be effective. The timer resolution may be increased due to multimedia playback or graphical animations. Current Timer Resolution (100ns units) 10000 Maximum Timer Period (100ns units) 156001

My current resolution is 1 ms (10,000 units of 100nS) and is followed by a list of the programs that requested the increased resolution.

This information as well as more detail can be found here: https://randomascii.wordpress.com/2013/07/08/windows-timer-resolution-megawatts-wasted/

Here is some code to increase the timer resolution (originally posted as the answer to this question: how to set timer resolution from C# to 1 ms?):

public static class WinApi
{
    /// <summary>TimeBeginPeriod(). See the Windows API documentation for details.</summary>

    [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Interoperability", "CA1401:PInvokesShouldNotBeVisible"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Security", "CA2118:ReviewSuppressUnmanagedCodeSecurityUsage"), SuppressUnmanagedCodeSecurity]
    [DllImport("winmm.dll", EntryPoint = "timeBeginPeriod", SetLastError = true)]

    public static extern uint TimeBeginPeriod(uint uMilliseconds);

    /// <summary>TimeEndPeriod(). See the Windows API documentation for details.</summary>

    [System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Interoperability", "CA1401:PInvokesShouldNotBeVisible"), System.Diagnostics.CodeAnalysis.SuppressMessage("Microsoft.Security", "CA2118:ReviewSuppressUnmanagedCodeSecurityUsage"), SuppressUnmanagedCodeSecurity]
    [DllImport("winmm.dll", EntryPoint = "timeEndPeriod", SetLastError = true)]

    public static extern uint TimeEndPeriod(uint uMilliseconds);
}

Use it like this to increase resolution :WinApi.TimeBeginPeriod(1);

And like this to return to the default :WinApi.TimeEndPeriod(1);

The parameter passed to TimeEndPeriod() must match the parameter that was passed to TimeBeginPeriod().

Infinite

Answer 2

There are situations when slowing down a thread can speed up other threads significantly, usually when one thread is polling or locking some common resource frequently.

For instance (this is a windows-forms example) when the main thread is checking overall progress in a tight loop instead of using a timer, for example:

private void SomeWork() {
  // start the worker thread here
  while(!PollDone()) {
    progressBar1.Value = PollProgress();
    Application.DoEvents(); // keep the GUI responisive
  }
}

Slowing it down could improve performance:

private void SomeWork() {
  // start the worker thread here
  while(!PollDone()) {
    progressBar1.Value = PollProgress();
    System.Threading.Thread.Sleep(300); // give the polled thread some time to work instead of responding to your poll
    Application.DoEvents(); // keep the GUI responisive
  }
}

Doing it correctly, one should avoid using the DoEvents call alltogether:

private Timer tim = new Timer(){ Interval=300 };

private void SomeWork() {
  // start the worker thread here
  tim.Tick += tim_Tick;
  tim.Start();
}

private void  tim_Tick(object sender, EventArgs e){
  tim.Enabled = false; // prevent timer messages from piling up
  if(PollDone()){
    tim.Tick -= tim_Tick;
    return;
  }
  progressBar1.Value = PollProgress();
  tim.Enabled = true;
}

Calling Application.DoEvents() can potentially cause allot of headaches when GUI stuff has not been disabled and the user kicks off other events or the same event a 2nd time simultaneously, causing stack climbs which by nature queue the first action behind the new one, but I'm going off topic.

Probably that example is too winforms specific, I'll try making a more general example. If you have a thread that is filling a buffer that is processed by other threads, be sure to leave some System.Threading.Thread.Sleep() slack in the loop to allow the other threads to do some processing before checking if the buffer needs to be filled again:

public class WorkItem { 
  // populate with something usefull
}

public static object WorkItemsSyncRoot = new object();
public static Queue<WorkItem> workitems = new Queue<WorkItem>();

public void FillBuffer() {
  while(!done) {
    lock(WorkItemsSyncRoot) {
      if(workitems.Count < 30) {
        workitems.Enqueue(new WorkItem(/* load a file or something */ ));
      }
    }
  }
}

The worker thread's will have difficulty to obtain anything from the queue since its constantly being locked by the filling thread. Adding a Sleep() (outside the lock) could significantly speed up other threads:

public void FillBuffer() {
  while(!done) {
    lock(WorkItemsSyncRoot) {
      if(workitems.Count < 30) {
        workitems.Enqueue(new WorkItem(/* load a file or something */ ));
      }
    }
    System.Threading.Thread.Sleep(50);
  }
}

Hooking up a profiler could in some cases have the same effect as the sleep function.

I'm not sure if I've given representative examples (it's quite hard to come up with something simple) but I guess the point is clear, putting sleep() in the correct place can help improve the flow of other threads.

---------- Edit after Update7 -------------

I'd remove that LoopDataRefresh() thread altogether. Rather put a timer in your window with an interval of at least 20 (which would be 50 frames a second if none were skipped):

private void tim_Tick(object sender, EventArgs e) {
  tim.Enabled = false; // skip frames that come while we're still drawing
  if(IsDisposed) {
    tim.Tick -= tim_Tick;
    return;
  }

  // Your code follows, I've tried to optimize it here and there, but no guarantee that it compiles or works, not tested at all

  if(signalNewFFT && PanelFFT.Visible) {
    signalNewFFT = false;

    #region FFT
    bool newRange = false;
    if(graphFFT.MaxY != d.fftRangeYMax) {
      graphFFT.MaxY = d.fftRangeYMax;
      newRange = true;
    }
    if(graphFFT.MinY != d.fftRangeYMin) {
      graphFFT.MinY = d.fftRangeYMin;
      newRange = true;
    }

    int tempLength = 0;
    short[] tempData;

    int i = 0;

    lock(d.fftDataLock) {
      tempLength = d.fftLength;
      tempData = (short[])d.fftData.Clone();
    }

    graphFFT.SetLine("FFT", tempData);

    if(newRange) graphFFT.RefreshGraphComplete();
    else if(PanelFFT.Visible) graphFFT.RefreshGraph();
    #endregion

    // End of your code

    tim.Enabled = true; // Drawing is done, allow new frames to come in.
  }
}

Here's the optimized SetLine() which no longer takes a list of points but the raw data:

public class GraphFFT {
    public void SetLine(String lineTitle, short[] values) {
      IPointListEdit ip = zgcGraph.GraphPane.CurveList[lineTitle].Points as IPointListEdit;
      int tmp = Math.Min(ip.Count, values.Length);
      int i = 0;
      peakX = values.Length;

      while(i < tmp) {
        if(values[i] > peakY) peakY = values[i];
        ip[i].X = i;
        ip[i].Y = values[i];
        i++;
      }
      while(ip.Count < values.Count) {
        if(values[i] > peakY) peakY = values[i];
        ip.Add(i, values[i]);
        i++;
      }
      while(values.Count > ip.Count) {
        ip.RemoveAt(ip.Count - 1);
      }
    }
  }

I hope you get that working, as I commented before, I hav'nt got the chance to compile or check it so there could be some bugs there. There's more to be optimized there, but the optimizations should be marginal compared to the boost of skipping frames and only collecting data when we have the time to actually draw the frame before the next one comes in.

If you closely study the graphs in the video at iZotope, you'll notice that they too are skipping frames, and sometimes are a bit jumpy. That's not bad at all, it's a trade-off you make between the processing power of the foreground thread and the background workers.

If you really want the drawing to be done in a separate thread, you'll have to draw the graph to a bitmap (calling Draw() and passing the bitmaps device context). Then pass the bitmap on to the main thread and have it update. That way you do lose the convenience of the designer and property grid in your IDE, but you can make use of otherwise vacant processor cores.

---------- edit answer to remarks --------

Yes there is a way to tell what calls what. Look at your first screen-shot, you have selected the "call tree" graph. Each next line jumps in a bit (it's a tree-view, not just a list!). In a call-graph, each tree-node represents a method that has been called by its parent tree-node (method).

In the first image, WndProc was called about 1800 times, it handled 872 messages of which 62 triggered ZedGraphControl.OnPaint() (which in turn accounts for 53% of the main threads total time).

The reason you don't see another rootnode, is because the 3rd dropdown box has selected "[604] Mian Thread" which I didn't notice before.

As for the more fluent graphs, I have 2nd thoughts on that now after looking more closely to the screen-shots. The main thread has clearly received more (double) update messages, and the CPU still has some headroom.

It looks like the threads are out-of-sync and in-sync at different times, where the update messages arrive just too late (when WndProc was done and went to sleep for a while), and then suddenly in time for a while. I'm not very familiar with Ants, but does it have a side-by side thread timeline including sleep time? You should be able to see what's going on in such a view. Microsofts threads view tool would come in handy for this:

Teillo

Yes, I've tried that- giving Thread.Sleep(500) to the thread that updates my ZedGraphs gives a positive effect on UI responsiveness, but makes the graphs much slower, and I need them to be close to realtime. Anything below 500ms has diminishing returns. – Luciennelucier

Usually Sleep(200) (5 updates per second) is enough to be perceived as real-time. Sleep(33) would give you 30 frames per second which is great for video (but usually overkill for an app). Check how you are updating the ZedGraphs. Probably your worker threads are waiting on GDI+ to render bitmaps which shouldn't be necessary. You should be collecting the data quickly without blocking the worker threads more than necessary. Once you have the raw data let the worker threads continue while your main thread updates the graphs and renders them (without blocking shared resources in the process). – Teillo

I just read your updates, try dragging an icon from your desktop over the app, if it speeds up while your mouse is moving, then it's because WndProc is triggered to process messages more frequently (checking if the mouse cursor should change when the item might become dropable on some control). In that case your workers are dependent on the WndProc messages being processed (and are probably working in the main thread?). Are you using a timer to trigger the work? – Teillo

Nothing significant happens if I drag an icon. No, I'm using a constantly looping thread that checks a flag for whether fresh data has arrived. Then it reads it from a buffer and sets it on the ZedGraph. But I also tried directly calling this function when data arrives (no loop), and the result is the same. – Luciennelucier

Sorry for double comment - there is a very visible difference between 5ms and 200ms gap between refreshes. 10ms is the longest that still feels "realtime". It makes sense what you say that the threads are waiting on GDI+. I update the ZedGraphs by calling .Refresh() on the zedGraphControl object, on the abovementioned looping thread (not the main thread). Is this incorrect? How can I avoid waiting on GDI+? – Luciennelucier

You should not call Refresh() in any other thread than your main thread. It will be running in your main thread any way. The fact that it is not throwing cross-thread exceptions (what I actually would expect) means it is most likely doing a if (InvokeRequired) check itself and delegating the action to your main thread via the message pump (WndProc) by calling Invoke(). The bad thing is that it blocks while waiting until your main thread has processed the message and executed the Refresh() function. Try to refactor so GUI related stuff stays in your main thread while keeping workers clear. – Teillo

Ok, but in case I have 5 device windows open? MainThread can't possibly cope with that much load. +It's not just the refresh, there's a whole lot decoding and buffering happening immediately before I call Refresh() on the ZedGraph. I've updated the question with a code snip. Also, I tried a quick edit where LoopDataRefresh executes on the main thread, and calls Application.DoEvents() regularly, but the program freezes. – Luciennelucier

I did a quick scan of your posted code but don't see anything significantly wrong. Try replacing zgcGraph.Refresh(); with zgcGraph.Invalidate(); and have a timer in your main thread do a zgcGraph.Update(). I'll see if I can scan your code more thoroughly this weekend. – Teillo

I tried that initially, but it gets even slower. The main thread seems overwhelmed by all the messages that ZedGraph pumps (see IdeaHat's reply here: social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/… ) – Luciennelucier

Your update7 proves that the draw method is being executed in your main thread (see Program.Main() at the top of the stack). In the fast part, it's probably skipping frames. It just depends on the frequency WndProc kicks in related to the frequency that your thread is calling update() (which doesn't seem to be blocking). I've been told that the WndProc frequency is more or less related to the screen's refresh rate (but I doubt it). In any case, I think skipping frames should not diminish the "real time" look and feel. Check out these graphs. – Teillo

No, that's not it, Program.Main() is on top of the 'stack' because this list is sorted by function runtime inclusive, and Program.Main() runs from the start of the program till the end, which makes it the longest running function. Anyway, it doesn't skip frames because the graph becomes visibly more fluent. It runs like those on the link you posted when it runs fast. – Luciennelucier

Daniel, that hierarchy really is a stack trace. The methods on the same level (siblings in the TreeView) are sorted by time including children (children (calls higher on the stack) are dented in). Your call to Update() isn't there. It starts from WndProc() to Paint() to Draw(). If you scroll down to another root-level node you should find your other thread starting with LoopDataRefresh(), and a call to RefreshGraph() one level under it, Thread.Sleep() will probably be the first item under LoopDataRefresh(). Be assured, all drawing is done in the main thread. 'Fast' is skipping frames! – Teillo

There is no other root-level node as far as I can remember (I'm at home now, will post back form work tomorrow). From what I gathered, nothing in this list is either sorted, or grouped, by thread, and there is no way to tell which thread executes which function - Only how much time it took. But even if the Main thread does the drawing (which I also believe and never argued), how does that mean it skips frames? If I can literally see more frames (more fluent movement)? – Luciennelucier

Ah, yes, I see :) Also yes, I know of Concurrency Profiler's Thread View and have been using it, unfortunately it gives me only one side of the medal, because the program runs fast always if any VisualStudio profiler is running. So I have nothing to compare the data to. There are no apparent thread blocks as far as I can see (no cascading execution blocks or repetitive lock patterns) – Luciennelucier

Try running Ants in "wall clock" mode instead of "CPU time", and look at the other threads, are they using as much CPU as they can or are they also sleeping? – Teillo

Unfortunately, for a few days now, I'm unable to reproduce the issue... I'm getting constant acceptable speed (which still appear a bit slower than what I had in the profiler two weeks ago) which isn't affected by any of the factors that used to affect it two weeks ago - profiler, video capturing or GPU driver window. I still have no explanation of what was causing it... – Luciennelucier

Answer 3

When I have never heard or seen something similar; I’d recommend the common sense approach of commenting out sections of code/injecting returns at tops of functions until you find the logic that’s producing the side effect. You know your code and likely have an educated guess where to start chopping. Else chop mostly all as a sanity test and start adding blocks back. I’m often amazed how fast one can find those seemingly impossible bugs to track. Once you find the related code, you will have more clues to solve your issue.

Complacent

I did try that but failed to isolate any problematic sections. I have a feeling that whatever the cause of this, it's not something in my code. Perhaps something about ZedGraph. Because it makes absolutely no sense if I made a mistake somewhere, that actions completely irrelevant to my running app would fix it (like attaching a profiler to notepad.exe) – Luciennelucier

I would try and comment out much more, even if the app starts with an empty window. It's the ultimate trial and error approach guaranteed to produce steady results but you have to commit to commenting out everything if you must. – Complacent

Answer 4

There is an array of potential causes. Without stating completeness, here is how you could approach your search for the actual cause:

• Environment variables: the timer issue in another answer is only one example. There might be modifications to the Path and to other variables, new variables could be set by the profiler. Write the current environment variables to a file and compare both configurations. Try to find suspicious entries, unset them one by one (or in combinations) until you get the same behavior in both cases.

• Processor frequency. This can easily happen on laptops. Potentially, the energy saving system sets the frequency of the processor(s) to a lower value to save energy. Some apps may 'wake' the system up, increasing the frequency. Check this via performance monitor (permon).

• If the apps runs slower than possible there must be some inefficient resource utilization. Use the profiler to investigate this! You can attache the profiler to the (slow) running process to see which resources are under-/ over-utilized. Mostly, there are two major categories of causes for too slow execution: memory bound and compute bound execution. Both can give more insight into what is triggering the slow-down.

If, however, your app actually changes its efficiency by attaching to a profiler you can still use your favorite monitor app to see, which performance indicators do actually change. Again, perfmon is your friend.

Houseboat

Answer 5

If you have a method which throws a lot of exceptions, it can run slowly in debug mode and fast in CPU Profiling mode.

As detailed here, debug performance can be improved by using the DebuggerNonUserCode attribute. For example:

[DebuggerNonUserCode]
public static bool IsArchive(string filename)
{
    bool result = false;
    try
    {
        //this calls an external library, which throws an exception if the file is not an archive
        result = ExternalLibrary.IsArchive(filename);
    }
    catch
    {

    }
    return result;
}

Dirt