Win32Exception Not enough storage is available to process this command

Question

Through my automated crash collection for MaxTo I got the following crash report:

V8.12.0.0 - System.ComponentModel.Win32Exception - :Void UpdateLayered():0
Version: MaxTo8.12.0.0
Exception: System.ComponentModel.Win32Exception
Error message: Not enough storage is available to process this command
Stack trace: 
  at System.Windows.Forms.Form.UpdateLayered()
  at System.Windows.Forms.Form.OnHandleCreated(EventArgs e)
  at System.Windows.Forms.Control.WmCreate(Message& m)
  at System.Windows.Forms.Control.WndProc(Message& m)
  at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
  at System.Windows.Forms.ContainerControl.WndProc(Message& m)
  at System.Windows.Forms.Form.WmCreate(Message& m)
  at System.Windows.Forms.Form.WndProc(Message& m)
  at MaxTo.MainForm.WndProc(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
  at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Another stacktrace:

Version: MaxTo2009.9.0.0
Exception: System.ComponentModel.Win32Exception
Error message: Not enough storage is available to process this command
Stack trace: 
  at System.Windows.Forms.Form.UpdateLayered()
  at System.Windows.Forms.Form.OnHandleCreated(EventArgs e)
  at System.Windows.Forms.Control.WmCreate(Message& m)
  at System.Windows.Forms.Control.WndProc(Message& m)
  at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
  at System.Windows.Forms.ContainerControl.WndProc(Message& m)
  at System.Windows.Forms.Form.WmCreate(Message& m)
  at System.Windows.Forms.Form.WndProc(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
  at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

In this latest stack trace there is no reference to MaxTo at all, and 90% of the crashes I get are with stack traces similar to the above.

Reading around on the net I find that this error is usual if you forget to release or dispose variables. When looking through my WndProc, which seems to sometimes have the problem pass through, I cannot find a single place that hangs on to references to any objects. All but one of the variables are local to WndProc, and should therefore be garbage collected when the method terminates.

protected override void WndProc(ref Message m)
{
    base.WndProc(ref m); // I'm assuming the first trace can be caught here
    IntPtr hwnd = m.WParam;
    // Our hook tells us something got maximized
    if (Win32Import.UWM_MAXIMIZE == (UInt32)m.Msg)
    {
        // Figure out if we are temporarily disabled or using alternative profiles
        KeyStateInfo keyState = KeyboardInfo.GetKeyState(Settings.AlternativeProfileKey);
        Rectangle r = FindRectangle(MousePosition, (Settings.EnableAlternativeProfile && keyState.IsPressed ? AlternativeRegions : Regions));
        // Did we find a rectangle to place it in?
        if (r != Rectangle.Empty)
        {
            Rectangle position = Win32Import.GetWindowRectangle(hwnd);
            Rectangle previousPos = GetLocation(hwnd);
            if (position == r && previousPos != Rectangle.Empty)
            {
                // We are restoring the original position
                Win32Import.SetWindowPos(hwnd, IntPtr.Zero, previousPos.X, previousPos.Y, previousPos.Width, previousPos.Height, Win32Import.SWP_NOZORDER | Win32Import.SWP_NOSENDCHANGING);
            }
            else
            {
                // We are maximizing to a region
                Win32Import.ShowWindow(hwnd, Win32Import.WindowShowStyle.Restore);
                Win32Import.SetWindowPos(hwnd, IntPtr.Zero, r.X, r.Y, r.Width, r.Height, Win32Import.SWP_NOZORDER | Win32Import.SWP_NOSENDCHANGING);
                // Make sure we remember this location
                RememberLocation(hwnd, position);
            }
        }
    }
    else if (MaxTo64WindowHandleMessage == m.Msg)
    {
        // Store the window handle of our 64-bit subprocess
        SubProcess64WindowHandle = m.WParam;
    }
}

I have not been able to reproduce the error, even while running the program over multiple days.

My assumption is that the system is low on either unfragmented memory or GDI handles, but I cannot confirm this anywhere. There does not seem to be any good documentation on this error.

Any ideas what else it could be? Can I do anything to prevent this error?

Update: The question was reopened with more stack traces, because of the lack of a decent solution. Simply ignoring it does not solve the problem.

Monodrama

Not related to the question but how do you collect crash reports? – Bribe

Using Fogbugz BugzScout (Google it), and a custom-written global error handler in the program. It is not too hard. – Monodrama

Is anything reported in the Windows Application event log at the time of the error? – Musser

Not that I know of. All I know of this error are the stack traces above. – Monodrama

Are you trying to show a non opaque window? I mean, a transparent one? Has your user an old graphics card (not supporting transparencies and glass effects, things like that?). – Oakland

Yes, we do have semi-transparent windows (using the Opacity setting with WinForms). How come? – Monodrama

Answer 1

Leaking or using to many GDI objects/handles. Those could cause a resource heap shortage. You might not be able to reproduce because your users might have other GDI resource heavy programs running or use Terminal Server in which case they have to share some of the heap with the other users. See System Error. Code: 8. Not enough storage is available to process this command

Here you can read about the Desktop Heap Monitor tool to diagnose desktop heap problems.

Here and here and here are GDI leak detection tools.

Troy

This must be it. I am assuming this crash was caused by other programs leaking GDI handles. Monitoring my program showed that there is no leak there. – Monodrama

Have you tried raising the desktop heap? My experiance is that this error happens more often on Terminal Server systems where parts of the desktop heap are shared between users. This might explain why this error happends more often in those cases. – Troy

The problem is that it has never been reproducible on any of my computers, so I don't have anything to reproduce it on. I am adding more data collection to the crash analysis, but it will take quite a while before I have more detailed data. – Monodrama

@Lars: using GDIView, I've confirmed that MaxTo while running does not increase the "GDI Total" or "All GDI" over time. In fact when running in the background it doesn't change at all. With all the windows it can create at once open, the "All GDI" count barely reached 200. – Monodrama

@Vagard: Maybe you could ask the users to increase their desktop heap and see if it helps? Or ask them to use the diagnostic tool and send you the results? – Troy

@Lars: I will try, but these are quite complicated instructions, and I'm afraid most users won't be able to comprehend what I am asking of them... :) – Monodrama

@VegardLarsen I'm running into the same exact issue(same error but no increasing number of GDI Total/All GDI(and I'm on the computer that at some point has this issue.). Did you finally found a solution/lead? – Delius

@Delius I haven't ever seen a machine where this issue occured. I've only seen the crash reports, and at some point they stopped coming in. – Monodrama

Answer 2

Your program is probably leaking kernel resources. Start diagnosing this problem with Taskmgr.exe. View + Select Columns, check User objects, GDI objects and Handle count. Run your program and observe if any of these is increasing steadily. Once one of them reaches 10,000 your program will die.

With a way to quickly see the leak in action, you can start commenting code to see where the leak occurs. It probably has something to do with your "hook".

Seagrave

Answer 3

The problem probably dones't lie in your WndProc - the reason you see it in your call stacks is because pretty much everything GUI-related on Windows go through the WIN32 window procedure. Overriding it in your control simply gives you a hook point to process low-level stuff before higher-level .NET framework processing is done.

This is going to be a complete shot in the dark, but perhaps this post could be relevant? - probably not with those stack traces, though.

Corlisscorly

I cannot see how it would be relevant, as I can't see the post you linked to producing this error. However, the MaxTo main window does have nested panels, just not anywhere near as many as 12-15 levels deep, unless the user is getting really freaky with their settings. :) – Monodrama

@Vegard: the stack traces would probably look different if it was the problem. Is there any way for you to get full Win32 stack traces and not just the .net trace? – Corlisscorly

I don't think so easily. I've never been able to reproduce on my own machines. The stack trace is simply from the Exception object, I have not idea how to get the Win32 stacktrace (and would have to provide the user with a custom build after I figure out how to get it from .NET). – Monodrama

Answer 4

I had many custom Windows controls with own resources, so when I create many controls this error is appear. To fix this problem I made Resource file in my library and used outside resources instead of resources on my component code. After that my exception is gone, already tested with 3 times more opened forms and this error gone. So looks like it's a solution.

Midsummer