Win32Exception Not enough storage is available to process this command

Question

Through my automated crash collection for MaxTo I got the following crash report:

V8.12.0.0 - System.ComponentModel.Win32Exception - :Void UpdateLayered():0
Version: MaxTo8.12.0.0
Exception: System.ComponentModel.Win32Exception
Error message: Not enough storage is available to process this command
Stack trace: 
  at System.Windows.Forms.Form.UpdateLayered()
  at System.Windows.Forms.Form.OnHandleCreated(EventArgs e)
  at System.Windows.Forms.Control.WmCreate(Message& m)
  at System.Windows.Forms.Control.WndProc(Message& m)
  at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
  at System.Windows.Forms.ContainerControl.WndProc(Message& m)
  at System.Windows.Forms.Form.WmCreate(Message& m)
  at System.Windows.Forms.Form.WndProc(Message& m)
  at MaxTo.MainForm.WndProc(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
  at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Another stacktrace:

Version: MaxTo2009.9.0.0
Exception: System.ComponentModel.Win32Exception
Error message: Not enough storage is available to process this command
Stack trace: 
  at System.Windows.Forms.Form.UpdateLayered()
  at System.Windows.Forms.Form.OnHandleCreated(EventArgs e)
  at System.Windows.Forms.Control.WmCreate(Message& m)
  at System.Windows.Forms.Control.WndProc(Message& m)
  at System.Windows.Forms.ScrollableControl.WndProc(Message& m)
  at System.Windows.Forms.ContainerControl.WndProc(Message& m)
  at System.Windows.Forms.Form.WmCreate(Message& m)
  at System.Windows.Forms.Form.WndProc(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
  at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
  at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

In this latest stack trace there is no reference to MaxTo at all, and 90% of the crashes I get are with stack traces similar to the above.

Reading around on the net I find that this error is usual if you forget to release or dispose variables. When looking through my WndProc, which seems to sometimes have the problem pass through, I cannot find a single place that hangs on to references to any objects. All but one of the variables are local to WndProc, and should therefore be garbage collected when the method terminates.

protected override void WndProc(ref Message m)
{
    base.WndProc(ref m); // I'm assuming the first trace can be caught here
    IntPtr hwnd = m.WParam;
    // Our hook tells us something got maximized
    if (Win32Import.UWM_MAXIMIZE == (UInt32)m.Msg)
    {
        // Figure out if we are temporarily disabled or using alternative profiles
        KeyStateInfo keyState = KeyboardInfo.GetKeyState(Settings.AlternativeProfileKey);
        Rectangle r = FindRectangle(MousePosition, (Settings.EnableAlternativeProfile && keyState.IsPressed ? AlternativeRegions : Regions));
        // Did we find a rectangle to place it in?
        if (r != Rectangle.Empty)
        {
            Rectangle position = Win32Import.GetWindowRectangle(hwnd);
            Rectangle previousPos = GetLocation(hwnd);
            if (position == r && previousPos != Rectangle.Empty)
            {
                // We are restoring the original position
                Win32Import.SetWindowPos(hwnd, IntPtr.Zero, previousPos.X, previousPos.Y, previousPos.Width, previousPos.Height, Win32Import.SWP_NOZORDER | Win32Import.SWP_NOSENDCHANGING);
            }
            else
            {
                // We are maximizing to a region
                Win32Import.ShowWindow(hwnd, Win32Import.WindowShowStyle.Restore);
                Win32Import.SetWindowPos(hwnd, IntPtr.Zero, r.X, r.Y, r.Width, r.Height, Win32Import.SWP_NOZORDER | Win32Import.SWP_NOSENDCHANGING);
                // Make sure we remember this location
                RememberLocation(hwnd, position);
            }
        }
    }
    else if (MaxTo64WindowHandleMessage == m.Msg)
    {
        // Store the window handle of our 64-bit subprocess
        SubProcess64WindowHandle = m.WParam;
    }
}

I have not been able to reproduce the error, even while running the program over multiple days.

My assumption is that the system is low on either unfragmented memory or GDI handles, but I cannot confirm this anywhere. There does not seem to be any good documentation on this error.

Any ideas what else it could be? Can I do anything to prevent this error?

Update: The question was reopened with more stack traces, because of the lack of a decent solution. Simply ignoring it does not solve the problem.

Answer 1

Leaking or using to many GDI objects/handles. Those could cause a resource heap shortage. You might not be able to reproduce because your users might have other GDI resource heavy programs running or use Terminal Server in which case they have to share some of the heap with the other users. See http://stackoverflow.com/questions/507853/system-error-code-8-not-enough-storage-is-available-to-process-this-command

Here you can read about the Desktop Heap Monitor tool to diagnose desktop heap problems.

Here and here and here are GDI leak detection tools.

Answer 2

Your program is probably leaking kernel resources. Start diagnosing this problem with Taskmgr.exe. View + Select Columns, check User objects, GDI objects and Handle count. Run your program and observe if any of these is increasing steadily. Once one of them reaches 10,000 your program will die.

With a way to quickly see the leak in action, you can start commenting code to see where the leak occurs. It probably has something to do with your "hook".

Answer 3

The problem probably dones't lie in your WndProc - the reason you see it in your call stacks is because pretty much everything GUI-related on Windows go through the WIN32 window procedure. Overriding it in your control simply gives you a hook point to process low-level stuff before higher-level .NET framework processing is done.

This is going to be a complete shot in the dark, but perhaps this post could be relevant? - probably not with those stack traces, though.