Access Violation issue with unhandled managed Exceptions in managed C++ .NET application

Question

This is actually a solved problem, but it's so esoteric I thought I'd share it for other users.

Also perhaps others might suggest reasons?

Anyway, I'm working on a "mixed mode" .NET application written in managed C++, but with heavy links into existing native libraries.

The problem was, unhandled managed exceptions ended up as Win32 Access Violations. What I mean by this is that instead of showing the nice .NET dialog one gets with an unhandled managed exception, instead I would get the older style "Unhandled win32 exception occurred in..." message.

Here's the interesting thing: if I start the app in the debugger, then the thrown managed exception gets correctly picked up. i.e., the debugger shows me the line.

However, when executing normally, it would turn into this Access Violation. Attaching the debugger at that point would produce little useful information (it wouldn't even show a sensible stack trace).

So, to me it suggests that something is happening in native code just before the unhandled managed exception reaches the exception handler.

So anyway, I managed to solve the issue by comparing my project to a clean new C++ managed project generated by Visual Studio 2008.

The fix was to do the following:

1. Change the /SUBSYSTEM flag (Project properties->Linker->System->SubSystem) from /SUBSYSTEM:WINDOWS to "Not Set"

2. Switched from using old style WinMain() to using the new style main().

i.e. it used to be

  int APIENTRY _tWinMain(HINSTANCE hInstance,
                       HINSTANCE hPrevInstance,
                       LPTSTR    lpCmdLine,
                       int       nCmdShow)

And it is now

int main(array<System::String ^> ^args)

[Why am I using this weird _tWinMain? This was what got generated by the Visual Studio .NET IDE many years ago when you create a sample mixed mode Windows app. It's always worked fine (until now) so I never bothered changing it. _tWinMain is just a macro to WinMain]

I made this change and the problem disappears. Unhandled .NET exceptions now get properly trapped so I can now actually debug them.

I also made the inverse change to the clean sample C++ app and proved that it was the cause.

So, really my questions is, what the heck is going on?

Was it just that I was using the old style WinMain instead of the new main(array <String^>^)?

Should I maybe report this to Microsoft (will anyone care ;-) )?

Heavenly

You can take the solution part out, make it an answer and accept it (it's the self learner thing) – Roentgenology

Answer 1

I am not really surprised.

If you write the top-level main in managed code, any managed exception that will bubble up, will be handled by managed code. The same as it would in C#. This is because the OS does not call directly into your int main(array<System::String ^> ^args) function. This is done by managed code which is either part of .net, or inserted at compile time. This is the code that notices and handles any escaping .net exceptions.

If you write your own native WinMain, there is no .net code calling into this function, so there is also no managed code able to handle the managed exceptions. When a managed exception is thrown, it appears to the OS as any other native windows exception.

Note that this is not an exact description of how main/WinMain is invoked. It's an educated guess based upon your problem description, and a little experience. I left out some details that I know about, and probably also detail which I am totally not aware of. But I am quite sure this is the essence of the story.

Supersaturated

But using WinMain DID always work in the past. In fact, as I say it was the canonical method in before Visual Studio 2008 - that's how the VS generated sample main objects. – Heavenly

@John: Right.... Now I see you do mention that. It would be interesting to check the call stack in the debugger, on the old visual studio (which version?) and 2008. – Supersaturated

Answer 2

I don't have experience with this in .NET or managed code but I have a fair amount of experience with the native side.

I don't see how or why changing the entry point (main vs WinMain) or the subsystem (win32 vs console vs none) should affect this. I'm not saying it didn't, just that it's not the root cause. (Aside: I don't know what subsystem:"not set" means. I'd think the subsystem, which is something the linker knows and stamps into the executable file, has to be set to something, and if the entry point is main(), it's probably set to console. Console subsystem apps can still interact with the GUI in all the normal ways, but they also always have a console window attached, and if you launch them from outside a console window they create one, so this is usually not a good choice for apps you're going to ship to anyone but yourself.)

At the Win32 API level, this kind of behavior is controlled by:

• whether you catch all exceptions, or let some dribble out of the top level
• SetUnhandledExceptionFilter(), http://msdn.microsoft.com/en-us/library/ms680634(v=vs.85).aspx

Perhaps something in your old WinMain(), or something called from there, or some .NET managed goo that runs before WinMain (even in native C/C++, with the default MSVC setup, WinMain isn't the real entry point as far as the linker is concerned -- there's a wrapper function provided by MSVC's C runtime library, which is the real entry point and which calls WinMain -- I'd expect .NET does the same thing but more so), was calling SetUnhandledExceptionFilter(NULL), and in your fixed version that's no longer happening.

If you know what function is responsible for putting up the "nice .NET dialog one gets with an unhandled managed exception", you could pass that to SetUnhandledExceptionFilter(), but it's probably not a good idea to call that for exceptions in unmanaged code.

An alternate theory for all this: the 'older style "Unhandled win32 exception occurred in..."' dialog actually said Access Violation, and meant it? It's also possible the .NET exception handler was kicking in and then actually crashing for some reason; if that's the case it should be possible to debug that, though that would require a lot more information.

Enervate