AccessViolationException when PInvoking C++ DLL (cdecl calling convention problem?) [closed]

Question

I've spent all day researching this, and I'm none the wiser:

I have a C# DLL which PInvokes a method in a C++ DLL. I have had no problems doing this when compiling in Debug mode, but when compiling in Release mode I get an AccessViolationException. Googling this problem tells me that it's probably an issue with incompliant calling conventions. The code now looks like this in C#:

[return: MarshalAs(UnmanagedType.U1)]
[DllImport("Native.dll", CallingConvention = CallingConvention.Cdecl)]
internal static extern Boolean AMethod(Int32 mode, byte frame);

and in C++:

extern "C" {
     DLL_EXPORT bool AMethod(int mode, BYTE frame)
     {
      ...
     }
}

I've set the C++ project to compile with the __cdecl calling convention in VS2010, but I still get the AccessViolationException and I have no idea what more I can do. I should note that my C++ DLL uses third-party DLLs, and I do not know what calling convention they use.

Any help would be appreciated!

Oh, and I don't get the exception on my development machine, only on my target system.

Hydr

Did you set up your C++ project to use the __cdecl convention in both Debug and Release builds? – Rensselaerite

Does your DLL uses ATL ? If true, then try setting the Use of ATL to "Static Link to ATL" – Hayse

Have you tried switching to __stdcall? I've never had good luck with the Cdecl calling convention with pinvoke, I always use __stdcall – Irs

Frederic: Yes, I did. Joao: The DLL uses MFC - but I'll give it a try anyway. Justin: I have, and it didn't seem to make a difference - I'm wondering if that's because the C++ DLL is dependent on other DLLs which are use cdecl? – Hydr

@PNielsen, calling conventions are not viral, so I don't think external dependencies are the culprit here. I'm more worried about the DLL_EXPORT macro, what exactly does it expand to? – Rensselaerite

Oh, sorry, I should have clarified that: #define DLL_EXPORT __declspec(dllexport) But thanks, the fact that calling conventions aren't viral helps a lot - that leaves me the option of experimenting more with __stdcall. – Hydr

Okay, so it turns out the exception was thrown for another reason (which I still haven't discovered), unrelated to the calling convention specified. Thanks for all your help, at least I learnt a few things about calling convention on the way, particularly from this website (in case anyone else has this problem): unixwiz.net/techtips/win32-callconv.html – Hydr

Answer 1

Try with this reordering of statements :

[DllImport("Native.dll", CallingConvention = CallingConvention.Cdecl)]
[return: MarshalAs(UnmanagedType.U1)]
internal static extern Boolean AMethod(Int32 mode, byte frame);

Gunar

Your answer intrigues me. How would reordering the method's attributes solve the questioner's problem? – Rensselaerite

I am also curious about how the reordering of the methods may solve the problem – Diestock