Why are Cdecl calls often mismatched in the "standard" P/Invoke Convention?

Question

I am working on a rather large codebase in which C++ functionality is P/Invoked from C#.

There are many calls in our codebase such as...

C++:

extern "C" int __stdcall InvokedFunction(int);

With a corresponding C#:

[DllImport("CPlusPlus.dll", ExactSpelling = true, SetLastError = true, CallingConvention = CallingConvention.Cdecl)]
    private static extern int InvokedFunction(IntPtr intArg);

I have scoured the net (insofar as I am capable) for the reasoning as to why this apparent mismatch exists. For example, why is there a Cdecl within the C#, and __stdcall within the C++? Apparently, this results in the stack being cleared twice, but, in both cases, variables are pushed onto the stack in the same reverse order, such that I do not see any errors, albeit the possibility that return information is cleared in the event of attempting a trace during debugging?

From MSDN: http://msdn.microsoft.com/en-us/library/2x8kf7zx%28v=vs.100%29.aspx

// explicit DLLImport needed here to use P/Invoke marshalling
[DllImport("msvcrt.dll", EntryPoint = "printf", CallingConvention = CallingConvention::Cdecl,  CharSet = CharSet::Ansi)]

// Implicit DLLImport specifying calling convention
extern "C" int __stdcall MessageBeep(int);

Once again, there is both extern "C" in the C++ code, and CallingConvention.Cdecl in the C#. Why is it not CallingConvention.Stdcall? Or, moreover, why is there __stdcall in the C++?

Thanks in advance!

Hitlerism

Answer 1

This comes up repeatedly in SO questions, I'll try to turn this into a (long) reference answer. 32-bit code is saddled with a long history of incompatible calling conventions. Choices on how to make a function call that made sense a long time ago but are mostly a giant pain in the rear end today. 64-bit code has only one calling convention, whomever is going to add another one is going to get sent to small island in the South Atlantic.

I'll try to annotate that history and relevance of them beyond what's in the Wikipedia article. Starting point is that the choices to be made in how to make a function call are the order in which to pass the arguments, where to store the arguments and how to cleanup after the call.

• __stdcall found its way into Windows programming through the olden 16-bit pascal calling convention, used in 16-bit Windows and OS/2. It is the convention used by all Windows api functions as well as COM. Since most pinvoke was intended to make OS calls, Stdcall is the default if you don't specify it explicitly in the [DllImport] attribute. Its one and only reason for existence is that it specifies that the callee cleans up. Which produces more compact code, very important back in the days when they had to squeeze a GUI operating system in 640 kilobytes of RAM. Its biggest disadvantage is that it is dangerous. A mismatch between what the caller assumes are the arguments for a function and what the callee implemented causes the stack to get imbalanced. Which in turn can cause extremely hard to diagnose crashes.

• __cdecl is the standard calling convention for code written in the C language. Its prime reason for existence is that it supports making function calls with a variable number of arguments. Common in C code with functions like printf() and scanf(). With the side effect that since it is the caller that knows how many arguments were actually passed, it is the caller that cleans up. Forgetting CallingConvention = CallingConvention.Cdecl in the [DllImport] declaration is a very common bug.

• __fastcall is a fairly poorly defined calling convention with mutually incompatible choices. It was common in Borland compilers, a company once very influential in compiler technology until they disintegrated. Also the former employer of many Microsoft employees, including Anders Hejlsberg of C# fame. It was invented to make argument passing cheaper by passing some of them through CPU registers instead of the stack. It is not supported in managed code due to the poor standardization.

• __thiscall is a calling convention invented for C++ code. Very similar to __cdecl but it also specifies how the hidden this pointer for a class object is passed to instance methods of a class. An extra detail in C++ beyond C. While it looks simple to implement, the .NET pinvoke marshaller does not support it. A major reason that you cannot pinvoke C++ code. The complication is not the calling convention, it is the proper value of the this pointer. Which can get very convoluted due to C++'s support for multiple inheritance. Only a C++ compiler can ever figure out what exactly needs to be passed. And only the exact same C++ compiler that generated the code for the C++ class, different compilers have made different choices on how to implement MI and how to optimize it.

• __clrcall is the calling convention for managed code. It is a blend of the other ones, this pointer passing like __thiscall, optimized argument passing like __fastcall, argument order like __cdecl and caller cleanup like __stdcall. The great advantage of managed code is the verifier built into the jitter. Which makes sure that there can never be an incompatibility between caller and callee. Thus allowing the designers to take the advantages of all of these conventions but without the baggage of trouble. An example of how managed code could stay competitive with native code in spite of the overhead of making code safe.

You mention extern "C", understanding the significance of that is important as well to survive interop. Language compilers often decorate the names of exported function with extra characters. Also called "name mangling". It is a pretty crappy trick that never stops causing trouble. And you need to understand it to determine the proper values of the CharSet, EntryPoint and ExactSpelling properties of a [DllImport] attribute. There are many conventions:

• Windows api decoration. Windows was originally a non-Unicode operating system, using 8-bit encoding for strings. Windows NT was the first one that became Unicode at its core. That caused a rather major compatibility problem, old code would not have been able to run on new operating systems since it would pass 8-bit encoded strings to winapi functions that expect a utf-16 encoded Unicode string. They solved this by writing two versions of every winapi function. One that takes 8-bit strings, another that takes Unicode strings. And distinguished between the two by gluing the letter A at the end of the name of the legacy version (A = Ansi) and a W at the end of the new version (W = wide). Nothing is added if the function doesn't take a string. The pinvoke marshaller handles this automatically without your help, it will simply try to find all 3 possible versions. You should however always specify CharSet.Auto (or Unicode), the overhead of the legacy function translating the string from Ansi to Unicode is unnecessary and lossy.

• The standard decoration for __stdcall functions is _foo@4. Leading underscore and a @n postfix that indicates the combined size of the arguments. This postfix was designed to help solve the nasty stack imbalance problem if the caller and callee don't agree about the number of arguments. Works well, although the error message isn't great, the pinvoke marshaller will tell you that it cannot find the entrypoint. Notable is that Windows, while using __stdcall, does not use this decoration. That was intentional, giving programmers a shot at getting the GetProcAddress() argument right. The pinvoke marshaller also takes care of this automatically, first trying to find the entrypoint with the @n postfix, next trying the one without.

• The standard decoration for __cdecl function is _foo. A single leading underscore. The pinvoke marshaller sorts this out automatically. Sadly, the optional @n postfix for __stdcall does not allow it to tell you that your CallingConvention property is wrong, great loss.

• C++ compilers use name mangling, producing truly bizarre looking names like "??2@YAPAXI@Z", the exported name for "operator new". This was a necessary evil due to its support for function overloading. And it originally having been designed as a preprocessor that used legacy C language tooling to get the program built. Which made it necessary to distinguish between, say, a void foo(char) and a void foo(int) overload by giving them different names. This is where the extern "C" syntax comes into play, it tells the C++ compiler to not apply the name mangling to the function name. Most programmer that write interop code intentionally use it to make the declaration in the other language easier to write. Which is actually a mistake, the decoration is very useful to catch mismatches. You'd use the linker's .map file or the Dumpbin.exe /exports utility to see the decorated names. The undname.exe SDK utility is very handy to convert a mangled name back to its original C++ declaration.

So this should clear up the properties. You use EntryPoint to give the exact name of the exported function, one that might not be a good match for what you want to call it in your own code, especially for C++ mangled names. And you use ExactSpelling to tell the pinvoke marshaller to not try to find the alternative names because you already gave the correct name.

I'll nurse my writing cramp for a while now. The answer to your question title should be clear, Stdcall is the default but is a mismatch for code written in C or C++. And your [DllImport] declaration is not compatible. This should produce a warning in the debugger from the PInvokeStackImbalance Managed Debugger Assistant, a debugger extension that was designed to detect bad declarations. And can rather randomly crash your code, particularly in the Release build. Make sure you didn't turn the MDA off.

Samhita

Thanks for the lesson. I have a new respect for P/Invoke. And I better appreciate __clrcall and the lack of Multiple Inheritance in C#. – Pneumonia

+1 I have a couple of mildly pendantic comments. You talk about __fastcall but that is actually an MS calling convention. Generically it can be considered as part of the family of fastcall conventions. MS fastcall, Borland fastcall etc. The MS version, __fastcall uses just two x86 registers: ECX, EDX. The Borland version, which lives on today over here in Delphi world as the register convention, uses three registers. EAX is added to the mix. – Marius

My other comment relates to name decoration. I discovered recently, whilst answering a question here, that different compilers have different decorations for __stdcall. It seems that MS, Borland and GNU toolsets all differ. I suspect that other compilers introduce yet more variation. – Marius

Hans, thank you for this explanation! I have been only able to garner bits and pieces via sifting through a myriad of points.. it's excellent to have all this information in one place! Also, thanks, you guys, for the comments to this, as well! It's still all sinking in for me, but this is very helpful, especially with regard to the history as to why things are in this current state of confusion/mixed examples! – Hitlerism

This is great. I haven't ever seen all this info in the same place, before. I wish I could give this answer multiple upvotes. – Bitt

Yes, I know, tar and feathers are ready to go. I do have to give them credit for going through extraordinary lengths to keep it compatible. How well that works in practice is something we'll find out. – Samhita

@HansPassant: Maybe I'm ignorant but what is really the point in __vectorcall after all? I'm currently working in a project with vector intrinsics and we simply pass vectors by (const) reference… Is there any documentation on this outside of MSDN? (It's not yet for example in Agner Fog's excellent guides…) – Aside

@Aside __vectorcall (both x32 and x64 versions, I believe) was added to help game/codec/video/audio developers optimise code, by introducing a calling convention designed specifically to optimise the passing of vector arguments on processors with vector registers; basically, it's to make SIMD stuff work better. At least, that's what the MSDN blog says. – Schadenfreude

Also, from what information I've been able to tell, the x64 calling convention is a variant of __fastcall, and/or is considered to be close enough to be called one. Not sure if this is correct or not, though, it's pretty much based entirely on third-party information. – Schadenfreude

The MSVC mangler does indeed make borderline-unreadable gibberish, but to their credit, that gibberish is generally the entire declaration, and actually easy to read if you know what the symbols mean. Objects are ?<name>@[<all containing scopes, right to left, each ending with '@'>]@<access modifier #><type><cv-qualifier>. Functions are ?<name>@[<all containing scopes, r-to-l, each ending with '@'>]@<access modifier & near/far><calling convention><ret type><param list><throw specifier>. [All functions are near in 32- or 64-bit environments.] Templates are ...gibberish. – Schadenfreude

For example, global int a; is ?a@@3HA, and void NS::func(); is ?func@NS@@YAXXZ. If a name (including its terminating '@') is encountered multiple times, subsequent uses of it are replaced with a digit (e.g. void NS::NS::NS() is ?NS@00@YAXXZ). Containing class & namespace names are treated identically. In function parameter lists, multi-character <type> codes are abbreviated like names, being replaced with a digit; the return type isn't considered type 0. Only the first 10 names and first 10 parameter types are abbreviated, as 0..9. – Schadenfreude

Numbers are encoded kinda awkwardly (for any number N, it's encoded as N - 1 if 0 < N < 11; otherwise, it's encoded as a hex number, where 0..F are replaced by A..P, and terminated with @. Negative numbers start with ?, I believe. – Schadenfreude

Pointers & references are awkward; their type symbol is a multi-character code consisting of <cv-qualified pointer type>[<modifiers>]<target cv-qualifier><target type>. Function pointers have function declaration gibberish (like YAXXZ) as their <target type>, except with <access modifier & near/far> as a digit (usually 6 for non-member & 8 for member), & don't have <target cv-qualifier>. Global pointer objects are awkward; their trailing <cv-qualifier> is an exact copy of <target cv-qualifier>, apparently including some (or all?) of the possible <modifiers>, if present. – Schadenfreude

(At least, I don't think function pointers have <target cv-qualifier>.) Pointers-to-members have a modified <target cv-qualifier> element, which consists of <target cv-qualifier letter (shifted)><containing class name>, where <containing class name> is the containing class' fully-qualified name (with each individual name terminated with @, all scopes listed right-to-left, and the entire name terminated with an additional @); for global pointer-to-member objects, the class name in the trailing <cv-qualifer> will be abbreviated. – Schadenfreude

For non-cv-qualified primitive types, the return type is encoded as the type's code. For cv-qualified and/or complex types, it uses pointer type syntax, except with ? (for "not a pointer/reference") as <cv-qualified pointer type>. Function parameters, unlike global objects, don't have a trailing <cv-qualifier> element, which I believe to be the reason pointer symbols contain <target cv-qualifier>; as mentioned, multi-character types are abbreviated after being encountered. (For example, void f(int, int); is ?f@@YAXHH@Z, but void g(bool, bool) is ?g@@YAX_N0@Z.) – Schadenfreude

If the parameter list is empty, it's encoded as X (for (void)). if it ends in an ellipsis (...), the ellipsis is encoded as Z. If neither of these is true, the parameter list is terminated with @. (For example, void f() is ?f@@YAXXZ, void g(int, ...) is ?g@@YAXHZZ, and void h(int) is ?h@@YAXH@Z.) The <throw specifier> apparently follows the same rules as the parameter list; however, Visual Studio ignores all throw specifiers, and encodes them as throw(...), which is why all function symbols end in Z. – Schadenfreude

Oh, yeah, I forgot some return type examples. int f() is ?f@@YAHXZ, const int g() is ?g@@YA?BHXZ, and CL h(), where CL is a class, is ?h@@YA?AVCL@@XZ. – Schadenfreude

Managed code (C++/CLI or Managed Extensions for C++) appears to add a modifier at the start of a function symbol's gibberish component, to indicate information about the native (non-managed) equivalent. Symbols I'm aware of are: $$F for normal functions (e.g. ?f@@$$FYAXXZ for a managed version of void f()), $$H for main() (not sure if this holds any other significance), and $$J<number> for extern "C" functions, where <number> appears to be based on the calling convention. Not sure of any other big changes there. – Schadenfreude

Member functions appear to have an additional symbol, <this cv-qualifier>, between <access modifier & near/far> and <calling convention> to indicate information about the this pointer. For example, for class CL, void CL::f() is ?f@CL@@QAEXXZ, void CL::g() const is ?g@CL@@QBEXXZ, and void CL::h() volatile is ?h@CL@@QCEXXZ. this pointer ref-qualifiers are also encoded, but I'm not sure exactly how yet. – Schadenfreude

As I don't have access to VS2015 right now, I've been using the [online compiler](webcompiler.cloudapp.net) to test. So far, I've been able to test function pointers with ref-qualifiers, but I'm having trouble with the functions themselves. For class CL, with public member functions void ffff(), void func() &, and void func() &&, the following pointer-to-member types are generated: P8CL@@AEXXZ (void (__thiscall CL::*)(void)), P8CL@@GAEXXZ (void (__thiscall CL::*)(void)&), and P8CL@@HAEXXZ (void (__thiscall CL::*)(void)&&). – Schadenfreude

Note that there is apparently a __based property, and some other properties, that can modify functions and/or pointers. I'm not sure how to read them yet. Nor am I sure how to read templates yet. – Schadenfreude

...And after all this, I would like to point out that there is a somewhat glaring flaw with this mangling scheme, that causes UnDecorateSymbolName(), the more powerful hidden __unDName(), and anything that depends on either of the two (such as UNDNAME and DUMPBIN /SYMBOLS), to have problems demangling any global pointer object where either the pointer or target is cv-qualified. Specifically, it doesn't know how to parse the trailing <cv-qualifier> properly for pointer objects, and outputs the target's cv-qualifiers as both the pointer and target cv-qualifiers. – Schadenfreude

For example, undname 0x2000 PAH, undname 0x2000 QAH, and undname 0x2000 PBH are correctly parsed as int *, int * const, and int const *, respectively. [Where 0x2000 is a flag to tell undname that it's being given a raw type symbol, not a full mangled name symbol.] However, if we try undname ?a@@3PAHA, undname ?a@@3QAHA, and undname ?a@@3PBHB, it outputs int * a, int * a, and int const * const a. Testing with ?a@@3PBH? and ?a@@3?AHA reveals that for a full mangled name, it treats the trailing <cv-qualifier> element as the pointer's cv-qualifiers. – Schadenfreude

Due to this, and the needless repetition of information that caused it, I personally think they would be better served by changing global objects to use the same syntax as function return types (where cv-qualifiers are omitted for non-cv, non-complex types, or the type is encoded in pointer syntax, with ? as the pointer type, when cv-qualifiers aren't omitted). – Schadenfreude

Could you add references to the info you provide (primarily the CLR bits as the main topic)? E.g. I cannot find any information about this "verifier built into the jitter". – Syncope

@HansPassant: i think still there is a doubt. __cdecl Is Default Calling Convention For C And C++. However, __thiscall is CallingConvention For Instance Methods Of C++ Exported Classes. This Means We Can Have Function Names same as "?ConvertRVBtoK@@YGHPAEJJJ0E@Z " with __cdecl Calling Convention. – Driskill

Use the undname.exe utility as mentioned in the answer. You'll see it is not an instance method and that it uses the stdcall convention. – Samhita

Excellent answer. Thanks. Just want to make a small point regarding ‘Default is stdcall’. That is not what I found for platform-invoke on wince7. There the default was cdecl. – Parallelogram

What I wrote only applies to x86 processors. On wince7 you'd likely be interoperating with an ARM processor. Similar to x64, lots of registers, it has only one fastcall-style calling convention. – Samhita

@HansPassant Re wince7 likely arm processor. Good point. It is an ARM processor. Thanks. – Parallelogram

Answer 2

cdecl and stdcall are both valid and usable between C++ and .NET, but they should consistent between the two unmanaged and managed worlds. So your C# declaration for InvokedFunction is invalid. Should be stdcall. The MSDN sample just gives two different examples, one with stdcall (MessageBeep), and one with cdecl (printf). They are unrelated.

Arawakan

Agreed; recommend doing some research on "calling conventions" to see why the difference is important. – Dominik