C# DllImport with C++ boolean function not returning correctly

Question

I have the following function in a C++ DLL

extern "C" __declspec(dllexport) bool Exist(const char* name)
{
 //if (g_Queues.find(name) != g_Queues.end())
 // return true;
 //else
 // return false;
 return false;
}

Inside my C# class I have the following:

[DllImport("Whisper.dll", EntryPoint="Exist", CallingConvention=CallingConvention.Cdecl)]
        public static extern bool Exist(string name);

Yet, whenever I call my function it ALWAYS returns true, even when I commented out my little function and made it return false. I have the feeling there is something wrong with my calling convention or any other issue with P/Invoking my DLL, probably corresponding with the string and const char*, but for now I am completely clueless. What am I doing wrong? Why does it return true instead of false?

EDIT: I have figured out this has nothing to do with the const char* or string, because the problem persists with an empty function. I've tried changing the calling convention between Cdecl and StdCall and neither work correctly. I've also managed to debug my DLL and it's being called correctly and does indeed return false, but once back into C# it somehow is true. Changing the CharSet also had no effect. I've made sure I've supplied my C# program with the latest and correct version of my DLL each time, so that shouldn't be an issue aswell. Again, I am completely clueless on why the result is true when I'm in fact returning false.

EDIT2: SOReader provided me with a suggestion which fixes another important issue, see my comment. Sadly, it does not fix the return issue.

EDIT3: I have concluded that changing the return type of Exist (bool) into (int) suddenly makes it return the correct number (true = 1, false = 0). That would mean that there may be an issue between C++'s bool and C#'s bool. I can continue using an int as a bool, but that would still not explain the original problem. Maybe somebody else can enlighten me on this one? Perhaps it has to do with the fact that I'm using x64 (although both pojects are compiled as x86)

Innocency

The first thing to check is that the function is in fact cdecl. If your makefile passes Gz or Gr to the compiler, then the function above is not cdecl. Add a __cdecl to your C code, or enable the pInvokeStackImbalance Managed Debugging Assistant. – Materialism

I don't think it will link if /Gr or /Gz are specified. Good point about the Managed Debugging Assistant. – Respectable

I tested it and __fastcall won't link with the /clr. But __stdcall (/Gz) links but then the Exist entrypoint isn't found at runtime as the function signature is different. – Respectable

Debug into Exist with a breakpoint on the function closing bracket. Look at the value of the EAX register when the breakpoint is hit. Whatever is in EAX is what false is defined to. – Ruhr

Answer 1

I found the solution for your problem. Your declaration should be preceded with this marshaling: [return:MarshalAs(UnmanagedType.I1)]

so everything should look like this:

[DllImport("Whisper.dll", EntryPoint="Exist", CallingConvention=CallingConvention.Cdecl)]  
[return:MarshalAs(UnmanagedType.I1)]  
public static extern bool Exist([MarshalAs(UnmanagedType.LPStr)] string name);

I tested it in my very simple example and it worked!

EDIT
Why this happens? C defines bool as 4 bytes int (as some of you have said) and C++ defines it as 1 byte. C# team decided to use 4 byte bool as default during PInvoke because most of the system API function use 4 bytes values as bool. If you want to change this behavior you have to do it with marshaling specifying that you want to use 1 byte value.

Oriel

That's odd. I've tested it in VS 2010 with 32bit builds of the DLL and C# app, AND 64bit builds of both and do not have the problem Shammah described. What you are saying would make sense in a Big Endian machine, but not in Little Endian architectures like the x86 based desktops we use. Remember, the least significant byte, the 1 or 0, in both cases for bool and 4 byte int would be the same. Which is probably why it works perfectly for me I guess. – Respectable

@SimonBrangwin: Unless you want to read the generated assembly or are a compiler vendor yourself, it's pretty much impossible to understand what happens when you get the return type wrong. It just doesn't work. There's a reason why C++ and C would both mark this as undefined behaviour. – Jannajannel

I would accept your reasoning re: C bools being 4 bytes and C++ bools being 1 byte, but this also happens when the CallingConvention is set to ThisCall, which should make it clear that it's a C++ method. Still, you're just the messenger, and after all, your solution fixed my problem, so thank you! – Torment

C doesn't define bool at all. – Evanescent

@GlennMaynard That was a shortcut. However most of the environments define #define TRUE 1 which is int which is 4B in size – Oriel

Lesson learned: extern "C" doesn't magically make bools 4 bytes. – Coalfish

Answer 2

C'sbool is actually int, as there is no boolean type in the original C language. That means that if C#'s DLLImport is designed to interop with C code, then they will expect that C#'s bool to correspond to C's int. While this still doesn't explain why false would become true, fixing it should fix the problem.

http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.unmanagedtype.aspx

This says that UnmanagedType.Bool is the Win32 BOOL, which is an int.

Jannajannel

Answer 3

This is actually caused by EAX not being fully cleared out by typical C++ code that returns a bool. It's typical for EAX to contain some bogus value when entering a function, and to return false the compiler would typically emit xor al, al. This clears out only the LSB of EAX, and causes C# code to interpret the resulting non-zero value as true instead of false.

Competence

Answer 4

Perhaps marshaling the argument of the function might help:

[MarshalAs(UnmanagedType.LPStr)]

Here is how the declaration should look like:

[DllImport("Whisper.dll", EntryPoint="Exist", CallingConvention=CallingConvention.Cdecl)]
        public static extern bool Exist([MarshalAs(UnmanagedType.LPStr)] string name);

Oriel

While debugging this I found out that this should indeed be used. If my string was "my_name", only "m" would get passed as an argument without the marshalling. With marshalling, the whole "my_name" would get passed. Sadly, this still doesn't fix the return issue :( – Innocency

I've reviewing my project wrote few years ago and I found that I used PreserveSig when bools where returned. Try this and let me know if that works: [PreserveSig] bool foo(); – Oriel

Actually I used only interface functions - not static ones, so I don't know if my tip will work ;( – Oriel

Answer 5

I tested your code and it returns false for me. So there must be something else going on.

Are you sure you are recompiling the DLL properly? Try deleting the .DLL and doing a rebuild.

Other than that everything seems to be fine assuming . By default the marshalling will handle the .NET string to const char* without having to decorate it with Marshal attributes, whether the DLL is compiled as ANSI or Unicode.

See http://msdn.microsoft.com/en-us/library/s9ts558h.aspx#cpcondefaultmarshalingforstringsanchor5

Respectable

Answer 6

I send the boolean variable, using the following system

__declspec(dllexport) const bool* Read(Reader* instance) {
    try {
        bool result = instance->Read();
        bool* value = (bool*)::CoTaskMemAlloc(sizeof(bool));
        *value = result;
        return value;
    } catch (std::exception exp) {
        RegistryException(exp);
        return nullptr;
    }
}

In C #, I do

DllImport(WrapperConst.dllName)]
public static extern IntPtr Read(IntPtr instance);

public bool Read() {
    IntPtr intPtr = ReaderWrapper.Read(instance));
    if(intPtr != IntPtr.Zero) {
        byte b = Marshal.ReadByte(intPtr);
        Marshal.FreeHGlobal(intPtr);
        return b != 0;
    } else {
        throw new Exception(GetLastException());
    }
}

Barrus

Answer 7

I use signed int which can return true/false correctly.

https://mcmap.net/q/428912/-c-from-c-c-function-in-a-dll-returning-false-but-c-thinks-it-39-s-true

Kilowatt

Answer 8

Defining the C++ method as a 'VARIANT_BOOL', rather than a 'bool' fixed a similar issue for me.

Newberry

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Hurds