How to call a C++ function with a struct pointer parameter from C#?

Question

Okay one more function that it's not yet working. I am basically calling some C++ functions from C# by using P/Invoke. The problematic function does query a show laser device for some device related information, such as minimal and maximal scan rates and maximal points per second.

The problematic function is:

int GetDeviceInfo(DWORD deviceIndex, DeviceInfo* pDeviceInfo);

Here's the C++ header file that I was given. That's a link to the very brief C++ SDK description. I don't have the sources to rebuild the DLL file and I also don't have the *.pdb file (the manufacturer can not supply it):

#pragma once

#ifdef STCL_DEVICES_DLL
#define STCL_DEVICES_EXPORT extern "C" _declspec(dllexport) 
#else
#define STCL_DEVICES_EXPORT extern "C" _declspec(dllimport)
#endif

enum SD_ERR
{
    SD_ERR_OK = 0,
    SD_ERR_FAIL,
    SD_ERR_DLL_NOT_OPEN,
    SD_ERR_INVALID_DEVICE,  //device with such index doesn't exist
    SD_ERR_FRAME_NOT_SENT,
};

#pragma pack (1)
struct LaserPoint
{
    WORD x;
    WORD y;
    byte colors[6];
};

struct DeviceInfo
{
    DWORD maxScanrate;
    DWORD minScanrate;
    DWORD maxNumOfPoints;
    char type[32];
};

//////////////////////////////////////////////////////////////////////////
///Must be called when starting to use
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int OpenDll();

//////////////////////////////////////////////////////////////////////////
///All devices will be closed and all resources deleted
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT void CloseDll();

//////////////////////////////////////////////////////////////////////////
///Search for .NET devices (Moncha.NET now)
///Must be called after OpenDll, but before CreateDeviceList!
///In pNumOfFoundDevs can return number of found devices (optional)
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int SearchForNETDevices(DWORD* pNumOfFoundDevs);

//////////////////////////////////////////////////////////////////////////
///Creates new list of devices - previous devices will be closed
///pDeviceCount returns device count
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int CreateDeviceList(DWORD* pDeviceCount);

//////////////////////////////////////////////////////////////////////////
///Returns unique device name
///deviceIndex is zero based device index
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int GetDeviceIdentifier(DWORD deviceIndex, WCHAR** ppDeviceName);

//////////////////////////////////////////////////////////////////////////
///Send frame to device, frame is in following format:
///WORD x
///WORD y
///byte colors[6]
///so it's 10B point (=> dataSize must be numOfPoints * 10)
///scanrate is in Points Per Second (pps)
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int SendFrame(DWORD deviceIndex, byte* pData, DWORD numOfPoints, DWORD scanrate);

//////////////////////////////////////////////////////////////////////////
///Returns true in pCanSend if device is ready to send next frame
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int CanSendNextFrame(DWORD deviceIndex, bool* pCanSend);

//////////////////////////////////////////////////////////////////////////
///Send DMX if device supports it - pDMX must be (!!!) 512B long
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int SendDMX(DWORD deviceIndex, byte* pDMX);

//////////////////////////////////////////////////////////////////////////
///Send blank point to position x, y
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int SendBlank(DWORD deviceIndex, WORD x, WORD y);

//////////////////////////////////////////////////////////////////////////
///Get device info
//////////////////////////////////////////////////////////////////////////
STCL_DEVICES_EXPORT int GetDeviceInfo(DWORD deviceIndex, DeviceInfo* pDeviceInfo);

This is the complete C# test code I am currently using. All the functions work fine, except for GetDeviceInfo(...):

using System;
using System.Threading;
using System.Runtime.InteropServices;

namespace MonchaTestSDK {

    public class Program {

        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // OK
        public static extern int OpenDll();
        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // OK
        public static extern void CloseDll();
        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // OK
        public static extern int SearchForNETDevices(ref UInt32 pNumOfFoundDevs);
        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // OK
        public static extern int CreateDeviceList(ref UInt32 pDeviceCount);
        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // OK
        public static extern int GetDeviceIdentifier(UInt32 deviceIndex, out IntPtr ppDeviceName);
        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // OK
        public static extern int SendFrame(UInt32 deviceIndex, LaserPoint[] pData, UInt32 numOfPoints, UInt32 scanrate);
        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // OK
        public static extern int CanSendNextFrame(UInt32 deviceIndex, ref bool pCanSend);
        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // OK
        public static extern int SendBlank(UInt32 deviceIndex, UInt16 x, UInt16 y);
        [DllImport("..\\..\\dll\\StclDevices.dll", CallingConvention = CallingConvention.Cdecl)]                                    // FAILS
        public static extern int GetDeviceInfo(UInt32 deviceIndex, ref DeviceInfo pDeviceInfo);

        [StructLayout(LayoutKind.Sequential, Pack=1)]
        public struct LaserPoint {
            public UInt16 x;
            public UInt16 y;
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 6)]
            public byte[] colors;
        }

        [StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
        public struct DeviceInfo {
            public UInt32 maxScanrate;
            public UInt32 minScanrate;
            public UInt32 maxNumOfPoints;
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)]
            public string deviceType;
        }

        public static void Main(string[] args) {
            Console.WriteLine("Moncha SDK\n");

            OpenDll();
            Console.WriteLine("StclDevices.dll is open.");

            UInt32 deviceCount1 = 0;
            int r1 = SearchForNETDevices(ref deviceCount1);
            Console.WriteLine("SearchForNETDevices() [" + r1+"]: "+deviceCount1);

            UInt32 deviceCount2 = 0;
            int r2 = CreateDeviceList(ref deviceCount2);
            Console.WriteLine("CreateDeviceList() ["+r2+"]: "+deviceCount2);

            IntPtr pString;
            int r3 = GetDeviceIdentifier(0, out pString);
            string devname = Marshal.PtrToStringUni(pString);
            Console.WriteLine("GetDeviceIdentifier() ["+r3+"]: "+devname);

            DeviceInfo pDevInfo = new DeviceInfo();
            pDevInfo.type = "";
            int r4 = GetDeviceInfo(0, ref pDevInfo);
            Console.WriteLine("GetDeviceInfo() ["+r4+"]: ");
            Console.WriteLine("  - min: "+pDevInfo.minScanrate);
            Console.WriteLine("  - max: " + pDevInfo.maxScanrate);
            Console.WriteLine("  - points: " + pDevInfo.maxNumOfPoints);
            Console.WriteLine("  - type: " + pDevInfo.deviceType);

            Thread.Sleep(5000);
            CloseDll();
        }

    }
}

On line 73 line 64 (cp. screenshot):

int r4 = GetDeviceInfo(0, ref pDevInfo); 

I receive the following error:

An unhandled exception of type 'System.NullReferenceException' occured in MonchaTestSDK.exe
Additional information: Object reference not set to an instance of an object

This is the stack trace (can't provide better stack trace without the DLL's *.pdb file I guess):

MonchaTestSDK.exe!MonchaTestSDK.Program.Main(string[] args) Line 73 + 0xa bytes C# mscoreei.dll!73a8d91b()
[Frames below may be incorrect and/or missing, no symbols loaded for mscoreei.dll]
mscoree.dll!73cae879()
mscoree.dll!73cb4df8()
kernel32.dll!74a08654()
ntdll.dll!77354b17()
ntdll.dll!77354ae7()

Some disassembly:

            int r4 = GetDeviceInfo(0, ref pDevInfo);
05210749  int         3  
0521074A  push        ebp  
0521074B  cwde  
0521074C  xor         ecx,ecx  
0521074E  call        0521011C  
05210753  int         3  
05210754  test        dword ptr [eax-1],edx  
05210757  ?? ?? 
05210758  dec         dword ptr [ebx-0AF7Bh]  
0521075E  dec         dword ptr [ecx-6F466BBBh]

Any idea what I am doing wrong here?

---

Update 1: Suggested debug options:

As suggested in the comments, I tried to enable native/unmanaged code debugging:

1. Debug > Windows > Exceptions Settings > "Win32 Exceptions" checkbox ticked

2. Project > Properties > Debug tab > "Enable unmanaged code debugging" checkbox ticked

I still don't get any meaningful exception stack. The manufacturer can't supply me the DLL's *.pdb file.

Here's an image showing the debugger when stopped at the problematic line (debug settings are also shown):

---

Update 2: Minimal Required Code (cp. comment of mpromonet)

This is the minimal required code to be able to call GetDeviceInfo(...):

public static void Main(string[] args) {
    OpenDll();
    UInt32 deviceCount = 0;
    CreateDeviceList(ref deviceCount);
    DeviceInfo pDevInfo = new DeviceInfo();
    GetDeviceInfo(0, ref pDevInfo);            // error occurs on this line
    CloseDll();
}

This leads to the exact same error as before:

An unhandled exception of type 'System.NullReferenceException' occured in MonchaTestSDK.exe
Additional information: Object reference not set to an instance of an object

Removing the call GetDeviceInfo(0, ref pDevInfo); from the code above allows the program to exit without any error.

---

Update 3: Removing char[] deviceType from DeviceInfo struct completely

I removed char[] deviceType from the struct defintion:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1)]
public struct DeviceInfo {
    public UInt32 maxScanrate;
    public UInt32 minScanrate;
    public UInt32 maxNumOfPoints;
    //[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)]
    //public string deviceType;
}

When I run my C# test code now, I successfully receive maxScanrate, minScanrate and maxNumOfPoints back from the C++ DLL. Here's the corresponding console output:

GetDeviceInfo() [0]:
   - min: 1000
   - max: 40000
   - points: 3000

Finally ending in the following error message:

Exception thrown at 0x67623A68 (clr.dll) in MonchaTestSDK.exe: 0xC0000005: Access violation reading location 0x00000000.

---

Final Update

I finally got an updated DLL from the manufacturer. There was indeed a bug within the SDK that caused the stack to get corrupted. So basically the following solution now works fine without any issues:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1)]
public struct DeviceInfo {

    public UInt32 maxScanrate;
    public UInt32 minScanrate;
    public UInt32 maxNumOfPoints;

    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)]
    public string deviceType;

}

private void queryDeviceProperties(UInt32 index) {
    HwDeviceInfo pDevInfo = new HwDeviceInfo();
    int code = GetDeviceInfo(index, ref pDevInfo);
    if(code==0) {
        Console.WriteLine(pDevInfo.minScanrate);
        Console.WriteLine(pDevInfo.maxScanrate);
        Console.WriteLine(pDevInfo.maxNumOfPoints);
        Console.WriteLine(pDevInfo.type);
    } else {
        Console.WriteLine("Error Code: "+code);
    }
}

Thank you all for the great support!

Separatrix

We can't see the mistake in the info you provided. Only way to get ahead is provide more details. Enable unmanaged debugging and show us the stack trace you see on the first-chance exception notification. – Physiography

@Hans Passant : Ok I will enable unmanaged debugging and report back. I posted my complete testing code on my initial question here: #50103924 – Separatrix

Ok I enabled native unmanaged debugging and update the original post with the complete test code and stack-trace. – Separatrix

DeviceInfo does not have Pack=1 attribute set (it should not need that, but it's worth a try). Or check how C++ compiler packs this structure and add proper pack there (maybe pack=4?) – Wheeler

There is another question hidden in there, resembles "how to use unmanaged debugging to diagnose a crash". Essential steps are to enable the symbol server so you get readable stack traces and to force the debugger to stop when the original unmanaged exception is thrown. In VS2017 you use Debug > WIndows > Exceptions Settings, tick the checkbox for "Win32 Exceptions". Update the question with the new stack trace you now see. – Physiography

Fwiw, a null pointer crash in unmanaged code is frequently caused by passing an invalid argument to a function. Be sure to check that 0 is a valid index, you need the return value of CreateDeviceList() to verify. And hopefully it doesn't start indexing at 1. – Physiography

I unexpectedly had to borrow the controller and the laser device to someone for a party this weekend. But I will get it back on Sunday evening. I will then post the stack trace immediately. Yes, I am 100% sure, that 0 is a valid index for all functions. It's described like that in the very brief manual of the SDK (I will put a link to these docs in the question). Also, when calling the search for devices function, the device index starts with 0 too. I will report back on Sunday evening with the stack traces. – Separatrix

I would also try assigning a full length string to pDevInfo.type = "";, so a string with 32 spaces in it and see if it helps in any ways – Surround

Exceptions from unmanaged code do not always hit the nail on the head. Sometimes another, older exception is shown which has been properly handled earlier somewhere else and has absolutely nothing to do with the origin of the error. So I guess that the problem is somewhere in the C++ code. – Fundy

I got the laser device and controller back. I did change the debugging options as suggested without any difference (please compare to the update section in the original post). More ideas how to debug? Moreover, SearchForNETDevices(DWORD* pNumOfFoundDevs) and CreateDeviceList(DWORD* pDeviceCount) both functions ask for a DWORD pointer as parameter. Currently I am using ref UInt32. What about using UIntPtr instead? – Separatrix

@Fundy : The C++ code is most probably fine, since my C++ test code runs fine. I can call every function defined in the header file without any problems. – Separatrix

@HansPassant : The return values of the three previously called methods are all 0; this probably means success? The results I get from the passed pointers are also correct. – Separatrix

@Separatrix Aren't the chars in string in C# and C++ different in size? If you are sending 32 bytes from C#, you should increase the size in C++ to 64 and use wide char. – Shelf

@Marcel: Line 73 is const UInt32 numOfPts = 600; which makes the faulting line Console.WriteLine(" - type: " + pDevInfo.type); which is much more reasonable for throwing a managed NullReferenceException. OP gave the wrong line but it didn't stop me. – Clipfed

@Clipfed : I am sorry about this mistake. You're right, the line number was wrong, the program stops at line 64 (I likely changed the code after I had copied the exception description). But the error indeed occurs on the line int r4 = GetDeviceInfo(0, ref pDevInfo);. – Separatrix

@Shelf : Yes I read about different char type lengths between C++ and C#. I will now dive deeper into this topic. – Separatrix

The error is probably a consequence of some corruption that occurs before the call to GetDeviceInfo method. Could you try to reduce the code sample to only the GetDeviceInfo call and add its implementation ? – Sain

@Sain : Please check 'Update 2 + 3' in my original post. Thank you! – Separatrix

@slocinx : I made a test similar and it works without problems in 32bits and in 64bits. Maybe some corruption in opendll, CreateDeviceList, GetDeviceInfo... continuing to reduce the code you will find the problem. – Sain

I'm voting to close this question as off-topic because this requires understanding of an obscure and now known to be bugged DLL for which OP doesn't have source either. – Clipfed

SDK update from the manufacturer finally solved the stack corruption. I updated the initial question with "Final Update" section. Thank you all! – Separatrix

Answer 1

[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 32)] declares that the field is stored a char[32] array as in the header, i.e. space for 31 characters and a null terminator.

Marshalling this to a string shouldn't be a problem, nothing that the dll writes to the array should be able to cause a NullReferenceException.

I can compile a stub dll that loads fine using your C# code and can send back ANSI strings, with addition of typedef byte... and a stub method body e.g.:

int GetDeviceInfo(DWORD deviceIndex, DeviceInfo* pDeviceInfo)
{
    std::string testString = "test string thats quite loooooooong"; 
    pDeviceInfo->maxScanrate = 1234;
    pDeviceInfo->minScanrate = 12345;
    pDeviceInfo->maxNumOfPoints = 100 + deviceIndex;
    sprintf_s(pDeviceInfo->type, "%.31s", testString.c_str());
    return 0;
}

This works for me with VS2017 C++ and .Net 4.6.1.

What happens if you change the C# declaration to this:

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public struct DeviceInfo
    {
        public UInt32 maxScanrate;
        public UInt32 minScanrate;
        public UInt32 maxNumOfPoints;
        public UInt64 deviceTypePart1;
        public UInt64 deviceTypePart2;
        public UInt64 deviceTypePart3;
        public UInt64 deviceTypePart4;

        public string GetDeviceType()
        {
            if (Marshal.SizeOf(this) != 44) throw new InvalidOperationException();
            List<byte> bytes = new List<byte>();
            bytes.AddRange(BitConverter.GetBytes(deviceTypePart1));
            bytes.AddRange(BitConverter.GetBytes(deviceTypePart2));
            bytes.AddRange(BitConverter.GetBytes(deviceTypePart3));
            bytes.AddRange(BitConverter.GetBytes(deviceTypePart4));
            return Encoding.GetEncoding(1252).GetString(bytes.ToArray());
        }
    }

[Edit]

I've no idea why hand cranking the marshaling fixes this - be sure to 'load test' in case there are heap/stack corruption bugs still lurking.

In your old code, does Marshal.SizeOf return something other than 44?

Fencible

Great! Wow, your solution finally works fine! The test program delivers all needed information and the process exits by returning 0 :-) I will check this as correct answer and award the bounty to you! – Separatrix

I was happy too early... When running the compiled executable, it does crash. I suspect that there‘s something wrong within the closed C++ DLL that scatters the heap. I will contact de manufacturer again. – Separatrix

Sorry - that's not entirely unexpected. Hopefully the mfr. can take your test code and debug using their source. If not you could always update the question with more details (i.e. what's the crash, does it crash at all while debugger is attached, does it crash running unattached when GetDeviceInfo is not called). – Fencible

Answer 2

A correct incantation is

string UnpackFixed(byte[] data, System.Text.Encoding encoding)
{
    int i;
    for (i = 0; i < data.Length; ++i)
        if(data[i] == (byte)0)
            break;
    return encoding.GetString(data, i);
}

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
struct DeviceInfo
{
    uint32 maxScanrate;
    uint32 minScanrate;
    uint32 maxNumOfPoints;
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 32)]
    byte type[];
};

DeviceInfo pDevInfo = new DeviceInfo();
pDevInfo.type = new byte[32];
int r4 = GetDeviceInfo(0, ref pDevInfo);
Console.WriteLine("  - type: " + UnpackFixed(pDevInfo.type));

I'm certain there is a way to do this with string but all the old obvious ways of doing it tended to pass the string to native code and get nothing back. Here, the exercise is to get a fixed-length byte string back. If you do solve it for string you will end up using System.Text.Encoding.Default which may or may not be right and there's no way to override it.

System.Text.Encoding.ASCII is plausibly wrong, in which case you need to deal with encodings. System.Text.Encoding.Default might work where ASCII didn't, in which case you should consider if you have weird failure modes on multi-byte character encodings. It's not clear if the device always uses the same encoding as the OS or if it assumes a fixed encoding (in which case you should specify the encoding).

Clipfed

There's no additional information about how the char[] is encoded. Please have a look at the linked PDF file in the OP. It says that GetDeviceIndentifier() uses 2B Unicode text, BUT there's no hint how the char[] in the struct DeviceInfo is encoded. Nevertheless, perhaps you're on the right way and I will do some testing now, according to your idea. Thanks so far! – Separatrix

Answer 3

I think you got a problem with public string type in DeviceInfo. If you needed to pass a string to the native part, that would be fine, but I understand that you're getting a char* from (allocated by) the native part, and in that case you're losing the address of type that is managed (and that cannot be known).

The right way to do this would be to have:

[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
public struct DeviceInfo {
    public UInt32 maxScanrate;
    public UInt32 minScanrate;
    public UInt32 maxNumOfPoints;
    public IntPtr type; // HERE: char*
}

and then process type in the managed part, for instance like this:

unsafe void GetType(IntPtr strPtr) => return new string((char*)strPtr);

If the native part doesn't do the allocation you'll need to use Marshal.AllocHGlobal.

Lovash

I don't know who and why your answer has/was downvoted. I now tried your approach and I get the first three fields back (maxScanrate, minScanrate and maxNumOfPoints) successfully :-) It's a step into the correct direction... :-) But I finally get this exception: An unhandled exception of type 'System.ExecutionEngineException' occurred in Unknown Module. – Separatrix

I also tried to use string devtype = Marshal.PtrToStringUni(pDevInfo.type); to unmarshal the IntPtr but then getting Exception thrown: 'System.AccessViolationException' in mscorlib.dll Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.. – Separatrix

Did you try the IntPtr type and the unsafe void GetType method ? (we should add nullptr check, but let's assume first that the char* was not disposed at the time) – Lovash

Which .NET version are you using ? – Lovash

I am using .NET 4.6. But the type in DeviceInfo is defined as char[] and not char*. Doesn't that make a difference? – Separatrix

I did experiment with Marshal.AllocHGlobal, but unsafe void GetType(IntPtr strPtr) => return new string((char*)strPtr); does not compile. Is this some kind of functional representation? – Separatrix

You need to Allow unsafe code in C# project properties / Build. It's just the fastest way to get through pinvoke char* or char[]. No difference here. About difference: #1704907 – Lovash

@Separatrix Even though you had Peter Wishart hack, can you please try the unsafe method ? Thanks – Lovash

Sure! Unsafe code allowed ticked in project properties. I then already did try and now once again. Unfortunately I get An unhandled exception of type 'System.ExecutionEngineException' occurred in Unknown Module. with your solution. Any ideas what to change? – Separatrix

@Separatrix It's a problem happening in CLR. I think it's because the storage of char[] initialized with new (c++) is not continuous unlike a char* initialized with malloc (c). Peter Wishart's hack is dealing with this problem. Even though it's giving the result you want, it's not very much satisfaying. I'll think about it. – Lovash

You‘re right. The other solution does only work when running from Visual Studio. When running the executable, it does crash too... I believe the problem is within the C++ DLL. I will contact the manufacturer again. – Separatrix

@Separatrix "the other solution" = ? – Lovash