C++ union in C# — weird behaviour

Question

I am trying to create some vhd/vhdx files using the VHD API in C#.

There's a C++ union that looks like this:

typedef struct _CREATE_VIRTUAL_DISK_PARAMETERS
{
    CREATE_VIRTUAL_DISK_VERSION Version;

    union
    {
        struct
        {
            GUID                  UniqueId;
            ULONGLONG             MaximumSize;
            ULONG                 BlockSizeInBytes;
            ULONG                 SectorSizeInBytes;
            PCWSTR                ParentPath;
            PCWSTR                SourcePath;
        } Version1;

        struct
        {
            GUID                   UniqueId;
            ULONGLONG              MaximumSize;
            ULONG                  BlockSizeInBytes;
            ULONG                  SectorSizeInBytes;
            ULONG                  PhysicalSectorSizeInBytes;
            PCWSTR                 ParentPath;
            PCWSTR                 SourcePath;
            OPEN_VIRTUAL_DISK_FLAG OpenFlags;
            VIRTUAL_STORAGE_TYPE   ParentVirtualStorageType;
            VIRTUAL_STORAGE_TYPE   SourceVirtualStorageType;
            GUID                   ResiliencyGuid;
        } Version2;

        struct
        {
            GUID                   UniqueId;
            ULONGLONG              MaximumSize;
            ULONG                  BlockSizeInBytes;
            ULONG                  SectorSizeInBytes;
            ULONG                  PhysicalSectorSizeInBytes;
            PCWSTR                 ParentPath;
            PCWSTR                 SourcePath;
            OPEN_VIRTUAL_DISK_FLAG OpenFlags;
            VIRTUAL_STORAGE_TYPE   ParentVirtualStorageType;
            VIRTUAL_STORAGE_TYPE   SourceVirtualStorageType;
            GUID                   ResiliencyGuid;
            PCWSTR                 SourceLimitPath;
            VIRTUAL_STORAGE_TYPE   BackingStorageType;
        } Version3;
    };
} CREATE_VIRTUAL_DISK_PARAMETERS, *PCREATE_VIRTUAL_DISK_PARAMETERS;

I'm trying to convert that to C#, but not having much luck. I'm not interested in Version3 at all, so am leaving that out.

I've tried a number of things and the best I could get to was having Version2 working (by doing something really bizarre), but I've never managed to get Version1 and Version2 working at the same time.

The solution that has wielded the best results so far has been this, but there has to be something wrong there because Version1 simply doesn't work, and SectorSizeInBytes in Version1 is a ulong rather than uint (if I change it to uint like it should be, I break Version2 and Version1 still doesn't work!)

[StructLayout(LayoutKind.Explicit, CharSet = CharSet.Unicode)]
public struct CreateVirtualDiskParameters
{
    [FieldOffset(0)] public CreateVirtualDiskParametersVersion1 Version1;

    [FieldOffset(0)] public CreateVirtualDiskParametersVersion2 Version2;
}

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct CreateVirtualDiskParametersVersion1
{
    public CreateVirtualDiskVersion Version;
    public Guid UniqueId;
    public ulong MaximumSize;
    public uint BlockSizeInBytes;
    public ulong SectorSizeInBytes;
    public string ParentPath;
    public string SourcePath;
}

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct CreateVirtualDiskParametersVersion2
{
    public CreateVirtualDiskVersion Version;
    public Guid UniqueId;
    public ulong MaximumSize;
    public uint BlockSizeInBytes;
    public uint SectorSizeInBytes;
    public uint PhysicalSectorSizeInBytes;
    public string ParentPath;
    public string SourcePath;
    public OpenVirtualDiskFlags OpenFlags;
    public VirtualStorageType ParentVirtualStorageType;
    public VirtualStorageType SourceVirtualStorageType;
    public Guid ResiliencyGuid;
}

I know theoretically the Version field should be set outside the Version structs and I have tried that as well, but it just breaks things even more funnily enough...

So, can someone advise how to properly translate the above to C#, leaving out the Version3 struct as that's not needed?

Horsa

ULONGLONG (64 bit) and ULONG (32 bit) both seem to be mapped to ulong (64 bit). – Plasia

Have you tried printing out the offsets of the members of your C++ structs? – Wrinkle

@TheodorosChatzigiannakis not quite sure how I'd do that bearing in mind it contains enums/structs and strings. I mean, what size would they be? I'm open to trying it, let me know what values to use and I'll give it a shot. – Horsa

@Horsa I mean by using the offsetof macro. Let it print the offsets for you and define those offsets explicitly on the C# side. – Wrinkle

One problem is that ParentPath and SourcePath are raw pointers in the native structs, but the string class in the C# code, which has more than just one pointer in it. – Cousteau

Don't define offsets explicitly. The compiler can do that. You need to check them though. – Reasonable

@1201 string marshals as a pointer to null terminated array. – Reasonable

Answer 1

Using Pack = 1 to StructLayout attributes eliminates any padding between struct members. In TCP connections structs are usually passed around without padding so that all programs using the struct can agree on its layout in memory.

However as @David Heffernan pointed out, that may not be the case when passing structs to Windows DLL's. I didn't test the actual call to CreateVirtualDisk because it seemed a bit risky, given that I haven't used this call before and didn't want to clobber my disk if I made a mistake. It looks as if the default packing of 8 bytes (Pack = 0 for default or Pack = 8) may be the correct setting, based on the following quote.

See 64-bit Windows API struct alignment caused Access Denied error on named pipe

The Windows SDK expects packing to be 8 bytes. From Using the Windows Headers

Projects should be compiled to use the default structure packing, which is currently 8 bytes because the largest integral type is 8 bytes. Doing so ensures that all structure types within the header files are compiled into the application with the same alignment the Windows API expects. It also ensures that structures with 8-byte values are properly aligned and will not cause alignment faults on processors that enforce data alignment.

Version is moved to the top of CreateVirtualDiskParameters. The two unions then follow. Both have the same offset sizeof(CREATE_VIRTUAL_DISK_VERSION).

Also SectorSizeInBytes is uint rather than ulong.

You can let the marshaller do the work of filling string members using the attribute, eg

[MarshalAs(UnmanagedType.LPWStr)] public string ParentPath;

Or, you can represent it as it appears in memory, which is a pointer to a Unicode string:

public IntPtr ParentPath;

and then extract the string yourself with

Marshal.PtrToStringAuto(vdp.Version1.ParentPath)

If you're passing the C# struct to an external DLL, populate it with an unmanaged string

vdp.Version1.ParentPath = (IntPtr)Marshal.StringToHGlobalAuto("I am a managed string");

then free the unmanaged string when you're finished with it

Marshal.FreeHGlobal(vdp.Version1.ParentPath);

Try this.

public enum CREATE_VIRTUAL_DISK_VERSION
{
    CREATE_VIRTUAL_DISK_VERSION_UNSPECIFIED = 0,
    CREATE_VIRTUAL_DISK_VERSION_1 = 1,
    CREATE_VIRTUAL_DISK_VERSION_2 = 2
};
public enum OPEN_VIRTUAL_DISK_FLAG
{
    OPEN_VIRTUAL_DISK_FLAG_NONE = 0x00000000,
    OPEN_VIRTUAL_DISK_FLAG_NO_PARENTS = 0x00000001,
    OPEN_VIRTUAL_DISK_FLAG_BLANK_FILE = 0x00000002,
    OPEN_VIRTUAL_DISK_FLAG_BOOT_DRIVE = 0x00000004,
    OPEN_VIRTUAL_DISK_FLAG_CACHED_IO = 0x00000008,
    OPEN_VIRTUAL_DISK_FLAG_CUSTOM_DIFF_CHAIN = 0x00000010
};

[StructLayout(LayoutKind.Sequential, Pack = 8, CharSet = CharSet.Unicode)]
public struct VIRTUAL_STORAGE_TYPE
{
    uint DeviceId;
    Guid VendorId;
};

[StructLayout(LayoutKind.Explicit, Pack = 8, CharSet = CharSet.Unicode)]
public struct CreateVirtualDiskParameters
{
    [FieldOffset(0)]
    public CREATE_VIRTUAL_DISK_VERSION Version;

    [FieldOffset(8))]
    public CreateVirtualDiskParametersVersion1 Version1;

    [FieldOffset(8))]
    public CreateVirtualDiskParametersVersion2 Version2;
}

[StructLayout(LayoutKind.Sequential, Pack = 8, CharSet = CharSet.Unicode)]
public struct CreateVirtualDiskParametersVersion1
{
    public Guid UniqueId;
    public ulong MaximumSize;
    public uint BlockSizeInBytes;
    public uint SectorSizeInBytes;
    //public IntPtr ParentPath;   // PCWSTR in C++ which is a pointer to a Unicode string
    //public IntPtr SourcePath;   //string
    [MarshalAs(UnmanagedType.LPWStr)] public string ParentPath;
    [MarshalAs(UnmanagedType.LPWStr)] public string SourcePath;
}

[StructLayout(LayoutKind.Sequential, Pack = 8, CharSet = CharSet.Unicode)]
public struct CreateVirtualDiskParametersVersion2
{
    public Guid UniqueId;
    public ulong MaximumSize;
    public uint BlockSizeInBytes;
    public uint SectorSizeInBytes;
    public uint PhysicalSectorSizeInBytes;
    //public IntPtr ParentPath;   //string
    //public IntPtr SourcePath;   //string
    [MarshalAs(UnmanagedType.LPWStr)] public string ParentPath;
    [MarshalAs(UnmanagedType.LPWStr)] public string SourcePath;
    public OPEN_VIRTUAL_DISK_FLAG OpenFlags;
    public VIRTUAL_STORAGE_TYPE ParentVirtualStorageType;
    public VIRTUAL_STORAGE_TYPE SourceVirtualStorageType;
    public Guid ResiliencyGuid;
}

Pleasant

Thank you John D that's great, but can I just ask why the [FieldOffset(sizeof(CREATE_VIRTUAL_DISK_VERSION))]? CREATE_VIRTUAL_DISK_VERSION is an enum (of int) so it should have a size of 4 bytes right regardless of bitness? – Horsa

Structs are usually aligned and not packed. Are you just making this up? – Reasonable

@cogumeIO That's true currently - it was trying to handle the case where CREATE_VIRTUAL_DISK_VERSION might change (eg a major and a minor version number). In that case you'd replace it with a struct. The C++ struct definition uses CREATE_VIRTUAL_DISK_VERSION rather than ULONG so it is a possibility. – Pleasant

@David Heffernan I tested it with Visual Studio 2013 C++ and Framework 4 and structure packing was an issue. I've worked with TCP connections where request and response were defined in structs, and in the area I was In, structs in the stream were assumed to be packed. Sometimes padding bytes were added to force alignment on word boundaries. It seems to be the easiest way to get sender and receiver to agree on how to interpret the struct, when they might be produced by different compilers on machines with different architectures. – Pleasant

If you want to guess, and if the asker wants to believe you, fine. Personally, I think it's better to base answers on facts. – Reasonable