Pinning an updateble struct before passing to unmanaged code?

Question

I using some old API and need to pass the a pointer of a struct to unmanaged code that runs asynchronous.

In other words, after i passing the struct pointer to the unmanaged code, the unmanaged code copies the pointer and returns immediately. The unmanaged code can access that struct in background, in another thread. I have no control over the unmanaged code that runs in another thread nor the thread itself.

The fixed { } statement can't be used for pinning because it not designed for async unmanaged pinning.

GCHandle can pin only references, so the struct must be boxed to use GCHandle. I tried it, and it works. The main problem with it, is that you can't update the struct from managed code. To update a struct, first of all we need to unbox it, then update, then box again, but... oops ... box again?!? this means the previous pointer in the memory still point to the old non-up-to-date struct, and the new struct have another pointer, and this means that i need to pass new pointer to the unmanaged code... inapplicable in my case.

How can i pin a struct in the memory without fixed { } statement, and so that i can update it from managed code without change it's pointer?

Thanks.

Edit:

Just thought... is there a way to pin the parent object that contains the struct, and then get the pointer of the struct rather than the container object?

Radman

I would have expected that GCHandle is the solution here. If everything fails, you could allocate the memory in unmanaged code, so it's not subject to be moved by the GC. – Nona

Good question. I think i've done that in CLI/C++, I'll need to review my work notes on it. Not sure if there's a C# equivalent mechanism, however. – Duran

Maybe make the call in a background thread of your own, use fixed {} and don't ever leave the fixed {} block (at least as long as the unmanaged code might access the structure)? – Wolters

Nah, you just need to Marshal the data yourself instead of letting the GC play with it. – Pliable

@DxCK, could you please clarify the statement "How can i pin a struct in the memory without fixed { } statement, and so that i can update it without change it's pointer?"... Is it managed code, or unmanaged code that you want to have updating the memory while its pinned? – Duran

+1, Just the question I was looking for, after I realised I was a) copying what was meant to be the same struct multiple times to a set of native APIs, and b) unable to see the updates made by said APIs! – Margarite

Answer 1

Is unsafe code an option?

// allocate unmanaged memory
Foo* foo = (Foo*)Marshal.AllocHGlobal(sizeof(Foo));

// initialize struct
foo->bar = 0;

// invoke unmanaged function which remembers foo
UnsafeNativeMethods.Bar(foo);
Console.WriteLine(foo->bar);

// update struct
foo->bar = 10;

// invoke unmanaged function which uses remembered foo
UnsafeNativeMethods.Qux();
Console.WriteLine(foo->bar);

// free unmanaged memory
Marshal.FreeHGlobal((IntPtr)foo);

This compiles and doesn't throw an exception, but I don't have an unmanaged function at hand to test if it works.

From MSDN:

When AllocHGlobal calls LocalAlloc, it passes a LMEM_FIXED flag, which causes the allocated memory to be locked in place. Also, the allocated memory is not zero-filled.

Nona

Thanks, working like a charm! actually, i didnt used the AllocHGlobal and FreeHGlobal, just took the IntPtr that the GCHandle gave me and updated the struct via unsafe code. Forgive me for still not marking this as answer, because i want to wait for more possible answers, maybe there will be better solutions without unsafe code. – Radman

Answer 2

Using pinned memory in this case is not a good idea, given that the memory for the struct needs to be valid for a long time. GCHandle.Alloc() will box the structure and store it on the heap. With it being pinned, it will be a long term burden to the garbage collector as it needs to constantly find a way around the rock in the road.

The simple solution is to allocate memory for the struct in unmanaged memory. Use Marshal.SizeOf() to get the size of the structure and Marshal.AllocCoTaskMem() to allocate the memory. That gets you the pointer you need to pass to the unmanaged code. Initialize the memory with Marshal.StructureToPtr(). And read updates to the structure written by the unmanaged code with PtrToStructure().

If you do this frequently, you'll be constantly copying the structure. That could be expensive, depending on the size of the structure. To avoid that, use an unsafe pointer to access the unmanaged memory directly. Some basic syntax:

using System;
using System.Runtime.InteropServices;

class Program {
  unsafe static void Main(string[] args) {
    int len = Marshal.SizeOf(typeof(Test));
    IntPtr mem = Marshal.AllocCoTaskMem(len);
    Test* ptr = (Test*)mem;
    ptr->member1 = 42;
    // call method
    //..
    int value = ptr->member1;
    Marshal.FreeCoTaskMem(mem);
  }
  public struct Test {
    public int member1;
  }
}

Galluses

What is the difference between using all those methods: Marshal.AllocCoTaskMem and Marshal.FreeCoTaskMem VS Marshal.AllocHGlobal and Marshal.FreeHGlobal, sizeof VS Marshal.SizeOf and what is the right methods i should use in my case and why? tnx. – Radman

I can't fit the answer in a comment box. Why don't you start a new thread with that question? – Galluses

#1887788 – Radman

Answer 3

Instead of pinning, you need to use Marshal.StructureToPtr and Marshal.PtrToStructure to marshal the struct into memory that's usable in native code.

Pliable

It seems he wants to update the struct from C# after it has been passed to unmanaged code... – Nona

"StructureToPtr copies the contents of structure to the pre-allocated block of memory..." This isn't quite what the OP asked for. He wants to allow the native code to directly manipulate the items memory. I assume this is for performance reasons... Perhaps I'm mistaken. – Duran

You can't. You have to copy it to a marshal-allocated memory block, update it in C++, then copy it back to the stack. There isn't a direct way to do it otherwise. If you don't want to do this, the best option is to use C++/CLI. – Pliable

+1 Reed has it right here. See my answer for an example or check out the example section of either link that Reed has above – Gangling

Answer 4

Struct example:

[StructLayout(LayoutKind.Sequential)]
public struct OVERLAPPED_STRUCT
{
   public IntPtr InternalLow;
   public IntPtr InternalHigh;
   public Int32 OffsetLow;
   public Int32 OffsetHigh;
   public IntPtr EventHandle;
}

How to pin it to the struct and use it:

OVERLAPPED_STRUCT over_lapped = new OVERLAPPED_STRUCT();
// edit struct in managed code
over_lapped.OffsetLow = 100;
IntPtr pinned_overlap_struct = Marshal.AllocHGlobal(Marshal.SizeOf(over_lapped));
Marshal.StructureToPtr(over_lapped, pinned_overlap_struct, true);

// Pass pinned_overlap_struct to your unmanaged code
// pinned_overlap_struct changes ...

// Get resulting new struct
OVERLAPPED_STRUCT nat_ov = (OVERLAPPED_STRUCT)Marshal.PtrToStructure(pinned_overlap_struct, typeof(OVERLAPPED_STRUCT));
// See what new value is
int offset_low = nat_ov.OffsetLow;
// Clean up
Marshal.FreeHGlobal(pinned_overlap_struct);

Gangling

Answer 5

How about having the struct include an ActOnMe() interface and method something like:

delegate void ActByRef<T1,T2>(ref T1 p1, ref T2 p2);
interface IActOnMe<TT> {ActOnMe<T>(ActByRef<TT,T> proc, ref T param);}
struct SuperThing : IActOnMe<SuperThing>
{
  int this;
  int that;
  ...
  void ActOnMe<T>(ActByRef<SuperThing,T>, ref T param)
  {
    proc(ref this, ref param);
  }
}

Because the delegate takes a generic parameter by reference, it should be possible in most cases to avoid the overhead of creating closures by passing a delegate to a static method along with a reference to a struct to carry data to or from that method. Further, casting an already-boxed instance of SuperThing to IActOnMe<SuperThing> and calling ActOnMe<T> on it will expose the fields of that boxed instance for updating, as opposed to creating another copy of them as would occur with a typecast to the struct.

Interstadial

Answer 6

To answer your edit:

Just thought... is there a way to pin the parent object that contains the struct, and then get the pointer of the struct rather than the container object?

I think so. If anything, you should be able to with a managed array of structures (possibly an array of one).

Here is an example code:

    [StructLayout(LayoutKind.Sequential)]
    struct SomeStructure
    {
        public int first;
        public int second;
        public SomeStructure(int first, int second) { this.first=first; this.second=second; }
    }
    
    /// <summary>
    /// For this question on Stack Overflow:
    /// https://mcmap.net/q/935951/-pinning-an-updateble-struct-before-passing-to-unmanaged-code
    /// </summary>
    private static void TestModifiableStructure()
    {
        SomeStructure[] objArray = new SomeStructure[1];
        objArray[0] = new SomeStructure(10, 10);
        
        GCHandle hPinned = GCHandle.Alloc(objArray, GCHandleType.Pinned);

        //Modify the pinned structure, just to show we can
        objArray[0].second = 42;

        Console.WriteLine("Before unmanaged incrementing: {0}", objArray[0].second);
        PlaceholderForUnmanagedFunction(hPinned.AddrOfPinnedObject());
        Console.WriteLine("Before unmanaged incrementing: {0}", objArray[0].second);
        
        //Cleanup
        hPinned.Free();
    }
    
    //Simulates an unmanaged function that accesses ptr->second
    private static void PlaceholderForUnmanagedFunction(IntPtr ptr)
    {
        int secondInteger = Marshal.ReadInt32(ptr, 4);
        secondInteger++;
        Marshal.WriteInt32(ptr, 4, secondInteger);
    }

And its output:

Before unmanaged incrementing: 42
Before unmanaged incrementing: 43

Hyperploid

Wait, this question was ten years old? How was I even seeing it? – Hyperploid