C#: convert generic pointer to array

Question

I want to convert a byte* to a byte[], but I also want to have a reusable function to do this:

public unsafe static T[] Create<T>(T* ptr, int length)
{
    T[] array = new T[length];

    for (int i = 0; i < length; i++)
        array[i] = ptr[i];

    return array;
}

Unfortunately I get a compiler error because T might be a ".NET managed type" and we can't have pointers to those. Even more frustrating is that there is no generic type constraint which can restrict T to "unmanaged types". Is there a built-in .NET function to do this? Any ideas?

Accad

Answer 1

The method that could match what you are trying to do is Marshal.Copy, but it does not take the appropriate parameters to make a generic method.

Although there it is not possible to write a generic method with generic constraints that could describe what is possible, not every type can be allowed to be copied using an "unsafe" way. There are some exceptions; classes are one of these.

Here is a sample code:

    public unsafe static T[] Create<T>(void* source, int length)
    {
        var type = typeof(T);
        var sizeInBytes =  Marshal.SizeOf(typeof(T));

        T[] output = new T[length];

        if (type.IsPrimitive)
        {
            // Make sure the array won't be moved around by the GC 
            var handle = GCHandle.Alloc(output, GCHandleType.Pinned);

            var destination = (byte*)handle.AddrOfPinnedObject().ToPointer();
            var byteLength = length * sizeInBytes;

            // There are faster ways to do this, particularly by using wider types or by 
            // handling special lengths.
            for (int i = 0; i < byteLength; i++)
                destination[i] = ((byte*)source)[i];

            handle.Free();
        }
        else if (type.IsValueType)
        {
            if (!type.IsLayoutSequential && !type.IsExplicitLayout)
            {
                throw new InvalidOperationException(string.Format("{0} does not define a StructLayout attribute", type));
            }

            IntPtr sourcePtr = new IntPtr(source);

            for (int i = 0; i < length; i++)
            {
                IntPtr p = new IntPtr((byte*)source + i * sizeInBytes);

                output[i] = (T)System.Runtime.InteropServices.Marshal.PtrToStructure(p, typeof(T));
            }
        }
        else 
        {
            throw new InvalidOperationException(string.Format("{0} is not supported", type));
        }

        return output;
    }

    unsafe static void Main(string[] args)
    {
        var arrayDouble = Enumerable.Range(1, 1024)
                                    .Select(i => (double)i)
                                    .ToArray();

        fixed (double* p = arrayDouble)
        {
            var array2 = Create<double>(p, arrayDouble.Length);

            Assert.AreEqual(arrayDouble, array2);
        }

        var arrayPoint = Enumerable.Range(1, 1024)
                                   .Select(i => new Point(i, i * 2 + 1))
                                   .ToArray();

        fixed (Point* p = arrayPoint)
        {
            var array2 = Create<Point>(p, arrayPoint.Length);

            Assert.AreEqual(arrayPoint, array2);
        }
    }

The method can be generic, but it cannot take a pointer of a generic type. This is not an issue since pointers covariance is helping, but this has the unfortunate effect of preventing an implicit resolution of the generic argument type. You then have to specify MakeArray explicitly.

I've added a special case for the structures, where it is best to have types that specify a struct layout. This might not be an issue in your case, but if the pointer data is coming from native C or C++ code, specifying a layout kind is important (The CLR might choose to reorder fields to have a better memory alignment).

But if the pointer is coming exclusively from data generated by managed code, then you can remove the check.

Also, if the performance is an issue, there are better algorithms to copy the data than doing it byte by byte. (See the countless implementations of memcpy for reference)

Kinin

Answer 2

Seems that the question becomes: How to specify a generic Type to be a simple type.

unsafe void Foo<T>() : where T : struct
{
   T* p;
}

Gives the error:
Cannot take the address of, get the size of, or declare a pointer to a managed type ('T')

Bickering

Answer 3

How about this?

static unsafe T[] MakeArray<T>(void* t, int length, int tSizeInBytes) where T:struct
{
    T[] result = new T[length];
    for (int i = 0; i < length; i++)
    {
        IntPtr p = new IntPtr((byte*)t + (i * tSizeInBytes));
        result[i] = (T)System.Runtime.InteropServices.Marshal.PtrToStructure(p, typeof(T));
    }

    return result;
}

We can't use sizeof(T) here, but the caller can do something like

byte[] b = MakeArray<byte>(pBytes, lenBytes, sizeof(byte));

Selfacting

you can use Marshal.SizeOf(typeof(T)); to get size. – Kaule

Answer 4

As of C# 7.3 this is possible as follows:

public unsafe static T[] Create<T>(T* ptr, int length) where T: unmanaged
{
    T[] array = new T[length];
    for (int i = 0; i < length; i++)
        array[i] = ptr[i];
    return array;
}

Monoceros

This makes a full copy – Trilley

Answer 5

I have no idea whatsoever if the following would work, but it might (at least it compiles :):

public unsafe static T[] Create<T>(void* ptr, int length) where T : struct
{
    T[] array = new T[length];

    for (int i = 0; i < length; i++)
    {
        array[i] = (T)Marshal.PtrToStructure(new IntPtr(ptr), typeof(T));
    }

    return array;
}

The key is to use Marshal.PtrToStructure to convert to the correct type.

Herbivore

This doesn't work because T might not be a struct - byte, int, long, etc. – Accad

Looks close but I don't see the pointer incrementing - it will always use the first byte. Also, is there a reason for void* ? – Bickering

'struct' means value type, so int, byte etc are valid. And you can't have a pointer to a none value type, so struct is the best you can do. – Herbivore

void* was used because you don't know what type the pointer is. I know very little about pointers to be honest - but this hopefully might get whoever is trying this in the right direction. – Herbivore

The reason is that structs may also contain references to classes, and the compiler then considers the struct to be ".NET managed" – Accad

Calling PtrToStructure in a loop can get inefficient if dealing with large arrays. Consider copying all the arrray data at once using for example a call to kernel32.dll!RtlMoveMemory. – Incommodity