The fastest way to check if a type is blittable?
Asked Answered
C

9

20

In my serialiser/deserialiser, I have the following snippet:

    if (element_type.IsValueType && collection_type.IsArray)
    {
        try
        {
            GCHandle h = GCHandle.Alloc(array_object, GCHandleType.Pinned);
            int arrayDataSize = Marshal.SizeOf(element_type) * c.Count;
            var array_data = new byte[arrayDataSize];
            Marshal.Copy(h.AddrOfPinnedObject(), array_data, 0, arrayDataSize);
            h.Free();
            WriteByteArray(array_data);

            return;
        }
        catch (ArgumentException)
        {
            //if the value type is not blittable, then we need to serialise each array item one at a time
        }
    }

The purpose of which is to try and write an array of value types to a stream, in the most efficient way possible (that is, just the content as a bunch of bytes).

The problem comes when the type is a value type but not blittable, and Alloc() fails. At the moment the exception is caught and control passed to code which deals with the array as if it consisted of reference types.

This check however (due to the throwing and catching of the exception which I understand is very slow) is proving to be a severe bottleneck due to the number of value types that are encountered in my application. So I am wondering, what is the fastest way to check if a type is blittable?

Choctaw answered 13/5, 2012 at 19:42 Comment(1)
I had the same problem, I've ended up caching results for each type (e.g. in static dictionary). Checking was done same as here, try/catch.Kinny
K
8

I'm using generic class to cache results. Test is done in same way (trying to allocate pinned handle).

public static class BlittableHelper<T>
{
    public static readonly bool IsBlittable;

    static BlittableHelper()
    {
        try
        {
            // Class test
            if (default(T) != null)
            {
                // Non-blittable types cannot allocate pinned handle
                GCHandle.Alloc(default(T), GCHandleType.Pinned).Free();
                IsBlittable = true;
            }
        }
        catch { }
    }
}
Kinny answered 15/2, 2015 at 22:22 Comment(5)
Caching is what I ended up doing, though I think your caching technique here is the most efficient I have seen!Choctaw
Note that this won't work on Mono, because GCHandle.Alloc doesn't throw an exception for non blittable types. See github.com/mono/mono/pull/4533Furmark
@JayLemmon If you're using Unity, there's UnsafeUtility.IsBlittable. Otherwise you'd probably have to "walk the fields" recursively.Kinny
This says that int[] isn't blittable, although learn.microsoft.com/en-us/dotnet/framework/interop/… explicitly says that a one-dimensional array of integers is. Am I missing something here, or does that default(T) != null check need to go? (Per the same reference, there are situations where a class with only blittable members can be blittable, depending on how it's marshalled.)Ultimo
@MattTsōnto The contents of the int array is blittable, but the reference to array itself (stored in int[] variable) is not blittable.Kinny
P
10

The current answer works for the questioner's case but, according to the specification, arrays of blittable value types are also blittable types themselves. Extended Ondřej's method a bit, so it takes this into account, and also works for reference types:

public static bool IsBlittable<T>()
{
    return IsBlittableCache<T>.Value;
}

public static bool IsBlittable(Type type)
{
    if(type.IsArray)
    {
        var elem = type.GetElementType();
        return elem.IsValueType && IsBlittable(elem);
    }
    try{
        object instance = FormatterServices.GetUninitializedObject(type);
        GCHandle.Alloc(instance, GCHandleType.Pinned).Free();
        return true;
    }catch{
        return false;
    }
}

private static class IsBlittableCache<T>
{
    public static readonly bool Value = IsBlittable(typeof(T));
}

As a side effect, this returns (albeit correctly) false for string, because GetUninitializedObject can't create it. Assuming Alloc really checks for blittability (except for string), this should be reliable.

Pretermit answered 17/7, 2015 at 21:52 Comment(5)
This will return false with int[] which is nevertheless blittable. Remove the NOT from !elem.IsValueType to fix :)Anaplastic
@Anaplastic Thank you!Pretermit
@IllidanS4supportsMonica: This fails to detect a struct that has been set up for marshaling, e.g. StructLayout(LayoutKind.Sequential), and MarshalAs() attributes on each field. On the other hand, a test involving Marshal.SizeOf(), creating an unmanaged buffer of that size with any number of techniques, and then checking if Marshal.PtrToStructure() succeeds? What do you think?Ultravirus
@Ultravirus Blittable does not mean marshallable. The fact that you are setting MarshalAs on fields in the first place indicates that such a struct cannot be blittable.Pretermit
@IllidanS4supportsMonica: Fair enough...I suppose my needs are slightly different. Thanks for the clarification.Ultravirus
K
8

I'm using generic class to cache results. Test is done in same way (trying to allocate pinned handle).

public static class BlittableHelper<T>
{
    public static readonly bool IsBlittable;

    static BlittableHelper()
    {
        try
        {
            // Class test
            if (default(T) != null)
            {
                // Non-blittable types cannot allocate pinned handle
                GCHandle.Alloc(default(T), GCHandleType.Pinned).Free();
                IsBlittable = true;
            }
        }
        catch { }
    }
}
Kinny answered 15/2, 2015 at 22:22 Comment(5)
Caching is what I ended up doing, though I think your caching technique here is the most efficient I have seen!Choctaw
Note that this won't work on Mono, because GCHandle.Alloc doesn't throw an exception for non blittable types. See github.com/mono/mono/pull/4533Furmark
@JayLemmon If you're using Unity, there's UnsafeUtility.IsBlittable. Otherwise you'd probably have to "walk the fields" recursively.Kinny
This says that int[] isn't blittable, although learn.microsoft.com/en-us/dotnet/framework/interop/… explicitly says that a one-dimensional array of integers is. Am I missing something here, or does that default(T) != null check need to go? (Per the same reference, there are situations where a class with only blittable members can be blittable, depending on how it's marshalled.)Ultimo
@MattTsōnto The contents of the int array is blittable, but the reference to array itself (stored in int[] variable) is not blittable.Kinny
C
4

The excellent code by @IllidanS4 on this page incorrectly returns false for arrays where the element is a blittable formatted type, meaning that the array is blittable also. Starting from that example, I fixed that problem and added handling for a few more mishandled cases, such as:

  • T[] where T : formatted-type (just mentioned)
  • jagged arrays int[][][]...
  • enums (but not System.Enum ittself)
  • interfaces, abstract types
  • generic types (never blittable).

I also added made the cases for avoiding the expensive Exception block a bit more exhaustive and ran unit tests for all the different kinds of types I could think of.

public static bool IsBlittable(this Type T)
{
    while (T.IsArray)
        T = T.GetElementType();

    bool b;
    if (!((b = T.IsPrimitive || T.IsEnum) || T.IsAbstract || T.IsAutoLayout || T.IsGenericType))
        try
        {
            GCHandle.Alloc(FormatterServices.GetUninitializedObject(T), GCHandleType.Pinned).Free();
            b = true;
        }
        catch { }
    return b;
}

The nice caching mechanism from the other answer should be used as-is.

Concurrence answered 7/2, 2017 at 9:18 Comment(1)
Nice idea to check for other types. There is only a slight mistake, bool and char, while primitive, aren't blittable (size depends on platform). Also jagged arrays shouldn't be blittable, as they are arrays of object references. Neither are multidimensional arrays, per MSDN, though my code has the same issue.Pretermit
S
1

I don't have enough reputation for adding a comment, so I will write my comment as an answer:

I've tested the code proposed by @IS4 and his function says a string is not blittable, which is correct. However, when using the Mono backend in Unity, his implementation also says a struct with a string field is blittable (which is not correct).

I've also tested the Unity's UnsafeUtility.IsBlittable() function and it returns the correct value for those structs, so if we want to implement an IsBlittable() function which works correctly on Mono, I think we have no choice but using Reflection to ensure all the fields in the struct are blittable too.

I have tested this implementation in Unity 2017.4 and Unity 2018.4 using the Mono scripting backend and it seems to work correctly with all the types I've tried so far:

using System;
using System.Reflection;
using System.Runtime.Serialization;
using System.Runtime.InteropServices;

public static class BlittableHelper
{
#if UNITY_2018_1_OR_NEWER || UNITY_2019_1_OR_NEWER || UNITY_2020_1_OR_NEWER
    // If we're using Unity, the simplest solution is using
    // the built-in function
    public static bool IsBlittableType(Type type)
    {
        return Unity.Collections.LowLevel.Unsafe.UnsafeUtility.IsBlittable(
            type
        );
    }
#else
    // NOTE: static properties are not taken into account when
    // deciding whether a type is blittable, so we only need
    // to check the instance fields and properties.
    private static BindingFlags Flags =
    BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance;

    public static bool IsBlittableType(Type type)
    {
        // According to the MSDN, one-dimensional arrays of blittable
        // primitive types are blittable.
        if (type.IsArray)
        {
            // NOTE: we need to check if elem.IsValueType because
            // multi-dimensional (jagged) arrays are not blittable.
            var elem = type.GetElementType();
            return elem.IsValueType && IsBlittableType(elem);
        }

        // These are the cases which the MSDN states explicitly
        // as blittable.
        if
        (
            type.IsEnum
            || type == typeof(Byte)
            || type == typeof(SByte)
            || type == typeof(Int16)
            || type == typeof(UInt16)
            || type == typeof(Int32)
            || type == typeof(UInt32)
            || type == typeof(Int64)
            || type == typeof(UInt64)
            || type == typeof(IntPtr)
            || type == typeof(UIntPtr)
            || type == typeof(Single)
            || type == typeof(Double)
        )
        {
            return true;
        }


        // These are the cases which the MSDN states explicitly
        // as not blittable.
        if
        (
            type.IsAbstract
            || type.IsAutoLayout
            || type.IsGenericType
            || type == typeof(Array)
            || type == typeof(Boolean)
            || type == typeof(Char)
            //|| type == typeof(System.Class)
            || type == typeof(Object)
            //|| type == typeof(System.Mdarray)
            || type == typeof(String)
            || type == typeof(ValueType)
            || type == typeof(Array)
            //|| type == typeof(System.Szarray)
        )
        {
            return false;
        }


        // If we've reached this point, we're dealing with a complex type
        // which is potentially blittable.
        try
        {
            // Non-blittable types are supposed to throw an exception,
            // but that doesn't happen on Mono.
            GCHandle.Alloc(
                FormatterServices.GetUninitializedObject(type),
                GCHandleType.Pinned
            ).Free();

            // So we need to examine the instance properties and fields
            // to check if the type contains any not blittable member.
            foreach (var f in type.GetFields(Flags))
            {
                if (!IsBlittableTypeInStruct(f.FieldType))
                {
                    return false;
                }
            }

            foreach (var p in type.GetProperties(Flags))
            {
                if (!IsBlittableTypeInStruct(p.PropertyType))
                {
                    return false;
                }
            }

            return true;
        }
        catch
        {
            return false;
        }
    }

    private static bool IsBlittableTypeInStruct(Type type)
    {
        if (type.IsArray)
        {
            // NOTE: we need to check if elem.IsValueType because
            // multi-dimensional (jagged) arrays are not blittable.
            var elem = type.GetElementType();
            if (!elem.IsValueType || !IsBlittableTypeInStruct(elem))
            {
                return false;
            }

            // According to the MSDN, a type that contains a variable array
            // of blittable types is not itself blittable. In other words:
            // the array of blittable types must have a fixed size.
            var property = type.GetProperty("IsFixedSize", Flags);
            if (property == null || !(bool)property.GetValue(type))
            {
                return false;
            }
        }
        else if (!type.IsValueType || !IsBlittableType(type))
        {
            // A type can be blittable only if all its instance fields and
            // properties are also blittable.
            return false;
        }

        return true;
    }
#endif
}

// This class is used as a caching mechanism to improve performance.
public static class BlittableHelper<T>
{
    public static readonly bool IsBlittable;

    static BlittableHelper()
    {
        IsBlittable = BlittableHelper.IsBlittableType(typeof(T));
    }
}
Shakitashako answered 15/2, 2021 at 9:56 Comment(2)
This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From ReviewLepido
Sorry, it was my first contribution to this site. I have spent some time doing more tests in order to provide a more useful answer.Chace
A
1

Starting with netcore2.0 there is System.Runtime.CompilerServices.RuntimeHelpers.IsReferenceOrContainsReferences<T> that allows you to check whether type is blittable

static bool IsBlittable<T>()
   => !RuntimeHelpers.IsReferenceOrContainsReferences<T>();

static bool IsBlittable(Type type)
{
    return (bool)typeof(RuntimeHelpers)
               .GetMethod(nameof(RuntimeHelpers.IsReferenceOrContainsReferences))
               .MakeGenericMethod(type)
               .Invoke(null, null);
}

I use this implementation to send arrays over network

ValueTask SendAsync<T>(T[] array, CancellationToken token) where T : unmanaged
{
     // zero allocations, no <AllowUnsafeBlocks> required
     return _stream.WriteAsync(MemoryMarshal.AsBytes((ReadOnlySpan<T>)array, token);
}

Unmanaged constraint enforces usage of blittable types. Reference

Alcinia answered 3/6, 2021 at 11:30 Comment(3)
This gives incorrect results. For example, it claims that bool is blittable and the int[] is not.Ultimo
@MattTsōnto , OP looking is for a way to write generic data into stream, not COM-interop. So I think that is correct (in terms of what C# and F# finds correct) for serialization scenario, but not for COM-interop. Maybe term blittable is not the right oneAlcinia
@JL0PD: The OP wants to avoid the exception when using GCHandle.Alloc even on totally unmanaged but still non-blittable types, such as bool, char, DateTime, decimal, etc. This has nothing to do with COM interop. The issue is not how to check whether a value type can be safely serialized but that GCHandle.Alloc refuses pinning some non-blittable objects even if they could be safely serialized.Wrest
S
0

Use http://msdn.microsoft.com/en-us/library/system.type.islayoutsequential.aspx and http://msdn.microsoft.com/en-us/library/system.type.isexplicitlayout.aspx:

element_type.IsValueType && collection_type.IsArray && (element_type.IsLayoutSequential || element_type.IsExplicitLayout)
Schick answered 13/5, 2012 at 22:34 Comment(1)
Thanks but unfortunately this does not work. The IsLayoutSequential property is true for at least one non-blittable type I tried (a simple struct with a string).Choctaw
E
0

Fastest way would be not allocating but reusing existing GCHandle like:

var gch = GCHandle.Alloc(null, GCHandleType.Pinned);
gch.Target = new byte[0];
gch.Target = "";

GCHandle.Alloc allocates or reuses existing slot each time with taking locks which is relatively expensive operation. And static readonly primitive types becomes constants when jitting but dont store GCHandle in generic type because each generic instantination will take its own copy.

Eightieth answered 10/3, 2019 at 8:41 Comment(0)
P
0

This works for me:

static bool IsBlittable(Type t)
{
  if (t.IsPrimitive) return true; if (!t.IsValueType) return false;
  var a = t.GetFields(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic);
  for (int i = 0; i < a.Length; i++) if (!IsBlittable(a[i].FieldType)) return false;
  return true;
}
Preraphaelite answered 13/6, 2021 at 5:57 Comment(1)
This gives incorrect results. For example, it says that bool is blittable and that int[] is not.Ultimo
C
0

Here's an alternative that's just a straightforward representation of what Microsoft's documentation says. It's not short, but it handles more cases correctly than the other solutions here. If you're concerned about the performance of the Reflection calls, you can wrap this in a simple cache.

static bool IsBlittable(Type type)
    => IsBlittablePrimitive(type)
    || IsBlittableArray(type)
    || IsBlittableStruct(type)
    || IsBlittableClass(type);
static bool IsBlittablePrimitive(Type type)
    => type == typeof(byte)
    || type == typeof(sbyte)
    || type == typeof(short)
    || type == typeof(ushort)
    || type == typeof(int)
    || type == typeof(uint)
    || type == typeof(long)
    || type == typeof(ulong)
    || type == typeof(System.IntPtr)
    || type == typeof(System.UIntPtr)
    || type == typeof(float)
    || type == typeof(double)
    ;
static bool IsBlittableArray(Type type)
    => type.IsArray
    && type.GetArrayRank() == 1
    && IsBlittablePrimitive(type.GetElementType())
    ;
static bool IsBlittableStruct(Type type)
    => type.IsValueType
    && !type.IsPrimitive
    && type.IsLayoutSequential
    && type.GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public).All(IsBlittableField);
static bool IsBlittableClass(Type type)
    => !type.IsValueType
    && !type.IsPrimitive
    && type.IsLayoutSequential
    && type.GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public).All(IsBlittableField);
static bool IsBlittableField(FieldInfo field)
    => IsBlittablePrimitive(field.FieldType) 
    || IsBlittableStruct(field.FieldType);

Test cases:

Is blittable?
- Int32: True
- Int32[]: True
- Int32[,]: False
- Int32[][]: False
- String: False
- String[]: False
- Boolean: False
- String: False
- Byte[]: True
- struct X { public int x; }: True
- struct Y { public int[] Foo { get; set; } }: False
- class CAuto { public int X { get; set; } }: False
- [StructLayout(LayoutKind.Sequential)]class CSeq { public int X { get; set; } }: True

Note: This reports Span as blittable, which seems unlikely to me, but I don't know for sure.

Cameleer answered 28/6, 2021 at 1:32 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.