Using assemblies compiled from IL with .NET Core & Xamarin
Asked Answered
R

3

7

Updated with a solution that works for me. See the bottom of this question.

Context:
I needed a way to evaluate the size of a generic type for the purpose of calculating array lengths to fit within a certain byte size. Basically, sizeof similar to what C/C++ provides.

C#'s sizeof and Marshal.SizeOf are not suitable for this, because of their many limitations.

With this in mind, I wrote an assembly in IL that enables the functionality I was looking for through the sizeof opcode. I'm aware that it essentially evaluates to IntPtr.Size with reference types.

I duplicated this for .NET Standard & Core, referencing what I believed were the correct equivalents of mscorlib. Note that the IL compiles fine, this question is about another issue.

Code:
Headers per target framework:

.NET: (Windows\Microsoft.NET\Framework\v4.0.30319\ilasm.exe)

.assembly extern mscorlib {}

.NET Standard: (ilasm extracted from nuget)

.assembly extern netstandard 
{ 
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89)
  .ver 0:0:0:0
}
.assembly extern System.Runtime
{
  .ver 0:0:0:0
}

.NET Core: (same ilasm as standard, though I've tested with both)

.assembly extern System.Runtime
{
  .ver 0:0:0:0
}

Source:

.assembly Company.IL
{
  .ver 0:0:1:0
}
.module Company.IL.dll

// CORE is a define for mscorlib, netstandard, and System.Runtime
.class public abstract sealed auto ansi beforefieldinit 
  Company.IL.Embedded extends [CORE]System.Object
{
  .method public hidebysig specialname rtspecialname instance void 
    .ctor() cil managed 
  {
    .maxstack 8
    ldarg.0
    call instance void [CORE]System.Object::.ctor()
    ret
  }

  .method public hidebysig static uint32 
    SizeOf<T>() cil managed 
  {
    sizeof !!0
    ret
  }
}

Problem:
When any dll compiled in this manner is referenced by a .NET Core or Xamarin application, I receive the following error:

The type 'Object' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Runtime, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'.

This issue doesn't occur when such dlls are referenced by .NET projects or .NET standard libraries which are then referenced by a .NET project.

I've read countless articles, posts, and repositories detailing this error with different versions and assemblies. The typical solution seems to be to add an explicit reference to the target framework's equivalent of mscorlib(breaking portability). There seems to be a lack of information about using IL compiled assemblies for .NET Standard & Core.

To my understanding, .NET Standard & Core use facades that forward type definitions so they may be resolved by the target framework's runtime, enabling portability.

I've tried the following:

  • Explicit versions of System.Runtime
  • Disassembling .NET Core libraries compiled from C#, using the exact same references. Oddly enough they seem to target explicit versions(such as System.Runtime 4.2 in .NET Core 2.0).
  • Generating the code dynamically using the Emit API. Compiling to memory appears to work, but isn't an option because I'm also targeting Xamarin.iOS(AOT only). Referencing such dynamically compiled assemblies from disk results in the same error as if I compiled them manually.

Update:
I attempted the solution in Jacek's answer(following the build instructions here), yet couldn't configure my system to compile corefx with the build script or VS 2017. However, after digging through the code of System.Runtime.CompilerServices.Unsafe I discovered a solution.

This probably seems obvious, but I was referencing the wrong version of System.Runtime.

Headers per target framework(copied from corefx):

.NET:

#define CORELIB "mscorlib"

.assembly extern CORELIB {}

.NET Standard:

#define CORELIB "System.Runtime"
#define netcoreapp

// Metadata version: v4.0.30319
.assembly extern CORELIB
{
  .publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
  .ver 4:0:0:0
}

.NET Core:

#define CORELIB "System.Runtime"

// Metadata version: v4.0.30319
.assembly extern CORELIB
{
  .publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
  .ver 4:0:0:0
}

In all source files, use CORELIB to reference types in mscorlib(i.e. [CORELIB]System.Object).

Riddance answered 31/10, 2017 at 0:49 Comment(6)
What was the result of your Reflection.Emit attempt?Lymphoid
@Lymphoid Updated with more details on that.Riddance
@KobyDuck: Just added the option for suppressing compilation error in project referencing IL only assembly. This may help as well.Capri
You are wasting your time on this, sizeof does not actually return the size of an object. It returns the size of a reference to the object, 4 or 8 bytes. It is fine for a value type. Not being able to get the actual size is very, very fundamental in .NET.Whereupon
@Hans Passant I understand that. It's acceptable for my use cases.Riddance
@kobyduck you can also see github.com/DotNetCross/NativeInts which contains code written entirely in IL and has a working solution.Almena
C
4

There is a very good example of how to do it correctly in the DotNet CoreFX repo. System.Runtime.CompilerServices.Unsafe is an IL only assembly and can be used by .NET Core and Xamarin.

https://github.com/dotnet/corefx/tree/master/src/System.Runtime.CompilerServices.Unsafe

There are two approaches to the problem: (i) try to recreate required elements of the build configuration in your project from scratch - what will be time consuming and very involved - corefx build system is really complex, (ii) use existing build infrastructure and create your IL project inside .NET Core CoreFX repo by replicating System.Runtime.CompilerServices.Unsafe project, changing naming and and replacing IL code with yours. In my opinion this is the fastest way to build IL based assembly which will be guaranteed to work properly with all targets.

To build your assembly while targeting any particular version of .NET Core or .NET Standard just create it in the release branches: release/2.0.0, release/1.1.0 etc.

The type 'Object' is defined in an assembly that is not referenced. You must add a reference to assembly 'System.Runtime, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'.

There is an option to try for suppressing that compilation error alone in a project referencing assembly that triggers it. Putting the following property to a new format csproj/vbproj should suppress it:

<PropertyGroup>
    <_HasReferenceToSystemRuntime>true</_HasReferenceToSystemRuntime>
</PropertyGroup>
Capri answered 4/11, 2017 at 17:34 Comment(4)
I've been incredibly busy and haven't had time to try this. I'll give this a shot later and get back to you. Thanks.Riddance
I wasn't able to compile corefx, but your answer led to me a solution that works. See my updated question for details. Your solution should work for those able to compile corefx.Riddance
This example used unreleased SDK 'Microsoft.NET.Sdk.IL'Squaw
Nope. The answer was written at the moment the 'Microsoft.NET.Sdk.IL' didn't exists yet. Usage of SDK is a new change in corefx repo added in 2nd half of 2018Capri
C
1

Ok, this is the result of my research (based on https://github.com/dotnet/corefx/tree/master/src/System.Runtime.CompilerServices.Unsafe and https://github.com/jvbsl/ILProj):

global.json:

{
  "msbuild-sdks": {
    "Microsoft.NET.Sdk.IL": "3.0.0-preview-27318-01"
  }
}

ILProj\ILProj.ilproj:

<Project Sdk="Microsoft.NET.Sdk.IL">

    <PropertyGroup>
        <TargetFramework>netstandard2.0</TargetFramework>
        <MicrosoftNetCoreIlasmPackageVersion>3.0.0-preview-27318-01</MicrosoftNetCoreIlasmPackageVersion>
        <IncludePath Condition="'$(TargetFramework)' == 'netstandard1.0'">include\netstandard</IncludePath>
        <IncludePath Condition="'$(TargetFramework)' == 'netstandard2.0'">include\netstandard</IncludePath>
        <IncludePath Condition="'$(TargetFramework)' == 'netcoreapp1.0'">include\netcoreapp</IncludePath>
        <IncludePath Condition="'$(TargetFramework)' == 'netcoreapp2.0'">include\netcoreapp</IncludePath>
        <IlasmFlags>$(IlasmFlags) -INCLUDE=$(IncludePath)</IlasmFlags>
    </PropertyGroup>

</Project>

ILProj\include\netstandard\coreassembly.h:

#define CORE_ASSEMBLY "System.Runtime"

// Metadata version: v4.0.30319
.assembly extern CORE_ASSEMBLY
{
  .publickeytoken = ( B0 3F 5F 7F 11 D5 0A 3A )
  .ver 4:0:0:0
}

ILProj\Class.il

#include "coreassembly.h"

.assembly ILProj
{
  .ver 1:0:0:0
}

.module ILProj.dll

.class public abstract auto ansi sealed beforefieldinit ILProj.Class
  extends [CORE_ASSEMBLY]System.Object
{
  .method public hidebysig static int32 Square(int32 a) cil managed
  {
    .maxstack 2
    ldarg.0
    dup
    mul
    ret
  }
}

If you have difficulties to put it all together, then look at the example here: https://github.com/Konard/ILProj

Cake answered 2/9, 2019 at 21:45 Comment(0)
S
0

I learn most recent System.Runtime.CompilerServices.Unsafe project and have created working example with multitargeting (.NET Core 6, NET Framework 4.8, netstandard2.0, netcoreapp3.1) and two types of signing

You need to create file global.json

{
    "msbuild-sdks": {
        "Microsoft.NET.Sdk.IL": "6.0.0"
    }
}

Then a dark magic begins:

<Project Sdk="Microsoft.NET.Sdk.IL">
    <PropertyGroup>
        <TargetFrameworks>net6.0;netcoreapp3.1;netstandard2.0;net48</TargetFrameworks>
        <DebugOptimization>IMPL</DebugOptimization>
        <DebugOptimization Condition="'$(Configuration)' == 'Release'">OPT</DebugOptimization>
        <CoreCompileDependsOn>$(CoreCompileDependsOn);GenerateVersionFile</CoreCompileDependsOn>
        <IlasmFlags>$(IlasmFlags) -DEBUG=$(DebugOptimization)</IlasmFlags>
        <IsPackable>true</IsPackable>
        <ProduceReferenceAssembly>false</ProduceReferenceAssembly>
        <AssemblyVersion>1.0.0.0</AssemblyVersion>
        <FileVersion>1.0.0.0</FileVersion>
        <Version>1.0.0-beta.1</Version>
    </PropertyGroup>
    <PropertyGroup Condition="$([MSBuild]::GetTargetFrameworkIdentifier('$(TargetFramework)')) == '.NETCoreApp'">
        <ExtraMacros>#define netcoreapp</ExtraMacros>
        <CoreAssembly>System.Runtime</CoreAssembly>
        <_FeaturePublicSign>true</_FeaturePublicSign>
    </PropertyGroup>
    <PropertyGroup Condition="$([MSBuild]::GetTargetFrameworkIdentifier('$(TargetFramework)')) == '.NETStandard'">
        <CoreAssembly>netstandard</CoreAssembly>
        <_FeaturePublicSign>true</_FeaturePublicSign>
    </PropertyGroup>
    <PropertyGroup Condition="$([MSBuild]::GetTargetFrameworkIdentifier('$(TargetFramework)')) == '.NETFramework'">
        <CoreAssembly>mscorlib</CoreAssembly>
        <_FeaturePublicSign>false</_FeaturePublicSign>
    </PropertyGroup>

    <PropertyGroup Condition="'$(TargetFramework)'=='net6.0'">
        <_FeatureUsePopCount>true</_FeatureUsePopCount>
    </PropertyGroup>
    <PropertyGroup Condition="'$(TargetFramework)'!='net6.0'">
        <_FeatureUsePopCount>false</_FeatureUsePopCount>
    </PropertyGroup>

    <PropertyGroup Condition="'$(_FeaturePublicSign)'=='true'">
        <AssemblyOriginatorKeyFile>..\solar_pub.snk</AssemblyOriginatorKeyFile>
        <PublicSign>True</PublicSign>
    </PropertyGroup>

    <PropertyGroup Condition="'$(_FeaturePublicSign)'!='true'">
        <AssemblyOriginatorKeyFile>..\solar.snk</AssemblyOriginatorKeyFile>
        <DelaySign>false</DelaySign>
    </PropertyGroup>

    <ItemGroup Condition="'$(_FeatureUsePopCount)'=='true'">
        <Compile Include="FlagEnumUtil.PopCount.msil"/>     
    </ItemGroup>
    <ItemGroup Condition="'$(_FeatureUsePopCount)'!='true'">
        <Compile Include="FlagEnumUtil.NoPopCount.msil"/>       
    </ItemGroup>
    

    <ItemGroup>
        <!-- mscorlib is passed in as an explicit reference from C# targets but not via the IL SDK. -->
        <Reference Include="$(CoreAssembly)"
                   Condition="'$(TargetFrameworkIdentifier)' != '.NETStandard'" />
    </ItemGroup>

    <Target Name="GenerateVersionFile"
            DependsOnTargets="GetAssemblyVersion;ResolveReferences"
            Inputs="$(MSBuildAllProjects)"
            Outputs="'$(VersionFilePath)">
        <PropertyGroup>
            <IncludePath>$([MSBuild]::NormalizeDirectory('$(IntermediateOutputPath)', 'version'))</IncludePath>
            <IncludePathTrimmed>$(IncludePath.TrimEnd('\'))</IncludePathTrimmed>
            <IlasmFlags>$(IlasmFlags) -INCLUDE="$(IncludePathTrimmed)"</IlasmFlags>
            <VersionFilePath Condition="'$(VersionFilePath)' == ''">$([MSBuild]::NormalizePath('$(IncludePath)', 'version.h'))</VersionFilePath>
            <_AssemblyVersion>$(AssemblyVersion.Replace('.', ':'))</_AssemblyVersion>
            <_coreAssemblyName Condition="'%(ReferencePath.FileName)' == '$(CoreAssembly)'">%(ReferencePath.FusionName)</_coreAssemblyName>
            <_assemblyNamePattern><![CDATA[[^,]+, Version=(?<v1>[0-9]+)\.(?<v2>[0-9]+)\.(?<v3>[0-9]+)\.(?<v4>[0-9]+), .*PublicKeyToken=(?<p1>[0-9a-f]{2})(?<p2>[0-9a-f]{2})(?<p3>[0-9a-f]{2})(?<p4>[0-9a-f]{2})(?<p5>[0-9a-f]{2})(?<p6>[0-9a-f]{2})(?<p7>[0-9a-f]{2})(?<p8>[0-9a-f]{2})]]></_assemblyNamePattern>
            <_coreAssemblyVersion>
                $([System.Text.RegularExpressions.Regex]::Replace(
                $(_coreAssemblyName),
                $(_assemblyNamePattern),
                '${v1}:${v2}:${v3}:${v4}'))
            </_coreAssemblyVersion>
            <_coreAssemblyVersionTrimmed>$(_coreAssemblyVersion.Trim())</_coreAssemblyVersionTrimmed>
            <_coreAssemblyPublicKeyToken>
                $([System.Text.RegularExpressions.Regex]::Replace(
                $(_coreAssemblyName),
                $(_assemblyNamePattern),
                '${p1} ${p2} ${p3} ${p4} ${p5} ${p6} ${p7} ${p8}').ToUpperInvariant())
            </_coreAssemblyPublicKeyToken>
            <_VersionFileContents>
                <![CDATA[
#define CORE_ASSEMBLY "$(CoreAssembly)"
#define ASSEMBLY_VERSION "$(_AssemblyVersion)"
#define CORE_ASSEMBLY_VERSION "$(_coreAssemblyVersionTrimmed)"
#define FILE_VERSION "{string('$(FileVersion)')}"
#define INFORMATIONAL_VERSION "{string('$(InformationalVersion)')}"
$(ExtraMacros)
// Metadata version: v4.0.30319
.assembly extern CORE_ASSEMBLY
{
  .publickeytoken = ($(_coreAssemblyPublicKeyToken) )
  .ver CORE_ASSEMBLY_VERSION
}
 ]]>
            </_VersionFileContents>
        </PropertyGroup>

        <Message Importance="high" Text="Building:$(TargetFramework) $([MSBuild]::GetTargetFrameworkIdentifier('$(TargetFramework)')) CoreAssembly $(CoreAssembly)#$(_coreAssemblyVersionTrimmed) PublicSign=$(_FeaturePublicSign) PopCount=$(_FeatureUsePopCount)"/>

        <WriteLinesToFile
          File="$(VersionFilePath)"
          Lines="$(_VersionFileContents)"
          Overwrite="true"
          WriteOnlyWhenDifferent="true" />

        <ItemGroup>
            <FileWrites Include="$(VersionFilePath)" />
        </ItemGroup>
    </Target>

    <!-- Decompile the ILResourceReference to get native resources. -->
    <Target Name="SetILResourceReference"
            BeforeTargets="DisassembleIlasmResourceFile"
            Condition="'@(ResolvedMatchingContract)' != ''">
        <ItemGroup>
            <ILResourceReference Include="@(ResolvedMatchingContract)" />
        </ItemGroup>
    </Target>
</Project>

Let's explain this project file

  1. DebugOptimization - We use OPT optimization only for release
  2. CoreCompileDependsOn - We need to generate version.h file with main assembly name and version (Generation in Target GenerateVersionFile)
  3. IlasmFlags - for passing DebugOptimization to ilasm tool
  4. IsPackable - not working, but it must create nuget package
  5. ProduceReferenceAssembly - we need to disable ref assembly searching for .NET 6
  6. Then we need to define some compile time constants for different frameworks
  7. Special file version.h looks like a C++ header and can contain some macros, we use it for mask real mscorlib/System.Runtime assembly name. .NET inline code in MSBuild detect platform and fil macros values in generated version.h file (this is made in most recent
  8. I use two extensions for different source including rules (all *.il files are automatically compiled, all *.msil files are manually included)

You can look entire project (with generics,enums and tests) at:

https://github.com/AlexeiScherbakov/Solar.IL

Squaw answered 5/1, 2022 at 3:23 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.