.Net Standard 4.7.1 Could not load System.Private.CoreLib during serialization

Question

I'm working on migrating most of my project to .Net Standard 2.0.

Last part of the project in .net standard 4.7.1 is a WCF front end. This executable communicate with a .net core application throught a library Akka.net by ClientServiceReceptionist.

Network communication between application by akka.net work. Except when an it try to serialize a ReadOnlyCollection.

In this case the system try to load a "System.Private.CoreLib.dll" that is not available.

I readed many issues with .net 4.6 using .net standard 2.0 libraries that must have been corrected in 4.7.1. I pass from package.config to PackageReference.

I tryed to used NewtonSoft as a serializer in place of Hyperion without progress.

Does anyone have an idea, a solution ?

Edit : 07-05-2018

The issue is throw in the WCF Entry Points when i sent a ClusterClient.Send object throught the ClusterClientReceptionist.

The object sent contains only boolean, string, Guid and array of string properties.

Edit 08-05-2018

The object sent look like this :

{
  (string) "Path": "/user/ProcessOrderWorkflow",
  "Message": {
    "Order": {
      "Data": {
        (string)"Sentence": "i love my name",
        "Options": {
            (boolean)"Simplify" : true,
            (IReadOnlyCollection<string>) LanguageAffinity : [ "FR", "EN" ]
        }
      },
      "OrderQuery": {
        "Verb": {
          (string)"Verb": "Extract"
        },
        "Arguments": [
          {
            (string)"Argument": "Sentence"
          },
          {
            (string)"Argument": "Meaning"
          }
        ]
      },
      (Guid)"ProcessId": "0bfef4a5-c8a4-4816-81d1-6f7bf1477f65"
    },
    (string)"ReturnTypeFullName": "Viki.Api.Server.Data.SentenceMeaningMsg,, Viki.Api.Server.Data, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
  },
  (boolean)"LocalAffinity": false
}

Each of the object used in the hierachi is builded using the constructor. All the properties are in readonly.

I tryed to serialize and deserialize the result on the WCF part before it's sent and it works.

var serializer = this._system.Serialization.FindSerializerFor(result);
var bytes = serializer.ToBinary(result);
var data = serializer.FromBinary(bytes, result.GetType());

The strange think is that it try to deserialize the object in the WCF part where it is sent and not in the LightHouse where the element should be received and transfert to a agent for processing.

Dulcle

It is a .netcore assembly, you should not have a dependency on .netcore when you target .netstandard and 4.7.1. The stack trace is hard to read, but smells like an Akka issue. – Hohenzollern

The app .net 4.7.1 only refert .net standart 2.0 libraries – Dulcle

Does this issue occur upon deserialization? If so, the problem might be that the namespace the client is using to transmit the type is different from the namespace the server is using to deserialize it. This is a common issue that can occur when doing a blend of .NET Core / .NET 4.* runtimes. Can you check to see if the client / server runtimes are different? – Ygerne

@Ygerne i edit my post to complet some informations. Hope it will give you enought informations – Dulcle

Do you have a simple repro for this? The serialized content would also help to see why the type is trying to be resolved from System.Private.CoreLib – Joggle

Updated: All the projects are in the same solutions if it's what your ask by "repro". Is theire a hook or a specific test that i could do to give you more informations ? – Dulcle

Answer 1

So this is what I thought the issue might be... The problem is that transmitting serialized content between a .NET Core application and a .NET Framework application using a polymorphic serializer like JSON.NET or Hyperion is an unsupported operation in Akka.NET at least, but also in any other runtime where the users uses CLR types as the wire format of the messages.

A brief explanation as to why we don't support this in Akka.NET, from the v1.3.0 release notes:

As a side note: Akka.NET on .NET 4.5 is not wire compatible with Akka.NET on .NET Core; this is due to fundamental changes made to the base types in the CLR on .NET Core. It's a common problem facing many different serializers and networking libraries in .NET at the moment. You can use a X-plat serializer we've developed here: #2947 - please comment on that thread if you're considering building hybrid .NET and .NET Core clusters.

The "fundamental changes" in this case being that the namespaces for types like string are different on .NET 4.* and .NET Core, thus a polymorphic serializer that doesn't account for those differences will throw this type of exception.

We do have a workaround available here if you'd like to use it:

https://github.com/akkadotnet/akka.net/pull/2947

You'll need to register that serializer compat layer with JSON.NET inside Lighthouse and your .NET 4.7.1 apps.

Ygerne

I copy the compat serializer and it work for me. I have to Add a specific Identifier and replace the Preprocess condition "#if CORECLR" by a runtime check if i'm runing in .net core or .net classic. – Dulcle

Thanks for that @MickaelThumerel - saw your comment on the pull request. Very helpful! – Ygerne

Answer 2

This is a top result for request "Could not load System.Private.CoreLib" so I post the workaround for ASP.NET Core APIs and .NET clients.

If json serializer type handling set to auto

settings.TypeNameHandling = TypeNameHandling.Auto;

serializer will include type information for polymorphic types. After migration to .NET Core some clients reported exception and the response body contained following type descriptor:

"$type":"System.String[], System.Private.CoreLib"

In the API model property type was defined as IEnumerable<string> which forced serializer to include actual type for an Array. The solution was to replace IEnumerable<string> with string[] or any other concrete type which allowed serializer to omit the type descriptor.

If the above does not work, for instance when you use Dictionary<string, object> you can implement custom SerializationBinder:

public class NetCoreSerializationBinder : DefaultSerializationBinder
{
    private static readonly Regex regex = new Regex(
        @"System\.Private\.CoreLib(, Version=[\d\.]+)?(, Culture=[\w-]+)(, PublicKeyToken=[\w\d]+)?");

    private static readonly ConcurrentDictionary<Type, (string assembly, string type)> cache =
        new ConcurrentDictionary<Type, (string, string)>();

    public override void BindToName(Type serializedType, out string assemblyName, out string typeName)
    {
        base.BindToName(serializedType, out assemblyName, out typeName);

        if (cache.TryGetValue(serializedType, out var name))
        {
            assemblyName = name.assembly;
            typeName = name.type;
        }
        else
        {
        if (assemblyName.AsSpan().Contains("System.Private.CoreLib".AsSpan(), StringComparison.OrdinalIgnoreCase))
            assemblyName = regex.Replace(assemblyName, "mscorlib");

        if (typeName.AsSpan().Contains("System.Private.CoreLib".AsSpan(), StringComparison.OrdinalIgnoreCase))
            typeName = regex.Replace(typeName, "mscorlib");    
            cache.TryAdd(serializedType, (assemblyName, typeName));
        }
    }
}

And register it in JsonSerializerSettings:

settings.SerializationBinder = new NetCoreSerializationBinder();

Note: .AsSpan() for string comparison was added for backwards compatibility with .NET Framework. It doesn't harm, but not required for .NET Core 3.1+, feel free to drop it.

Ro

This issue is open on the newtonsoft repo and he suggests what @Andrii wrote. github.com/JamesNK/Newtonsoft.Json/issues/1378 – Hamate

Out of curiosity, is there particular value in using AsSpan if the content isn't going to be substringed/modified with any frequency? Does the Span.Contains check perform better than String.Contains for some reason? It doesn't seem like it's reducing allocations here which I believe was its originally intended purpose, but admittedly I haven't used it much so I thought I'd ask. – August

@TheXenocide, probably your intention was to comment on some other issue, but anyway, if you check the source code of both Span<char>.Contains and String.Contains you'll find that the implementation is pretty much identical, both use same helper classes and methods, so performance will be same. – Ro

That's essentially what I was wondering about. If they have the same performance I don't see any purpose in converting the strings to spans (lightweight structs, though they may be), just wanted to make sure there wasn't some particular reason to do so. If this wasn't using Regex to modify the string, I could see how spans could possibly be used to modify the returned string more efficiently. Anyway, not meant as a criticism as much as just checking to understand. – August

@August I see what you mean now, sorry I didn't catch it from start. I didn't remember that I used .AsSpan() in this answer :). When I was writing my first reply I checked the source of netcoreapp3.1 and both approaches end up in same CompareInfo.Invariant.IndexOf. So there is no reason in doing .AsSpan() anymore. I will update the answer. I don't remember if the implementation was different back then. – Ro

you must override BindToType method,and replace assemblyName .typeName from System.Private.CoreLib to mscorlib – Dispatcher

@我零0七 maybe it depends on the use-case. The names we receive from our client apps bind correctly to .NET Core types. – Ro

@AndriiLitvinov The AsSpan() actually is required to compile this .Net Framework. Since the String.contains implementation with the two parameters was only introduced in .NET 5. – Pulsar

@Smolakian, good point. That must've been the reason for this my original answer. I am not sure whether it's worthwhile reverting it back. Probably not, since .NET 6 LTS is out and .AsSpan can be more confusing going forward. – Ro

@AndriiLitvinov IMHO, adding the .AsSpan version inside an #if NETFRAMEWORK preprocessor block would be the most helpful for future readers going forward. Definitely lots of projects out there can't update to .NET Core/6+ for various reasons. (Tried to suggest edit but queue was full) – Pulsar

@Pulsar no problem. I've reverted to a version with .AsSpan() and added a note that it's not required and can be dropped for newer versions of .NET. – Ro

Answer 3

If this issue happens during deserialization in .NET Framework application and you can't do anything about the JSON you receive, then you can extend DefaultSerializationBinder and use it in JsonSerializerSettings to deserialize JSON generated by .NET Core application:

internal sealed class DotNetCompatibleSerializationBinder : DefaultSerializationBinder
{
    private const string CoreLibAssembly = "System.Private.CoreLib";
    private const string MscorlibAssembly = "mscorlib";

    public override Type BindToType(string assemblyName, string typeName)
    {
        if (assemblyName == CoreLibAssembly)
        {
            assemblyName = MscorlibAssembly;
            typeName = typeName.Replace(CoreLibAssembly, MscorlibAssembly);
        }

        return base.BindToType(assemblyName, typeName);
    }
}

and then:

var settings = new JsonSerializerSettings()
{
    SerializationBinder = new DotNetCompatibleSerializationBinder()
};

Sandon

Answer 4

for the base Type try to use

serializer.TypeNameHandling = TypeNameHandling.Auto

can ignore the System.Private.CoreLib namespace in Json

Coverture

TypeNameHandling.Auto can solve your problem – Coverture