How to correctly implement unit tests for .net Standard Library

Question

So as far as I have been understanding from my research. A .net Standard Library can not be used on its own, so it needs to be tested through a different framework, either with .net Framework or .net Core reference. That's just how I interpreted it. Now I am trying to create a Standard Library as I need it to be compatible on most devices as possible, the problem is I don't know how to create the unit tests correctly. Every time I create a MSTest Project I get the following error:

Test run will use DLL(s) built for framework .NETCoreApp,Version=v1.0 and platform X64. Following DLL(s) do not match framework/platform settings. Otchi.Ebml.Tests.dll is built for Framework 2.1 and Platform AnyCPU.

I have experimented a lot with the different architectures and other settings, but nothing seems to work to remove that warning. Am I doing something incorrectly or is there a different, more suitable approach to creating unit tests for a Standard Library

Correggio

As the linked answer says, your tests have to target either Core or Framework at the same .NET Standard level that your targeting your assembly. If you're targeting .NET Standard 2.0, for instance, the tests should target either .NET Core 2.0+ or .NET Framework 4.6.1+ (see learn.microsoft.com/en-us/dotnet/standard/…). Edit the project file for Otchi.Ebml.Tests.dll and see what's in the <TargetFramework> node. – Ostentation

Hmm ok I see, I didnt really understand the answer I linked thats why I opened a new question. I went and looked at the Tests.dll file and the value of <TargetFramework> is: netcoreapp2.1. and the library is netstandard2.0. IMO The Test TargetFramework should be netcoreapp2.0 if I am to follow your link. Is that correct? – Correggio

If your wish is to be compatible on most devices, you have to create several test projects against each of them (.NET Framework 4.6.1+, .NET Core 2.0+, Xamarin, Unity, UWP, Tizen and so on). The MSTest error is simply because you created a .NET Core 1.0 project which does not even support .NET Standard 2.0. – Beeeater

@LexLi I have set up my Test Project to target the .NET Core 2.1 (netcoreapp2.1) that's why I am confused. Nothing I can find is set to .NET Core 1.0. My Library uses the Standard version and the Test is as said before. So Why is it saying I am using .NET Core 1.0?? – Correggio

You might get some idea from MSBuild logging learn.microsoft.com/en-us/visualstudio/msbuild/… – Beeeater

Ok I have done as you said and set the logging level to normal. As said both my projects build the versions they should. But I've found that a the MSTestAdapter Nuget uses the netcoreapp1.0 version. So probably this is causing the problem. – Correggio

Answer 1

A .NET Standard Library does not contain the necessary components for execution. It only defines the execution patterns. When you deploy to a specific machine / processor architecture, the code must know which instruction set to execute upon.

This is where the .NET framework and .NET core come into play. The .NET framework contains execution details for instruction sets supported by the Microsoft Windows operating system. .NET core, similarly, contains the instruction set for additional architectures not natively supported by Windows.

To write Unit Tests, you need to create a NEW project, either in .NET or in .NET core, to execute your code. If you want to cover all your bases, or if you have some binaries that are compiled differently for different architectures (third party libraries come to mind) then you might want to unit test for multiple distributions.

Your library should be a standalone library. Your unit tests should be executed before you update the library to validate that the code being entered into the library passes. That way anyone who uses the library after that can feel confident that it will work.

Autointoxication

So basically the .net Standard contains only the essential libraries that every .net Code Base that can be executed has to implement. That way it is certain that code written in Standard will be able to be implemented in all other .net Projects? – Correggio

I have added my test Project as a seperate Project. But I don't understand why I'm getting that warning. I followed the Msdn Documentation on how to create a unit test for a c# project, so it shouldn't throw that error. – Correggio

Correct! .NET standard is the most basic pattern that all the other frameworks depend upon. Your specific error seems to be for the package 'Otchi.Ebml.Tests.dll'. This DLL seems to already be compiled using .NET framework 2.1. This is an old framework & is is probably not supported in the newest .NET 4.8 - If you REALLY must use this DLL, then your project must be built using .NET 2.1. This is different than .NET Core 2.1, which was released very recently. You will need to install the relevant SDK, this might be it... microsoft.com/en-us/download/details.aspx?id=15354 – Autointoxication

Weird, I only created my library which is the standard 2.0, and the test library that apparantly contains the framework but it should he the core 2.0 or 2.1 version. I never specified any .net framework projects. Do you know how i could change that? – Correggio

In the solution explorer, right click on your .SLN and "Add" a new project. You should get a prompt of dialogs to help you choose which project type to use. If you can't find the project types you are looking for then you must install the SDK's for those project types. If you want to manually change the targeted framework, you can do so in the project properties (Right click the .csproj, properties) or EDIT the .csproj to set "TargetFramework" manually. This method WILL NOT WORK if you do not have the relevant SDK installed or if Visual Studio cannot find it. – Autointoxication

What about issues related to Assembly fails when running from .Net Framework unit test project? – Galenism

Answer 2

Writing unit tests for libraries targeting .NETStandard is tricky since you have to test your code against various implementations of your chosen version of .NETStandard. As already outlined by Zakk Diaz this is due to the fact that .NETStandard merely defines types that are then implemented in actual platforms.

The most common problems are differences in implementation across various platforms like .NETFramework, .NETCore, Mono, etc. However there are also differences in implementations across various versions of platforms that can mess with the way your library works on said platforms (see this example).

This is why running your tests on the lowest version of a platform that implements your chosen version of .NETStandard doesn't quite cut it. You have to take all runtimes into account that your library could be used with.

In general you have two options to solve this issue.

Option 1

Use multi targeting to build your unit test project for all valid runtimes. This also means that you have to have a very big set of target frameworks for your unit test project which will have to be extended whenever new versions of a platform are released.

If you want to cover all relevant scenarios then you'll have dozens of target frameworks and just as many versions of your unit test project being built. Let's just assume that this can get out of hand quickly.

Option 2

Use a unit testing platform that can actually handle .NETStandard by design. Building your unit tests for .NETStandard and having those tests execute on matching runtimes will reduce complexity of your test project.

As of today, most unit testing platforms can't do any of this and you're stuck with Option 1. The one that fits this approach and actually solves your problem is Nuclear.Test.

Please note that Nuclear.Test requires .NETStandard 2.0 and only handles .NETFramework and .NETCore in its current release. This is subject to change however and i'm working on reducing the required version of .NETStandard to 1.0 and include test workers for Mono and UWP as well.

Joubert