An exception occurred while invoking executor 'executor://xunit/VsTestRunner2/uap': Could not load file or assembly 'System.IO.FileSystem

Question

I'm facing to an issue when I'm running my unit tests on a VSTS build. When I'm launching all my tests on Visual Studio locally, everything works fine. No error at all. Perfect. But when I'm launching my tests via a tests task on a VSTS build, I have this error:

An exception occurred while invoking executor 'executor://xunit/VsTestRunner2/uap': Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=***********' or one of its dependencies. The system cannot find the file specified.

My test project is set like this:

• "xunit.runner.visualstudio" on version 2.4.0 with a "targetFramework" set to "net462" (set inside the packages.config)

• .NET Framework 4.6.2

On the build server I have:

• Visual Studio 2017, v15.7.5

• Visual Studio 2015, v14.0

I'm running out of clues... I'm surely missing something... Thanks for your help

PS:

• All tests passed correctly on the VSTS build
• On the settings of my build task, when I set "Test platform version" to "Visual Studio 2015", it works. But when I'm choosing "Visual Studio 2017" or "Latest" I have the same error...

EDIT

I have Nuget packages "FluentAssertions" v5.4.1 and "System.ValueTuple" v4.5.0. I already tried to downgrade one of them and both. It failed. But I saw that "FluentAssertions" has a dependency to "System.ValueTuple"

EDIT 2

As @PatrickLu-MSFT said, passing the vs test task version to 1 solved the issue. But when I'm looking closely, the Test platform versions available are 2015 and 2013:

When I'm choosing VS test task version 2, I have those choices:

I would like to stay on vs test task version 2 (in case of future updates, I don't want to be stuck...). What I saw is when I'm on Test platform version set to "Latest" or "Visual Studio 2017", it does not work. When I'm choosing "Visual Studio 2015", it working... So what I point is:

• Having a vs test task version set to version 1 make it work (maybe because the latest test platform version is "Visual Studio 2015"...?)
• Having a vs test task version set to version 2 with Test platform version set to "Visual Studio 2015" make it work

So:

• A VS test task version set to version 1 seems to work the same way as a VS test task version set to version 2 with a "Test platform version" set to "Visual Studio 2015".
• How can I make it work with a VS test task version set to version 2 and a "Test platform version" set to "Latest"?

EDIT 3

After connected remotely to the server of my build agent and run tests from VS2017, I compare the output of Visual Studio 2017 of the host server to the output of my local Visual Studio 2017:

• Local Visual Studio 2017:

• Remote Visual Studio 2017:

Are there some issues with "xUnit.net VSTest Adapter v2.4.0 (32-bit Universal Windows)" on the host?

Simmonds

Are you using vs test task and version 2 , you could select the version in the top left corner? If so, have you tried to use version 1, when you say everything works fine. What's the environment on locally are you using VS2015 and VS2017, suggest you make both the environment locally and server side the same, this will narrow down the issue. – Ioyal

I was using vs test task and version 2. After passing it to version 1, everything works fine! Thanks. But why @PatrickLu-MSFT? – Dora

I update my question @Ioyal (see edit 2) – Dora

Seems this works well with VS2015 not VS2017, when you test locally which version of VS are you using? "On the build server I have: Visual Studio 2017, v15.7.5/ Visual Studio 2015, v14.0" Could you remote to your build agent and run the build test directly through VS2017, this will narrow down if the issue related your environment. – Ioyal

When I'm testing localy (on my machine) I have Visual Studio 2017 v15.7.5 and everything works. I connected remotely to the server of my build agent and run tests from VS2017 and I had the same result from the output: "[7/26/2018 11:24:21 AM Error] An exception occurred while invoking executor 'executor://xunit/VsTestRunner2/uap': Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=**********' or one of its dependencies. The system cannot find the file specified." When I'm building the project, there is no error... I have no more trace. – Dora

TFS Visual Studio test task is also calling local Visual Studio to run the build, so your issue seems more related to your environment maybe something installed erroneously of your VS2017. According to the error info , take a look at this thread in Github -- github.com/xunit/xunit/issues/1188 Have you added a reference of your project? – Ioyal

This happened in my TFS build as well. I too had upgraded a few packages through nuget, with one of them also referencing System.ValueTuple. Downgrading the packages again did not fix the problem though. Only after I set "Clean:true, AllBuildDirectores" in the GetSources step did the error disappear. I would like to upgrade the packages though. So it seems the problem is the System.ValueTuple.dll existing in the output directory? – Errolerroll

@Errolerroll I already set "Clean:true, AllBuildDirectores" in the GetSources :). I saw that "FluentAssertions" needs "System.ValueTuple" and install the needed package – Dora

I have 2 projects that are using "System.ValueTuple": the test project and the project that is tested. Bot of them have a reference to "System.ValueTuple" – Dora

Answer 1

I hit this same problem and solved it by changing the "Path to custom test adapters" to point specifically to the correct test adapter by setting it to "Source\packages\xunit.runner.visualstudio.2.4.0\build_common". (You should modify the path to account for where the packages folder is in your repository.)

This apparently happens because by default VSTS tries to be helpful and scans the entire working directory structure for test adapters and loads all of them, but the xUnit Visual Studio test adapter Nuget package installs test adaptors for .Net Core and UWP as well, so VSTS happily loads them and they blow up.

Mose

It's works! I put as "Path to custom test adapters" the correct path to the folder ($(System.DefaultWorkingDirectory)\packages\xunit.runner.visualstudio.2.4.0\build_common). But in the future, with some updates of xunit, I'll depend of the version number... At each update I will have to change this path... For the moment I'll use this solution. Thanks! – Dora

for me it worked with the path "$(System.DefaultWorkingDirectory)\packages\xunit.runner.visualstudio.2.4.0\build\_common" – Unlimited

Worked for me too. The disadvantage is, that when ever there is a NuGet update to the test adapter I have to change my build definition at this point.... :-( – Maclay

hmm.. this did work for me as well - our TFS build is not yelling – Proffer

Answer 2

In my case, using VS2022 and xunit 2.9.0, I was solved this removing obj and bin folders from projects and tried again.

Oglesby