nunit-console "could not load file or assembly" using MySolution.sln

Question

I'm to use nunit-console to run all of the tests in my solution.

I did this:

c:\some\path>nunit-console-x86.exe MySolution.sln
NUnit-Console version 2.6.2.12296
Copyright (C) 2002-2012 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.

Runtime Environment -
   OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1
  CLR Version: 2.0.50727.5466 ( Net 3.5 )

ProcessModel: Default    DomainUsage: Default
Execution Runtime: net-3.5
Could not load file or assembly 'MyNamespace.Administration, Version=0.0.0.1, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

So, I decided to try nunit-x86.exe I did File > Open Project > MySolution.sln and got this:

---------------------------
NUnit
---------------------------
Test load failed!

System.IO.FileNotFoundException : Could not load file or assembly 
'MyNamespace.Administration, Version=0.0.0.1, Culture=neutral, 
PublicKeyToken=null' or one of its dependencies. The system cannot 
find the file specified.

For further information, use the Exception Details menu item.
---------------------------
OK   
---------------------------

The exception can be found here

What is happening and how do I fix it? (without having to maintain a MySolution.nunit file)

More information

• MyNamespace.Administration is not even one of the dlls that contains tests, which means that nunit fails trying to load it to look for tests to run. Knowing this I edited the file created by nunit-x86.exe (MySolution.nunit) and removed all dlls that did not have tests. Sure enough, the tests work (in both gui and console). This is not acceptable for me because it would mean that I have to keep yet another configuration file. Nunit supporting .sln files was supposed to avoid this.
• My tests run fine using TestDriven.Net (but I really need to run them using nunit-console)
• I have looked at this answer but I cannot make sense of what the fusion log viewer says. Would posting that log help? Assembly binding Log Viewer, lists 3 files being created:
  • nunit-agent-x86.exe, this one seems to be trying to find MyNamespace.Administration.dll/EXE inside the nunit directories
  • Tests_24398275 x2 - one looking for nunit.core in my project folders and another looking for unit.core.interfaces inside my project folders. I would pay little attention to these two since they also appear in my manually edited .nunit project).
• (per andreister comment) The problem seems to be with the project/assembly itself and not the creation method. If I create a .nunit project and try to add MyNamespace.Administration to it (using 'Add Assembly...' or 'Add VS project...') it fails.
• Calling nunit-console-x86 somepath/bin/Debug/MyNamespace.Administration.dll directly works.

Jigaboo

Apparently, MyNamespace.Administration.dll contains the code used by the tests. Are you sure you didn't forget eg, to put CopyLocal=True when referencing the project? – Carlstrom

It is referenced by a test project (dll), but that project works when I use MySolution.nunit, which leads me to believe that it really fails trying to load that project just to see if it has tests (I'm assuming it opens projects to see which ones have tests). – Whist

How about adding that project into .nunit explicitly, to see if it would fail? Normally for a non-tests project NUnit just reports it doesn't have tests. – Carlstrom

Sure .NET versions are fine? You don't feed a .NET4 assembly into an older NUnit or something? Maybe try a clean build. And try to remove dependencies from Administration.dll, then start adding them one by one and see if that makes a difference. – Carlstrom

I'll try to remove the dependencies. I have tried passing ` /framework=4.5` and ` /framework=net-4.5` with no success. Also 'NUnit-Console version 2.6.2.12296'. – Whist

let us continue this discussion in chat – Whist

Answer 1

Unfortunately, even after posting on nunit-discuss group I was unable to find a proper solution for this problem.

nunit-discuss group confirmed that my tests are failing because of having a dependency that is one level up.

I did however found an acceptable work-around.

Since calling the .dlls directly didn't have the same issues. I could do this with globs, but I'm on windows... but I have git bash installed.

Taking advantage of my somewhat rigid project structure and naming convention I managed to do this:

"C:\Program Files (x86)\Git\bin\bash.exe" -c 'nunit-console-x86.exe //framework=net-4.5 //xml:nunitresults.xml MysolutionFolder/Tests/*/bin/Debug/*.Tests.dll'

Please note that I took advantage of my naming convention. This is very important to do in order to reduce the number of arguments.

When I did nunit-console-x86 MysolutionFolder/*/*/bin/Debug/*.dll instead of MysolutionFolder/Tests/*/bin/Debug/*.Tests.dll I got an error from nunit-console-x86 saying Bad file number. Besides, it's faster if I just provide the right files.

If you have a more recent version of bash (4.0+, I think) you can instead use the following command (note the use of **):

"C:\Program Files (x86)\Git\bin\bash.exe" -c 'nunit-console-x86.exe //framework=net-4.5 //xml:nunitresults.xml MysolutionFolder/**/bin/Debug/*.Tests.dll'

Which is shorter and more permissive on the project structure.

Jigaboo

Answer 2

Reposting my reply on nunit-discuss:

The NUnit feature of loading VS solutions is really fairly limited and intended to work with simple projects or as a quick way to create an NUnit project file - i.e. load the solution and save as an NUnit project, then edit the xml file that is created. Since the solution file format doesn't indicate which files are tests, NUnit attempts to load each project to check if it contains any tests. (This is the same thing that Visual Studio 2012 and later does when using the test explorer window, btw.)

As you suggest, I think the particular assembly fails to load because of having a dependency that is one level up. When loading either a VS solution file or an NUnit project file, NUnit sets the application base to the directory containing the solution or project. That's why an NUnit project file one level up works.

The designers' intent in this sort of situation is that you would create an NUnit project file. I recognize that this is somewhat inconvenient, since it gives you another configuration file to maintain. I'm open to suggestions regarding the use of globs either on the command line or within the project file. Any such changes would probably go into the next major upgrade, NUnit 3.0.

Hiawatha

Personaly, an extra command argument that would allow to indicate that the tests should be run in each project's root would be nice. For solutions with dozens of projects, the use of the .sln file is VERY conveniente, but there should still be some way to garanty config file and dependency isolation. – Chinfest