NUnit vs Visual Studio 2010's MSTest?

Question

I realise that there are many older questions addressing the general question of NUnit v MSTest for versions of Visual Studio up to 2008 (such as this one).

Microsoft have a history of getting things right in their 3rd version. For MSTest, that is VS2010.

Have they done so with MSTest? Would you use it in a new project in preference to NUnit?

My specific concerns:

• speed
• running tests within CruiseControl.NET (either commandline or MSBuild task)
• code coverage reports from CC.NET
• can you run MSTest tests in debug mode

(We use ReSharper, so test-runners are not an issue for us. We have used NUnit for the last few years. We do not have TFS.)

Pied

See what-are-the-preferred-options-today-for-unit-testing-in-vs-2010 – Man

Answer 1

• List item speed is same, but MsTest may be a bit slower because it creates folder for test run every time
• MSBuid and CC.Net is big pain. You can't run MSTest on computer without VS on it (not 100 sure about 2010, but with 2008 it is so)
• not sure, sorry
• yes you can, from visual studio

My recommendation is following: if NUnit satisfies you - use it, forget about MSTest

Latin

Concur with last statement. – Trici

@Preet. I agree. Also MSTest changes are tied to releases of Visual Studio – Horoscopy

+1 for the last statement. I was on a project and we started unit testing with MSTest (2008). We got so annoyed with it, we switched over to NUnit. I'm using VS 2010, and there's not much improvement over VS 2008. – Janellejanene

Answer 2

To correct some old information on the thread;

1. It IS possible to run 64 bit tests in 2010
2. From VS2008 forward it is not neccesary to have MSTEST create directories anc opy the binaries in, just disable deployment, in 2010 thats the default but you have to set it in 2008
3. 2010 MSTEST is faster but as its a generalised test framework that also runs load/web/UI tests there are compromises in the design that will lead to it being slower. Jamie Cansdale appears to have managed to get perf increases with the lastest releases of TestDriven.net's support for MSTEST

Nett

Answer 3

I've mainly used NUnit, some xUnit and some MSTest. They seem functionality equivalent, but I don't like the MSTest test runner. It runs in visual studio so it either crowds the screen or is on another monitor getting in the way everytime I tab to visual studio. (I run NUnit on another monitor, but it doesn't cover everything on that monitor everytime I focus visual studio). It takes too many clicks to find out what test failed and why.

NUnit can run in the background until a test fails, at which point it shows you information about the breaking test. This seems like the ideal for keeping red/green/refactor going smoothly.

Barmecide

Answer 4

Nope. Same issues regarding appdomains and assembly resolving still exist. I would avoid unless you want the new goodness for other functional testing or integration with Team System.

Trici

Answer 5

I don't know much about CruseControl.net, but you can debug tests. We currently don't use TFS either, and the MSTest is working for us.

Enthusiasm

Answer 6

If you think you'll ever run your tests in 64 bit mode, use NUnit. MsTest is only x86.

Tantivy

A workaround is described here, rupertrawnsley.blogspot.com/2011/04/mstest-and-64bit.html , but it seems possibly a bit kludgy? – Vinasse

Answer 7

One major difference between the two is that MSTest makes a copy of the current DLLs every time it runs a test. If you're doing TDD and running your tests frequently, this can eat up a lot of hard drive space.

If you're using MSTest, you can change this setting in Tools > Options > Test Tools > Test Execution. "Limit number of old Test Results to" is set to 25 by default in Visual Studio 2010. I usually change it to 1.

Unsettle

Answer 8

MSUnit runs your test cases under conditions that are different from the actual execution environment. Specifically, the deployed files differ from those that are deployed when you run your actual project. Nethertheless, there is the [DeploymentItem]-Attribute to specify which files shall be deployed by MSUnit. So if your application depends on any external files, such as

• database files
• database configuration file
• application configuration file
• ...

then MSUnit is not the right choice, because the MSUnit tests never cover what your file system is going to look like in execution environment. The Visual Studio Project File settings for deploying files (Copy always, Content, etc.) are ignored by the MSUnit runner. So those settings cannot be tested.

Crudden