VS 2013 MSTest vs nUnit vs xUnit

Question

I realize there have been a LOT of questions on this topic but somehow I haven't found one that addressed my needs.

My team is looking to start automated Unit testing our application. We have never done it before and nobody on the team has much experience with it. I have been asked to research and find a framework for automating our Unit Testing. So far, I have narrowed the choices down to MSTest, NUnit and xUnit.

All across the internet, I read negative reviews about MSTest but it seems the most convenient tool to me for our purpose.

1) Our application is .NET

2) We use licensed VS IDE for our development across the team.

3) Source Control is Team Foundation Server 2010

4) We plan to integrate CI/CD as part of our software delivery process.

I feel that MSTest integrates right into this setup and provides the cleanest interface for us to work on. Is there a significant advantage with NUnit or xUnit over MSTest that we should consider?

Flange

Written in 2010, still valid today: blog.ploeh.dk/2010/04/26/WhyImmigratingfromMSTesttoxUnit.net – Educational

Possible duplicate of Visual Studio 2013 MSTest vs NUnit – Dogmatist

Answer 1

I am biased because I work on NUnit, but the advantage of NUnit or xUnit is that both frameworks offer more functionality like data driven tests, parallel execution and a host of advanced features.

That said, there is nothing wrong with MSTest especially if your team is not very experienced with unit testing. It is well integrated with Visual Studio and with TFS, so adding tests is fairly painless. Both NUnit and xUnit integrate well too, but they may require more setup.

Go ahead and start with MSTest. It is an easy introduction to unit testing. If you start running into some of the limitations of MSTest, then it is a fairly easy migration to NUnit or xUnit. For simple cases, it is just a matter of changing your NuGet packages and then find/replace the attributes.

When it comes to choosing between xUnit and NUnit, both are great choices but each has a different philosophy. NUnit tends to be more flexible and allows for a range of testing styles and can be used for integration tests. It is however flexible enough to allow you to shoot yourself in the foot and allows you to write tests that purists would frown upon. NUnit leaves the choice up to you.

xUnit however tends to be a bit more opinionated and pushes you towards the 'pit of success' with unit testing. In my experience, it is great for green field projects, but it can make testing some poorly architected brown-field projects difficult without refactoring.

In the end, you can't go wrong with any of these test frameworks.

Roman

I was in a similar position around 2000, trying to choose among about six .NET test frameworks. I selected NUnit and here I am. :-) – Alcina

An interesting post for xUnit github.com/xunit/xunit/issues/350. Assertion error message has gone and won't coming back in xUnit. When I see them closing the issue while it is still controversial like that, I am not sure I should have faith for their future development. How could we know they won't remove anything else for the sake of "pureness" again? – Spinelli

Isn't one of the points of writing tests to refactor poorly architected code? – Utopian

@Utopian Yes, but if you can't write the tests without refactoring first, you're kind of stuck ;) – Publication

makes sense @Stijn – Utopian

Answer 2

I would like to add one negative aspect of XUnit. It does not have even a basic documentation, nothing beyond "Get Started". No API, nothing.

Plateau

Perhaps that's changed? I found it more than adequate. xunit.github.io/#documentation – Reliable

Answer 3

Here is a brief Analysis of the 3 Frameworks and ranked by color.

Legend of the Coloring:

MSTest it is good only because it is simple and integrated into Visual Studio and it comes from Microsoft. Its not flexible enough and not very extensible. But more than just a unit test framework, it also supports Integration tests, Web (load) performance tests. The other 2 are pure Unit Test Frameworks. So I would say MSTest is "Jack of all Trades, Master of none."

NUnit is my choice because its been the best consistently long enough. It is more flexible and simple with very good user base, documentation and community support.

XUnit is least of my choice because it follows a slightly different style and personally I do not see much advantage over NUnit. In some cases you will need to do some refactoring your code to fit into the test cases. But this is just my opinion as there are many who prefers XUnit.

Elfriedeelfstan

Answer 4

I have listed down the comparisons in two categories:

Framework:

Implementation:

The source of these comparisons

Perri