How to use MsTest in Continuous Integration without VS?
Asked Answered
W

4

13

My problem is quite simple, I have a CI server which runs msbuild and mstest.

The problem is that the Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll doesn't exist (and I thinks other files related to mstest...) if we don't install VS in the server which is pretty stupid for a CI server...

How can I fix this?

Wessel answered 5/6, 2009 at 9:6 Comment(1)
They have a seperate installer for this purpose now, called "Agents for Visual Studio". There's one for 2010, and for 2012.Wolcott
C
8

Unfortunately, there is no supported or easy way around having to install VS on the build agent machine in 2005 or 2008 (There will be a test agent installer in 2010). UPDATE: See this post from Steve Smith for more info which says pretty much the same thing

It's not just a matter of the assemblies being missing - if you want to run the tests, the runner is not just a separate little EXE and a DLL.

Yes, hard to believe! Needless to say, very few other test frameworks on the planet have this restriction, so unless you have a lot of tests, you could consider moving, for a variety of reasons which are covered in many places, example: The fundamental problems and impracticality of using MSTest...

EDIT: Prompted by Rihan's reply, I binged up the following Running mstest without Visual Studio. - It's not fully supported, but it 'works'...

EDIT 2: Running MSTest without Visual Studio - Gallio to the rescue looks a lot more promising in terms of being supported and non-hacky. NB see @Johannes Rudolph's comment on this post tho'

EDIT 3: Added info re 2010 status on this question

NOTE: I have a similar question for 2008 regarding what's required to support the /publish parameter of MSTest:- Running MSTEST.exe /publish on a TeamBuild server, what are the prerequisites?

Charbonneau answered 5/6, 2009 at 9:17 Comment(6)
I know that mstest sucks but it's integrated to VS, and my client don't want an other things like nunit... I try to test the Rihan's way for the moment... Best regards LoïcWessel
Fair enough. I do understand. Other good approaches are TestDriven.NET, the Gallio GUI runner, The NUnit GUI (you can get it to auto-run on compile which works really well - it means all you need to do is hit Shift-F6 and whatever set of tests you want (based on categories or namespace tree, or multi-selecting via checkboxes) are instantly re-run. The other thing is to consider routing them to NUnit (easier sell to an MSTEST minded shop than xUnit.NET even if I'd generally default to that) and then using Resharper or TestDriven.NET to deliver a superior experience to VS integrated testing.Charbonneau
Yeah I Know about TestDriven.Net but 10 license cost more than 1000$ same thing about resharper but some dev already have their license. For the moment Rihan's way work but I want to know if it's legit or not :/ If not, i will try to move dev in NUnit but it will be difficult, they already don't want to do Unit Test wich is interated with VS UI so NUnit... But i will try. Thanks for your help and if anyone have an answer about legality&licensing it will be cool :)Wessel
I think legality is fine -- there is a link from one of the posts I cited to a comment from Jeff Beehler from MS. The installation is the more difficult bit - there isnt a checkbox on an installer anyway. I personally use RedGreen (a DXCore plugin) to run tests but its not perfect. You've definitely got work to do if integration of the test runner into the IDE is a dealbreaker for getting started on running unit tests, but I've been in your shoes and it's tricky and needs to be whatever convinces people...Charbonneau
Unfortunately, Gallio still relies on MStest to be present on the box. See #3883680.Psychotherapy
Here's what it took me to get it working: jorudolph.wordpress.com/2011/04/02/…Psychotherapy
S
3

@Ruben Bartelink : You can get mstest.exe in your machine by installing test agents. you can find it @given location. Post installation, invoke mstest.exe with /testcontainer and /test options. It runs successfully and creates trx files. Check for something which can process trx and generate reports.

http://www.microsoft.com/en-us/download/details.aspx?id=38186

Symphysis answered 26/5, 2014 at 15:31 Comment(0)
L
2

The specific solution for me and a Visual Studio 2010 was:

  1. Install "Test Agent 2010" from Visual Studio Agents 2010
  2. Restart Windows
  3. Install Visual Studio 2010 Service Pack 1
  4. Restart Windows
  5. Ensure the projects References are updated to point at "Microsoft.VisualStudio.QualityTools.UnitTestFramework" "10.1.0.0" and not "10.0.0.0".
Lollapalooza answered 7/11, 2012 at 13:51 Comment(0)
S
0

We have Cruise Control setup on our integration server. I have not installed visual studio, however I have copyied the MSTEST assemblies into the GAC. We then used Cruise Control Documentation to setup our test reports. Works very well.

Solutrean answered 5/6, 2009 at 9:31 Comment(3)
How did MSTEST.EXE get onto the machine? Is there somewhere that details the steps involved in doing that part (and the qhich QT dlls need to go into the GAC). Obviously if this works, it's great...Charbonneau
It is a shared development enviroment, and another person got the MSTests to run, not sure what they did to get it going. What also helps is that in your source code checkout we have a resources folder that contains handy little tools like the mstest.exe and tinyget.exe that helps in just doing a http get after our build to check the site works. Makes the checkout bigger but that has not being any issue at this point. Always looking for better simpler ways to do automation.Solutrean
I'd venture that some of the stuff as covered in the links I added to in my response is what happened... I'd be recommending using something like Gallio, or switching to NUnit or xUnit.NET - especially if you're the type of shop that has chosen CC.NET vs TeamCity or VSTSCharbonneau

© 2022 - 2024 — McMap. All rights reserved.