Can I use mstest.exe without installing Visual Studio?

Question

I want to use mstest.exe to run my unit test on build server, but I don't want to install Visual Studio on the build server. Can I just install MSTest without Visual Studio?

Underpass

Can you provide a little more detail. What version? Build server? etc – Analogical

Related post - How do I use MSTest without Visual Studio? – Heat

Answer 1

It is possible to run mstest.exe without visual studio.
Download one of the Agents for Visual Studio ISO's below and install the Test Agent on the server:

• Visual Studio 2022: run winget install --id=Microsoft.VisualStudio.2022.BuildTools -e. Open the visual studio installer and select core testing tools from the individual packages to have the vstest.console.exe (successor of mstest.exe).
• Visual Studio 2019
• Visual Studio 2017 (127MB disk space, less than that for download)
• Visual Studio 2015 and older: visit https://www.visualstudio.com/vs/older-downloads/ and follow the instructions

This installs everything needed for running mstest.exe from the command line and is much lighter weight than visual studio. ~500mb download and around ~300mb to install just the test agent if I remember correctly.

Aggregation

Where MsTest.exe is placed after install TestAgent and TestController? – Amberlyamberoid

C:\Program Files\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe – Lavinialavinie

Our build server is 2013, I use VS 2013. There is only installed VS2010 on build server. Should I install Agents for 2012 or 2010 on build server? – Sperrylite

Is there any agent for Visual Studio 2008? – Mightily

My group ran into an issue after installing the Visual Studio 2012 Test Agents. We think it modified some of the registry values and consequently made a mess of Visual Studio 2010 stuff that was on the same machine. Beware. – Sherikasherill

In order to publish test results to TFS I had to install Team Explorer and manually register my Premium license keys on the build server using Windows Registry as described in one of the other answers. – Echt

i might be missing something, but after installing the agents TC 9 agents screen still shows "Implicit requirements: system.MSTest.11.0 defined in Build step: MSTest". i tried updating PATH variable to include the location of mstest.exe .. where does TC look for this? – Heartbroken

ok so I had to go into build steps, msbuild step, and change the build agent version from the dropdown from 2012 to 2013. I had both versions installed but changing it was what finally made the build agent compatible. So perhaps it was just a way to refresh the settings – Heartbroken

Does installing the VS2013 Test Agent require another Visual Studio license? Does it require any license? – Greenfield

Link for 2015 is dead. Test agents can now be found via visualstudio.com/downloads/?q=agents – Metabolize

For VS 2019, the agent is here aka.ms/vs/16/release/vs_testagent.exe or you can use chocolatey package chocolatey.org/packages/visualstudio2019testagent – Bushy

Answer 2

This answer pertains specifically to Visual Studio 2017, and the answer is yes. Please refer to this answer as to how to locate the MSBuild.exe and/or MSTest.exe executables.

• If you only need to build your unit test project(s), install the package MSTest.TestFramework into those project(s) and remove the reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework from them. Now all you need is to install the Visual Studio 2017 Build Tools and invoke msbuild.exe to perform the build.
• If you need to run your tests as well, things become trickier:
  • The simplest solution is to install VS2017 Community Edition (which includes both msbuild and mstest) - but I am unsure of the legality of this, and am not a lawyer, so be careful!
  • A legally safer solution (and far lighter, in terms of disk space) is to install the Visual Studio 2017 Test Agent and then Build Tools for Visual Studio 2017 (exact order is vital¹); this will give you MSTest.exe and vstest.console.exe which you can then call out to. Do note that actually figuring out where these executables reside is a pain, because they won't exist in the same directory structure as MSBuild.exe in Build Tools.

Finally, and very importantly: if you do use MSTest.TestFramework and still need to be able to discover and run tests from within the Visual Studio IDE, you'll also need MSTest.TestAdapter installed in your unit test project(s).

^{1: While VS2017 supports side-by-side installs, it uses a single registry key that only records the most recent install. Hence, if you install Test Agent last, the key will point to its install directory... but Test Agent doesn't include MSBuild.exe, so any code that relies on this registry key to figure out that executable's path, will fail. Why Microsoft couldn't have made the Test Agent an optional part of Build Tools (so that all the EXEs live in the same directory hierarchy) is anyone's guess.}

Waki

Installed via Test Agent method but mstest.exe still fails with File extension specified '.webtest' is not a valid test extension. for me. – Tennant

There's now vswhere - see other questions, e.g. "Programmatically finding the VS2017 installation directory" or "MSBuild.exe not found, cmd.exe" which use it. – Eyesight

@WaiHaLee vswhere is better than nothing, but you still have to make an assumption as to where MSBuild is located relative to the VS install directory... if (when?) Microsoft decides to relocate any of those executables, that assumption is no longer valid. Why Microsoft can't or won't create a programmatic way to get the path to MSBuild/MSTest/VSTest (as opposed to the VS install dir, which nobody really cares about) remains beyond me. – Waki

Yeah, the is it/isn't it there game is annoying. I'm sure they have their reasons, but.... – Eyesight

@Ian Kemp Per above , I have installed test agent and build tools but I am receiving error No test is available in C:\Users\..\Desktop\MyExeName.exe... . I have opened a question on SO . If you have any clue, I'd appreciate your help. – Euchologion

Answer 3

@crocpulsar, you need to install Visual Studio on your build server, but you do NOT need to buy an additional licence.

There are just way too many dependencies to getting build & MSTest to work without VS installed, and it is most definitely not supported.

As long as the person who starts the build has a license, you do not need one for the build server. This has been the case since the dark days of 2005, and as long as there is edition parity then you are OK.

If everyone in your team has Ultimate, then you are free to install it on the build server; but if one of your team member's has Premium, then you should ideally install Premium on the build server. This also enables lots of other bits like Code Coverage, Test Impact Analysis, and Architecture Validation among others.

Heterochromous

For reference here's a post from microsoft on this blogs.msdn.com/b/jeffbe/archive/2008/03/18/… – Caddy

Answer 4

I think you probably can, but its definitely not supported.

I've found this blog article written by someone who claims to have MSTest working without Visual Studio installed.

• http://blog.foxxtrot.net/2010/02/hacking-mstest-out-of-visual-studio.html

Ryswick

Any URL with the word Hacking really requires questioning the value of effort if MS decide to change its implementation your environment could be volatile and unexpected things could happen with each Windows update (we keep regular updates, right?). – Vietcong

In this case, it seems to be unlikely that a windows update will break the hack. But newer version of mstest (for example, within a newer version or service pack of VS) may work differently. – Margertmargery

@Vietcong - Personally I'd either shell out for an extra VS license, or just use NUnit instead. – Ryswick

Thanks for all the replies, I will just get an extra VS license. – Underpass

As the author of the above Blog Post, I can attest that it does work, and that it is highly unlikely to break. At least for VS2008. I haven't looked at this for VS2010. MSTest being so deeply integrated into VS is a huge weakness of the test kit, but I don't think MS will fix it, since it encourages people to use Team System. – Exclusive

At some point they moved the licenses in the registry to HKEY_LOCAL_MACHINE/SOFTWARE/Wow6432Node/Microsoft/VisualStudio/{version}/Licenses – Echt

Answer 5

I just got this working on my servers without installing the Visual Studio 2017 IDE. My requirement was

• Build projects
• Build test projects
• Run tests using VSTest (I believe it is a similar process for MSTest)

I had to do a combination of a few things stated in other answers and then also another one here.

VS2017:

1. BuildTools - This can be found on the Microsoft downloads page, and then scroll down to "Tools for Visual Studio 2017" -> "Build Tools for Visual Studio 2017"
2. TestAgent - This can be found on the Microsoft downloads page, and then scroll down to "Tools for Visual Studio 2017" -> "Agents for Visual Studio 2017"
3. Nuget Package to include the visual studio unit testing dll - This can be found here

Step 3 was to fix the following issue:

"Could not resolve this reference. Could not locate the assembly "Microsoft.VisualStudio.QualityTools.UnitTestFramework""

Which then caused:

"error CS0234: The type or namespace name 'VisualStudio' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) "

I did not have to add any references to the project. However, the path to the vstest.console.exe is contained in the TestAgent folder (for me it was "C:\Program Files (x86)\Microsoft Visual Studio\2017\TestAgent\Common7\IDE\CommonExtensions\Microsoft\TestWindow")

Duclos

Answer 6

Here are the steps I took to get my build server to run MsTest without installing VS 2012:

1. Created 'Mstest' directory folder in c:\dev.
2. Copied 'Mstest.exe' and 'Mstest.exe.config' from C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE to 'Mstest' directory
3. Copy Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll to 'Mstest' directory
4. Created 'assemblies' directory in 'Mstest' folder
5. Extracted all v11 Microsoft.VisualStudio.QualityTools.*.dlls from C:\Windows\assembly to 'Mstest/assemblies' directory
6. Copy all 'v11' Microsoft.VisualStudio.QualityTools..dlls and Microsoft.VisualStudio.TestTools..dlls from C:\Windows\Microsoft.NET\assembly\GAC_MSIL to 'Mstest/assemblies'
7. Copy all v11 Microsoft.VisualStudio.QualityTools..dlls and Microsoft.VisualStudio.TestTools..dll from C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\PrivateAssemblies to 'Mstest/assemblies'
8. Add 'assemblies' to the 'privatePath' attribute of in 'Mstest.exe.config'
9. Export 'HKEY_LOCAL_MACHINE/SOFTWARE/Wow6432Node/Microsoft/VisualStudio/11.0/Enterprise/QualityTools and apply it to hudson box.
10. Copy QTAgent32.exe and QTAgent32.exe.config to 'MsTest' directory from C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE
11. Add 'assemblies' to the 'privatePath' attribute of in 'QTAgent32.exe.config'
12. Copy 'msdia110.dll' from 'C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Packages\Debugger' to 'MsTest/assemblies'

13. Register 'msdia110.dll' with c:/windows/syswow64/regsvr32.exe /i '../mstest/assemblies/msdia110.dll'(This threw an error, but for some reason it still worked. I ran it a couple times and tried different regsvr32.exe versions before I checked, but it's there in the registry)

14. Add environment variable 'MSTEST_HOME' and set it to 'c:\dev\mstest\' or to your path. I used the environment variable in my build script.

Debugging MsTest execution error:

Add to 'MsTest.exe.config'

<system.diagnostics> 
  <trace autoflush="true" indentsize="4"> 
    <listeners> 
      <add name="EqtListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="C:\MsTest.log" /> 
    </listeners> 
  </trace> 
  <switches> 
    <add name="EqtTraceLevel" value="Verbose" /> 
  </switches> 
</system.diagnostics>

Garniture

Answer 7

MSTest was announced for .NET Core users. The announcement has usage example with the dotnet tool. I haven't figured myself how to get the standalone mstest executable.

Furniture

Link is broken :-( – Walkthrough

@Walkthrough Just fixed the link – Furniture

Answer 8

If you need to run mstest.exe webtest tool then you can install Visual Studio Enterprise trial and make sure to run it at least once (just start it) under account under which test will be running with nothing additional needs to be done. So if your test run under System account then you need to use something like below

PS C:\agent> psexec -s cmd.exe
C:\Windows\system32>"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\mstest.exe"
Microsoft (R) Test Execution Command Line Tool Version 15.0.27520.0
Copyright (c) Microsoft Corporation. All rights reserved.

Please specify tests to run, or specify the /publish switch to publish results.
For switch syntax, type "MSTest /help"

Tennant