How to access TestRunParameters within RunSettings file
Asked Answered
F

9

47

Reading through https://msdn.microsoft.com/en-us/library/jj635153.aspx I have created a .RunSettings files with a few parameters similar to the example:

  <TestRunParameters>
    <Parameter name="webAppUrl" value="http://localhost" />
    <Parameter name="webAppUserName" value="Admin" />
    <Parameter name="webAppPassword" value="Password" />
  </TestRunParameters>

I plan on having a .RunSettings file for each of our environments with appropriate URLs and credentials for running a CodedUI test on the specified RunSettings file's environment.

I can see that from command line to reference the Settings file I can run:

vstest.console myTestDll.dll /Settings:Local.RunSettings /Logger:trx
vstest.console myTestDll.dll /Settings:QA.RunSettings /Logger:trx

etc...

But I don't see any way that calls out how to actually utilize the TestRunParameters from within the codedUI test.

What I would like to do is set up test initializers that use the TestRunParameters to determine where to log in, and what credentials to use. Something like this:

[TestInitialize()]
public void MyTestInitialize()
{

    // I'm unsure how to grab the RunSettings.TestRunParameters below
    string entryUrl = ""; // TestRunParameters.webAppUrl
    string userName = ""; // TestRunParameters.webAppUserName
    string password = ""; // TestRunParameters.webAppPassword

    LoginToPage(entryUrl, userName, password);
}

public void LoginToPage(string entryUrl, string userName, string password)
{
    // Implementation
}

Information on how to reference the TestRunParameters is greatly appreciated!

EDIT

/// <summary>
/// Summary description for CodedUITest1
/// </summary>
[CodedUITest]
public class CodedUITest1
{

    public static string UserName = string.Empty;

    [ClassInitialize]
    public static void TestClassInitialize(TestContext context)
    {
        UserName = context.Properties["webAppUserName"].ToString();
        Console.WriteLine(UserName);
    }

    [TestMethod]
    public void CodedUITestMethod1()
    {
        this.UIMap.RecordedMethod1();
        // To generate code for this test, select "Generate Code for Coded UI Test" from the shortcut menu and select one of the menu items.
    }

    // Rest of the default class - TestContext instantiation, UI map instantiation, etc
}

The exception I'm getting when running:

NullReference Exception

enter image description here

@williamfalconeruk I have updated my test class as above, but I am still getting the same error, any idea what I'm doing wrong?

Fireboat answered 29/7, 2015 at 17:38 Comment(5)
I have the same problem as you in VS2013 (the TestRunParameters not showing up in the TestContext properties). Looking at the MSDN example, msdn.microsoft.com/en-GB/library/jj635153(v=vs.120).aspx, I noticed that the TestRunParameters section isn't specified until you change the version to VS2015, so perhaps it's a new feature there.Xmas
@Xmas ah interesting, i had not noticed that. I wonder if that's the problem I'm having - I'm currently using VS2012. I'll have to give it a shot in 2015Fireboat
@Xmas actually scratch that. I can tell in the screenshot above that that was on 2015 since I can see the code lens information. Still having the issue on VS2015Fireboat
it's the Resharper test runner, who is ignoring your *.runsettingsFalsecard
@Tsar I don't have Resharper :(Fireboat
B
30

For those that use Resharper with this issue, I discovered the fix (no need to disable Resharper):

  1. Go to Visual Studio top menu -> Resharper -> Options

  2. Find the Tools section, expand "Unit Testing"

  3. Click on "MsTest". The checkbox should be on enabled, but the Test Settings file path below that may be blank. If it is, click on browse and select the runsettings file you want to use.

  4. Click save, rebuild and try to run the tests, the parameters should now work.

Not sure why but simply selecting the Test Settings file from the Tests menu -> Test Settings does not actually work when using Resharper, so this file needs to be explicitly pointed to directly in the Resharper options.

Berghoff answered 4/7, 2016 at 9:5 Comment(0)
C
24

I also came across this recently, as we wanted to move away from legacy environment variable usage. The solution provided below was suitable for our needs, but there may be a better one...

It turns out you can access these in the TestContext of a ClassInitialize method of your Test fixture. There is a Properties dictionary which has these parameters, and you could access the values here:

[ClassInitialize]
public static void TestClassinitialize(TestContext context)
{
    var webAppUrl = context.Properties["webAppUrl"].ToString();

   //other settings etc..then use your test settings parameters here...
}

Note: these are static so if you need access to this you may need to set up static properties to access within your Test code.

An alternative suggested is using a Data Driven Test approach. Here's some basic info on data driven tests here which may also help: https://msdn.microsoft.com/en-us/library/ms182527.aspx

As I said, the above solution suited our needs at a basic level.

UPDATE: see image below in response to test settings returning null...

Test Settings in Visual Studio

Colophon answered 5/8, 2015 at 11:7 Comment(11)
Hmm my initializer does not take in a textcontext, and attempting to grab the property from the class textcontext throws a nullreference. Are you doing something specific to get a initializer with that signature? And how are you calling it with a provided context?Fireboat
note the Annotation [Classinitialize] - this is when the test class is set up to run and is in a static context, the [Testinitialize] annotation is used is called each time a test is run as an instance - thats the key difference.Colophon
here's some links to explain how these work...: #23000316 and geekswithblogs.net/abhi/archive/2013/11/18/…Colophon
ok - have you set the test .runsettings file in the Visual Studio menu Test > Test Settings > Select Test Settings File ? this is how i've done it to debug...Colophon
Yeah, unfortunately I had already performed that step when I got the issue. Not sure what else it could be. I upvoted your answer but am not going to accept yet so hopefully the thing I'm missing will be pointed out by someone... eventually :O thanks for the start anyway!Fireboat
@Fireboat Clicking on the "Select test settings file" menu item should pop up a file choosing box. After choosing a file you should see the chosen file named and ticked in the menu.Dionnedionysia
For completeness, there is another places where testsettings files can be set. The context menu for ".testsetting" files in solution explorer has a tickable "Active Web and Load Test Settings" entry. This one only applies to web and load tests. I have no idea why Visual Studio has two very different ways of specifying these files.Dionnedionysia
@Fireboat did you manage to get this worked out in the end - note i am still on VS2013 (but should be the same)Colophon
@Colophon Unfortunately no, never did get this working.Fireboat
I found that it ONLY works when you use the VS Test Explorer - Run Tests, NOT when you Debug Tests or use R# Test Runner. My setup is VS2015.Update 2. with R# Ultimate 10.0.1Kino
Gotcha: if you find your tests are not run after adding [ClassInitialize], be sure that your method signature is correct. It must consist of a single TestContext parameter.Tavey
D
6

This works for me (VS2017-pro):

namespace TestApp.Test
{
    [TestClass]
    public class UnitTest1
    {
        // This enables the runner to set the TestContext. It gets updated for each test.
        public TestContext TestContext { get; set; }
        [TestMethod]
        public void TestMethod1()
        {
            // Arrange
            String expectedName = "TestMethod1";
            String expectedUrl = "http://localhost";

            // Act
            String actualName = TestContext.TestName;
            // The properties are read from the .runsettings file
            String actualUrl = TestContext.Properties["webAppUrl"].ToString();

            // Assert
            Assert.AreEqual(expectedName, actualName);
            Assert.AreEqual(expectedUrl, actualUrl);
        }

        [TestMethod]
        public void TestMethod2()
        {
            // Arrange
            String expectedName = "TestMethod2";

            // Act
            String actualName = TestContext.TestName;

            // Assert
            Assert.AreEqual(expectedName, actualName);
        }
    }
}

Make sure to select the runsettings file you wish to use, here: Test -> Test Settings.

Damien answered 27/12, 2017 at 9:27 Comment(0)
K
3

An alternative to disable Resharper is to enable MSTest support and select the test setting file on Resharper Options dialog (->Tools->Unit Testing->MsTest).

Kurman answered 23/3, 2016 at 13:35 Comment(0)
B
3

I was trying to do this exact thing as well. As many of you may know, running tests through MTM exposes some additional properties to the TestContext, including the name of the Run Settings used. I used this property as a "Foreign Key" of sorts for our test data, allowing us to specify environment URLs etc. without hardcoding them or using the incredibly lackluster "Data Driving" tools that come with out of the box testing.

Of course, there's no way to expose any run-time properties when executing tests as part of a BDT or release workflow besides what @kritner is attempting which microsoft describes HERE. However if you read the comments of that link you'll discover what you may be able to infer here:

  • You need to use VS 2013 R5 or VS 2015 to use this solution
  • It will only work for Unit Tests!

Those of us who are trying to execute UI or Load tests as part of a CI or CD workflow are completely screwed. You don't get any additional properties in testContext, even when executing a Plan/Suite with certain test configurations (not settings) created in MTM. @Adam may have been able to get this to work when running vs debugging, but that may have only worked with unit tests. Through CodedUI I've been unable to retrieve the properties without getting a NullReferenceException. Here's an example of the janky code I was using to investigate:

if (testContextInstance.Properties["__Tfs_TestConfigurationName__"] != null) //Exposed when run through MTM
{
TFSTestConfigurationName = testContextInstance.Properties["__Tfs_TestConfigurationName__"].ToString();
}
else TFSTestConfigurationName = "Local"; //Local
var configName = testContextInstance.Properties["configurationName"] ?? "No Config Found";
Trace.WriteLine("Property: " + configName);

And the XML of my .runsettings file:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <!-- Parameters used by tests at runtime. These are required as a substitute for TFS/MTM test settings.-->
  <!-- File instructions: https://msdn.microsoft.com/en-us/library/jj635153.aspx#example -->
  <!-- TFS instructions: https://blogs.msdn.microsoft.com/visualstudioalm/2015/09/04/supplying-run-time-parameters-to-tests/ -->  
  <TestRunParameters>
    <Parameter name="configurationName" value="Local" />
  </TestRunParameters>
</RunSettings>

And an excerpt from the .trx produced by the BDT workflow:

Property: No Config Found 
Bogosian answered 25/4, 2016 at 19:53 Comment(0)
A
3

With NUnit 3, I was able to find the properties in the runsettings file, using

TestContext.Parameters

So in this case it would be:

string entryUrl = TestContext.Parameters["webAppUrl"];
string username = TestContext.Parameters["webAppUserName"];
string password = TestContext.Parameters["webAppPassword"];
Acicula answered 15/5, 2021 at 23:34 Comment(0)
S
2

For the NullReferenceException issue:

I was also facing the same issue recently and the solution to it is to have the latest update of Visual Studio 2013. Right now the latest update is Update 5. I am not sure which particular update fixes this issue. I applied Update 5 and was able to successfully access the TestRunParameters in the ClassInitialize method.

You can find the updates @ https://support.microsoft.com/en-us/kb/2829760 So I had two machines, on one it was all working fine and on the other I was getting the exception. I investigated that the only difference is the Update of VS; applied it and that solved the problem. :)

Sexagesima answered 27/10, 2015 at 8:4 Comment(2)
Unfortunately I'm on VS2015 and still experiencing the issue.Fireboat
I'm also experiencing this in VS2015. I have set my RunSettings file.Echolocation
R
1

I was able to resolve this for Unit tests by disabling Resharper. Wish I could say the same for Coded UI tests.

Russi answered 16/9, 2015 at 0:5 Comment(1)
Unfortunately I don't have resharper. It's Unfortunate that I don't have it, and unfortunate because I don't have it, it can't be my issue. :(Fireboat
U
-4

Why dont you use Application settings?

AppSettings

you can then read them anywhere like

var yesICan= Properties.Settings.Default.IToldYou;

you can create Properties out from them, pretty much you can do a lot.

public string URL_WEBOFFICE
    {
        get
        {
            return Properties.Settings.Default.WEBOFFICE_URL.Replace("***", Properties.Settings.Default.SiteName);
        }

    }
Unifoliate answered 14/4, 2016 at 11:40 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.