How to run QUnit test and get back test result in C# via JavaScript callback?

Question

In my several projects, I use MVC pattern for separating code (of concerns) into 3 tiers. Both of Model and Control tiers run on C# so I use testing framework like MSTest or NUnit to validate functional requirement for these tiers. For View tiers, I use QUnit to test JavaScript files.

However, I cannot execute QUnit as automated test because MSTest doesn't directly support for testing web page. I need to run it in MSTest like the following logic.

[TestMethod]
public void JavaScriptTest()
{
    var result = QUnit.Test('~/QUnit/test1.htm');

    Assert.IsTrue(result.Failed <= 0)
}

Solution must use callback function in QUnit (not while-loop checking) to ensure that test method will execute immediately after test done.

Zestful

Answer 1

For cross browser testing, Selenium WebDriver is the best option for solving this problem because we can easily switch between browser by changing only one line of code.

1. Install Selenium WebDriver package to project via NuGet.

2.Download preferred driver to your project, add to your project and set "Copy to Output Directory" equals "Copy if newer". For this example, I use Chrome driver to run Selenium with Google Chrome on my machine.

3.In test method, create driver and set max execution time before running QUnit.

var browser = new ChromeDriver();
var navigator = browser.Navigate();

// Currently, max execution time is one minute.
browser.Manage().Timeouts()
       .SetScriptTimeout(new TimeSpan(0, 1, 0));

4.Please make sure that you already set autostart of QUnit to false.

QUnit.config.autostart = false;

5.Navigate to QUnit page. In this case, I use local webpage in solution folder.

 browser.Manage().Timeouts().SetScriptTimeout(new TimeSpan(0, 1, 0));
navigator.GoToUrl(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, @"../../../QUnit example/qunit-demo.htm"));

6.Use ExecuteAsyncScript method in browser object to register callback for QUnit.done function and manually start QUnit testing.

var response = browser.ExecuteAsyncScript
(
    "var callback = arguments[arguments.length - 1];" +
    "QUnit.done(callback); QUnit.start();"
);

7.When QUnit test done, it will return a response. We need to convert it as suitable type and get the test result.

var testResult = response as Dictionary<string, object>;

if(testResult == null) throw new Exception("Unhandle error occur while running QUnit."); 

Console.WriteLine("Test complete in " + (long)testResult["runtime"] + " ms.");
Console.WriteLine("---------------------------");
Console.WriteLine("total: " + (long)testResult["total"]);
Console.WriteLine("passed: " + (long)testResult["passed"]);
Console.WriteLine("failed: " + (long)testResult["failed"]);

8.Don't forget to close browser every time use finish testing.

browser.Close();

PS. I also provide Visual Studio 2012 solution (source code) for this answer.

Click here to download

Update 1

• Fix bug that sometime QUnit start testing before system register callback to done function.
• Include local QUnit page.
• Include IEDriver to download solution. However, this version doesn't support for IE10 on Windows 8. But it work fine on IE8.

Zestful

#1 Limitation, unable to capture console.log output from JavaScript that normally use in complex system. – Zestful

#2 Limitation, maximum execute async script is one minute. – Zestful

Answer 2

PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG. It is full web stack an optimal solution for Headless Website Testing, Screen Capture, Page Automation and Network Monitoring.

I suggest you to use this framework when you want to test some JavaScript library and don't want to use installed browser on test machine.

1.Please make sure that you already set autostart of QUnit to false.

QUnit.config.autostart = false;

2.Download PhantomJS executable file for Windows, add to your project and set "Copy to Output Directory" equals "Copy if newer".

3.Create process to run PhantomJS.exe with 2 arguments that are JavaScript file and tested page url.

var scriptPath = Path.GetFullPath(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "../../PhantomScript/main.js"));
var pageUrl = "file:///" + Path.GetFullPath(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "../../QUnitExample/qunit-demo.htm")).Replace('\\', '/');

var process = new Process
{
    StartInfo =
    {
        UseShellExecute = false,
        RedirectStandardOutput = true,
        CreateNoWindow = true,
        WindowStyle = ProcessWindowStyle.Hidden,
        FileName = "phantomjs.exe",
        Arguments = "\"" + scriptPath + "\" \"" + pageUrl + "\""
    }
};

4.Start process and check the exit code of this process.

process.Start();
process.WaitForExit();

Assert.AreEqual(process.ExitCode, 0);

In JavaScript file, I use eveluateAsync for accessing context of page to run QUnit test, wait until it finish and log amount of test failed.

page.evaluateAsync(function ()
{
    QUnit.done(function(response)
    {
        console.log('!Exit' + response.failed);
    });

    QUnit.start();

    // If QUnit finish after 2500 ms, system will exit application with code -1.
    setTimeout(function()
    {
        console.log('!Exit-1');
    }, 2500);
});

To handle log, I use the following code to exit process with exit code.

var exitCodeName = '!Exit';
page.onConsoleMessage = function (msg)
{
    if (msg.indexOf(exitCodeName) == 0)
    {
        var exitCode = parseInt(msg.substring(exitCodeName.length).trim(), 10);

        phantom.exit(exitCode || 0);
    }
};

PS. I also provide full source code (VS2012) to my SkyDrive. You can download it at the following link.

PhantomJS Test project

This project demo you how to run PhantomJS in MSTest.

PhantomJS Form project

This project is PhantomJS wrapper that is wrote in C# Windows Form. I use this to test "main.js" and "core.js" files before use it in test project.

Zestful