Mixing Client-/Server-Side Tests

Question

How should I structure tests containing a mixture of Selenium code (for setting up the page), and client-side Mocha code (for actually performing the tests on the client-side JS being tested)?

I am testing a client-side javascript library. Unit tests work great with Karma, Mocha, and Grunt. The unit-tests can be run locally, or via SauceLabs. The unit tests are able to test basically everything up to the part where a file is actually submitted. This part requires browser automation, and/or manual interaction, and this is what I'm struggling with.

The library in question is a Javascript file-upload library. It has features like chunking, auto-resuming, and much more. It does not require jQuery, and it works on a variety of browsers (IE7-10, FF, Safari (Mac & iOS), Chrome, Android stock browser). So, there are a lot of cases and exceptions to be covered in these tests.

Basically, I need to use Selenium to set up the page. Since I'm testing a client-side JS file-upload library I need to use Selenium to programmatically submit a file so it can actually be uploaded. Once this file has been submitted (the equivalent of selecting a file from the dialog window that appears when you try to upload a file or files online) then client-side tests can be run to ensure the UI has been drawn properly, the file has been found, etc.

I'm trying to wrap my head around how I can test this properly, and how I can set up my test suite to be extensible for the future, robust, and simple/easy to use.

• Should I inject JS code to be run client-side with Selenium?

  One idea is to inject the client-side JS code at opportune times with Selenium. Here's some pseudo-code of what I'm thinking:

describe("A Test", function () {

   it("injects some JS", function (done) {
        // Assume `getClientSideScript` loads some .js file from the filesystem,
        // reads its contents, and returns the contents as a string (to be executed).
        browser.safeExecute(getClientSideScript('initialize'), function (err, result) {
            // At this point, the client-side library is initialized on the page.

            // Assume that `uploadFile` will submit a file to the `<input type='file'>`
            // element
            utils.uploadFile('file.jpg', function (file) {

                browser.safeExecute(getClientSideScript('test_file_submission'), function (err, result) {
                    assert.ok(/* something about the result */);
                });

            });

        }); 
   });
});

This, to me, would work, but seems like a giant PITA. I would have to come up with some convention for storing and loading the client-side scripts that are loaded. Also, two different test would have two different injected scripts so we'd have a huge collection of one-time-use scripts chillin' out in our repo. I'm also afraid of going down this road and finding out that it is a bad decision or impossible.

Anyone have any experience with this?

• Mock server...I'm going to need one.

  I could use Sinon.js to make mock servers client-side, or nock to have a separate server-side mock server. The server is super simple, but I believe I need to able to assert client-side that the response from the server was what was expected.

• Intercepting client-side test results

  Similar to the above, how would I be able to intercept asserts within the client-side code when using Selenium to automate the browser. For example, let's say I wanted to test a request-response cycle between my library and a mock server and I wanted to verify that if my library sends request A to the server it will expect response A. Selenium can't test that. I need to be able to have asserts captured client-side and outputted through Selenium.

Been bangin' my head on the wall trying to figure out the best method to do this. If anyone has any experience with anything similar, please, chime in!

Haswell

I wouldn't recommend having 2 different test frameworks in 1 test case.. that could lead to maintenance hell. What'd be useful is to have 2 seperate test files, and a driver that runs your tests, and runs the mocha in parallel with the selenium. – Pennington

Your goal ("then client-side tests can be run to ensure the UI has been drawn properly, the file has been found, etc.") is achievable with selenium alone (unless there are some UI quirks that Selenium is not able to handle). So why really not use Selenium to achieve this goal directly? – Javelin

Have you looked at what other libraries use? For example, github.com/blueimp/jQuery-File-Upload uses Qunit – Peltz

Answer 1

I wouldn't mix these two types of tests at all. You have several different concerns here. You can still get full test coverage without having to combine tests like you are describing.

• Client side unit tests (using Karma, Mocha, etc.) validate that the JavaScript code is behaving as expected.
• Server side unit tests (using JUnit or whatever unit testing framework exists for your back end) validate that the server code is behaving as expected.
• Integration or end-to-end tests (using Selenium) validate generally validate that multiple components are working together correctly. You can use a mock back-end for these if necessary.

You can create Karma/Mocha unit tests to verify the details of whatever processing may exist for that file upload action. Here, you should be testing edge cases of the individual JavaScript functions underneath the UI.

You can then create a Selenium test to purely test the upload action and expected result of uploading the file. You can verify that the correct elements exist on the page and have the expected properties after the upload. The JavaScript is a black box in this case. You are testing from the user's perspective. This will exercise some of the code you unit tested already, but the point of it is to test the connection between front and back ends.

Intreat