Better way to integrate maven/qunit/phantomjs?

Question

I have been investigating the best way to do JS unit testing in our maven CI environment. What I currently have cobbled together is the following in my maven project:

• qunit resources (JS/CSS files)
• qunit test html files (one for each file under test) with html fixture if required
• index html file which references the test html files as an ordered list of hyperlinks
• PhantomJS runner file, which:
  • opens the index html file and parses out list of test files
  • opens each test file
  • takes a screenshot of the qunit test results for each file
  • If there are any failures, exit with a status of "1"
  • If there are no failures, exit with a status of "0"
• shell file which will exit with "0" if phantomjs isn't installed, will call the phantomjs tests if it is installed

• changes to pom.xml to run phantomjs tests during test phase of build:

  <plugins>
      <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>exec-maven-plugin</artifactId>
          <version>1.1</version>
          <executions>
              <execution>
                  <id>PhantomJS Unit Testing</id>
                  <phase>test</phase>
                  <goals>
                      <goal>exec</goal>
                  </goals>
              </execution>
          </executions>
          <configuration>
              <executable>${project.basedir}/src/main/webapp/unittest/phantomcheck</executable>
              <arguments>
                  <argument>${project.basedir}/src/main/webapp/unittest/qunit-runner.js</argument>
                  <argument>${project.basedir}/src/main/webapp/unittest/tests/index.html</argument>
                  <argument>${project.build.directory}/surefire-reports</argument>
              </arguments>
          </configuration>
      </plugin>
  </plugins>
  

So, this works nicely. It runs the qunit tests during builds on our dev and build machines (as long as PhantomJS is installed). The tests run in a headless browser environment with no restrictions on the qunit tests. Other maven/qunit integrations I've seen fall short due to running the tests in Rhino, or other JS environments which place restrictions on the type of tests we can write. Plus phantomjs gives us the ability to have the screenshots of the test runs, which are helpful in troubleshooting any failures.

The drawback to my approach is that a PhantomJS installation is required on the build/dev machine. I don't know how to bundle phantomJS into a dependency such that developers don't need to worry about installing PhantomJS. Can anyone give me a push in this direction? How do I get started?

Hanover

Check out my Maven plugin (phantomjs-qunit-runner). code.google.com/p/phantomjs-qunit-runner Details on usage here : kennychua.net/blog/… – Nightfall

@KennyChua : the question asker wants a tool for downloading PhantomJS as a dependency and your plugin does not do this. :-( – Electra

Hi. You mentioned you were able to run multiple tests (your statement was: "index html file which references the test html files as an ordered list of hyperlinks"). How did you do this? I've tried several approaches and none of them work. Can you show your code? Perhaps my test runner is different than yours; which one did you use. Thanks in advance. – Ardoin

Answer 1

The phantomjs-maven-plugin provides an install goal for installing phantomjs so you don't need it pre-installed. After it installs phantomjs it sets a property with the path to the executable that other plugins can then use. It also has an exec goal for executing phantomjs scripts. Full disclosure: I wrote the plugin.

Tsar

Thanks for the great plugin, I was able to use it to solve this issue for myself, and write a great answer. :) – Electra

@Tsar great work by creating the plugin....@jonathan-benn I am using your plugin in exact way as described on github ,,,,but could not get it to work .....please check here the problem and help me find what is wrong in my pom.xml https://mcmap.net/q/751314/-better-way-to-add-phantomjs-binary-to-a-maven-project/2079692 – Valvate

@Tsar how is phantomjs-maven-plugin invoked during build process and where does it install the binary in a java-ee project? Can I set it to use a custom location? – Valvate

@AnudeepSamaiya, you can find all that information in the documentation: kylelieber.com/phantomjs-maven-plugin/install-mojo.html – Tsar

@Tsar does not have anything on setting a custom location ....I want to run my tests in tomcat container ....thats why loc is important – Valvate

@AnudeepSamaiya, look at the outputDirectory parameter. Also, you should be asking these things in a new question. – Tsar

@Tsar thanks for finding time...it helped me ....although I did asked the question here few days back https://mcmap.net/q/751314/-better-way-to-add-phantomjs-binary-to-a-maven-project/2079692 – Valvate

Answer 2

Building on Kyle's answer I was able to find a solid solution to this issue. Thank you Kyle!

The solution is to use the phantomjs-maven-plugin Maven plugin. I add the plugin to my pom.xml like so (you will need to upgrade Maven to v3.1 or higher to use the plugin):

<plugin>
    <groupId>com.github.klieber</groupId>
    <artifactId>phantomjs-maven-plugin</artifactId>
    <version>0.4</version>
    <executions>
        <execution>
            <goals>
                <goal>install</goal>
                <goal>exec</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <version>1.9.7</version>
        <checkSystemPath>false</checkSystemPath>
        <script>src/test/qunit/run-qunit-testsuite.js</script>
        <arguments>
            <argument>src/test/qunit/testsuite.qunit.html</argument>
        </arguments>
    </configuration>
</plugin>

Important Caveat: in the pom.xml code above, make sure to use relative (not absolute) references to the files, as I've done. I wasted a few hours after using absolute references (starting at ${basedir}) only to find out that it does something strange to PhantomJS's working directory. Using relative references in your pom.xml will enable relative references inside your HTML file (which will maximize code portability).

In the plugin code above, I reference two files: run-qunit-testsuite.js and testsuite.qunit.html. The HTML file is just the QUnit file that executes all of your tests. The JS file is the driver for PhantomJS; it accepts one argument: the HTML QUnit test file to load.

To complete this solution, you can download sample driver and test files from GMarik's GitHub Gist page. You can and should adapt these files to your needs (although be aware that GMarik's page does not include an open source license, you will need to ask for permission for any copyright-infringing use).

When adding this plugin to your Maven code, after executing a Maven build you will see output like the following (adapted from GMarik's page):

[INFO] --- phantomjs-maven-plugin:0.4:exec (default) @ project.name ---
[INFO] Executing phantomjs command
'waitFor()' finished in 200ms.
Tests completed in 21 milliseconds.
5 tests of 5 passed, 0 failed.

If the tests pass then your build will pass. If the tests fail then your build will fail!

Electra

Hi @Jonathan. I have followed all the steps proposed by you but still its not working for me. For some reason none of the tests in my HTML file are executed – Overeat

@ideate Have you tried executing the tests in a regular browser? Do the tests get executed in the browser? Are you using QUnit? I only tested this with QUnit. – Electra

Yes the tests execute fine in a browser and am using Quint. Problem is I used the sample driver from GMarik s github page and I think that is not working for me. – Overeat

@ideate Oh, ok. I remember that GMarik's driver worked for me out of the box, but you may need to fiddle with it. Also, PhantomJS is not perfect and depending on what you're doing in your tests it may not be able to handle them. E.g. I found that it could not run certain versions of OpenUI5. If I were you, I'd start with a simple hello-world QUnit test and see if you can run that. Then add in more complex QUnit tests one at a time and see which one breaks PhantomJS. You may be able to isolate the problem this way – Electra

Answer 3

Using Kyle's answer and another plugin I was able to get a full solution that doesn't require anything but maven preinstalled and sets up phantomjs and qunit to allow the running of tests. I started with a maven-grunt plugin (github.com/eirslett/frontend-maven-plugin) and followed the steps in this guide (http://blog.trifork.com/2014/10/07/setting-up-maven-to-use-gruntnodejs/) to get it set up. Then I tried to use qunit within maven and I ran into phantomjs trouble and came across this post and found out about Kyle's plugin (github.com/klieber/phantomjs-maven-plugin). I had to use a custom qunit source explained in this guide (http://techblog.dorogin.com/2013/08/issues-with-grunt-contrib-qunit.html). This allowed me to use kyles plugin to install phantomjs then link the binary through grunt options to the custom qunit. In the end my pom looked like:

`    <plugin>
        <groupId>com.github.klieber</groupId>
        <artifactId>phantomjs-maven-plugin</artifactId>
        <version>0.4</version>
        <executions>
          <execution>
            <phase>generate-resources</phase>
            <goals>
              <goal>install</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <version>1.9.8</version>
        </configuration>
      </plugin>
      <plugin>
        <groupId>com.github.eirslett</groupId>
        <artifactId>frontend-maven-plugin</artifactId>
        <version>0.0.20</version>
        <executions>
          <execution>
            <id>install node and npm</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>install-node-and-npm</goal>
            </goals>
            <configuration>
              <nodeVersion>v0.10.33</nodeVersion>
              <npmVersion>1.3.6</npmVersion>
            </configuration>
          </execution>
          <execution>
            <id>npm install</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>npm</goal>
            </goals>
            <configuration>
              <arguments>install</arguments>
            </configuration>
          </execution>
          <execution>
            <id>grunt build</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>grunt</goal>
            </goals>
            <configuration>
              <arguments>--phantomPath=${phantomjs.binary}</arguments>
            </configuration>
          </execution>
        </executions>
      </plugin>
`  

My Gruntfile.js looked like:

`    module.exports = function(grunt) {
      grunt.loadNpmTasks('grunt-croc-qunit');
      grunt.initConfig({
      pkg: grunt.file.readJSON('package.json'),
      qunit: {
        options: {
          'phantomPath': grunt.option('phantomPath')
        },
        all:['src/test/*.html']
      }
  });
  grunt.registerTask('default',['qunit']);
};`  

And my package.json looked like:

`    {
  "name":"reporting",
  "version":"0.0.1",
  "dependencies": {
    "grunt": "~0.4.5",
    "grunt-cli": "~0.1.13",
    "grunt-croc-qunit":"~0.3.0"
  },
  "devDependencies":{ }
}`  

Reynold

Also sorry about the formatting. This was my first post – Reynold

Answer 4

We just check phantomJS.exe into source control. And then we are certain that the same version of phantomJS is being used on all machines.

Kaila

Normally, you want to avoid checking large binary files into source control. That's one of the problems Maven is trying to solve in the first place, by providing an easy framework for downloading binary dependencies. – Electra

A general rule for CI is to keep everything you need to build and test your software under source control. So on a fresh machine you can just get the version of the code you require, and build it without having to manually install any tools. We use the NuGet package manager to help with some framework dependencies, but not all tools are available. By having phantonJS.exe in source control we can be sure that every build is using the same version, and as an extra bonus we do not have a dependency on an extra repository - only our own source control system needs to be available. – Kaila

Answer 5

This is an old question, but I thought I would link to a project of mine that uses PhantomJS and QUnit to run with TestNG:

The project is called qunit-testng. I also have a sample project that shows the library in use.

Here's a screenshot of test output:

Imitable