importing javascript xml junit tests to SONAR using jstestdriver fails

Question

What I am doing? I am currently working on setting up Continuous integration / delivery pipeline for front end applications. We are planning to use SonarQube as code quality tool and want to import junit test results/code coverage in SONAR for further analysis.

I was able to successfully import javascript code coverage report to SONAR. We are using Gulp / Karma / PhantomJS / Jasmine etc and using Karma coverage / Junit reporters to generate test results. Junit test file name is TEST-xunit.xml.

Where I am stuck? I need your help/expert guidance on importing Javascript test results to SONAR. I tried different things but somehow not able to import the unit test sources/results to SONAR.

What all I have tried? Earlier we were using SONAR Runner 2.3, SONAR Qube 4.5 / Javascript plugin 2.2 and able to import Junit high level stats like tests passed / failed etc. However, we were not able to navigate to individual tests to see which test passed / failed.

On researching further, noticed that Javascript plugin 2.3 added more messaging for the scenario if it does not find test sources. Also, there were some other dependencies so upgraded SONAR qube to 5.0. Tested with SONAR runner 2.4 and observed following messages. Currently, high level stats are also not getting imported.

JSTestDriverSensor.java code is from SONAR Javascript plugin on github)

LOG.warn("Test result will not be saved for test class \"{}\", because SonarQube associated resource has not been found using file name: \"{}\"",
    getUnitTestClassName(classKey), getUnitTestFileName(classKey));

sonar-runner.properties generated by SONAR gulp plugin (uses SONAR Runner 2.3)

sonar.projectBaseDir=C:/xxxxx/homepage2
sonar.host.url=http://xxxxx:9000/sonarqube/
sonar.jdbc.url=jdbc:mysql:// xxxxx:3306/sonar
sonar.jdbc.username= xxxxx
sonar.jdbc.password= xxxxx
sonar.projectKey=homepage:0.0.0
sonar.projectName=homepage
sonar.projectVersion=0.0.0
sonar.sources=src/app,src/components
sonar.exclusions=src/**/*spec.js
sonar.tests=test
sonar.language=js
sonar.sourceEncoding=UTF-8
sonar.javascript.lcov.reportPath=reports/coverage/lcov.info
sonar.javascript.jstestdriver.reportsPath=reports/unit

I even tried separate SONAR runner 2.4 installation but keep getting same error messages.

Question? 1. There must be something which I am overlooking and somehow not able to determine what might be causing this. I think either I am setting test directory path wrong or JSTestDriverSensor is interpreting test directory file wrong. Suggestions? 2. To start with, I just want to import a sample junit xml file to see if it gets imported successfully. After that, I can modify karma junit generator output to match input junit xml. Need some reference examples here.

Attaching some of the logs. I modified junit xml to set different classnames to see if JsTestDriverSensor finds it. But it’s not!

[23:31:24] 23:31:24.122 DEBUG - Language of file 'src/app/main/textOverride.controller.js' is detected to be 'js'
23:31:24.122 DEBUG - Language of file 'src/app/main/main.routes.js' is detected to be 'js'
23:31:24.123 DEBUG - Language of file 'test/app/main/testSpec.js' is detected to be 'js'

[23:31:24] 23:31:24.138 DEBUG - Language of file 'src/components/navbar/navbar.controller.js' is detected to be 'js'

[23:31:24] 23:31:24.169 INFO  - 7 files indexed

[23:31:26] 23:31:26.632 INFO  - Quality profile for js: Sonar way

[23:31:26] 23:31:26.670 DEBUG - Sensors : QProfileSensor -> JavaScriptSquidSensor -> JsTestDriverSensor -> CoverageSensor -> InitialOpenIssuesSensor -> ProjectLinksSensor -> VersionEventsSensor -> FileHashSensor -> SCM Sensor (wrapped) -> CPD Sensor (wrapped)
23:31:26.671 INFO  - Sensor QProfileSensor...

[23:31:26] 23:31:26.680 INFO  - Sensor QProfileSensor done: 9 ms
23:31:26.680 INFO  - Sensor JavaScriptSquidSensor...

[23:31:26] 23:31:26.950 INFO  - 6 source files to be analyzed

[23:31:27] 23:31:27.081 INFO  - 6/6 source files analyzed

[23:31:27] 23:31:27.113 INFO  - Sensor JavaScriptSquidSensor done: 433 ms

[23:31:27] 23:31:27.114 INFO  - Sensor JsTestDriverSensor...
23:31:27.114 INFO  - Parsing Unit Test run results in Surefire format from folder C:\xxxxx\homepage2\reports\unit

[23:31:27] 23:31:27.176 WARN  - Test result will not be saved for test class "app.main.testSpec", because SonarQube associated resource has not been found using file name: "app/main/testSpec.js"

[23:31:27] 23:31:27.177 WARN  - Test result will not be saved for test class "src.app.main.testSpec", because SonarQube associated resource has not been found using file name: "src/app/main/testSpec.js"

Necessitarianism

Where are the test files located? Are these present where sonar is searching i.e., app/main/testSpec.js? – Howes

yes, its present in app/main/testSpec.js... – Necessitarianism

I'm having a similar problem and discovered that the plugin does not seem to locate test resources properly when they are not in the root of a tests location and the analysis is run on Windows. Are you running on Windows too? – Misanthrope

yes @ioscode, i am on windows.. does it work fine for you on unix? did you find any workaround? – Necessitarianism

Yeah, it works on unix based. No workaround really, it seems that only tests that are in the root of a folder specified as a test folder are detected. Any test files that are in subfolders are not found because of incorrect comparing of forward and backslash characters in the path. I posted about it to the user list, but no response as of yet. – Misanthrope

Indeed there was an issue with the JavaScript plugin which was unable to find test files from report. It should be fixed in version 2.7 which is under vote. Could you maybe try the snapshot version and see if it fix the problem for you ? – Barnard

I solved the same problem with mocha and grunt to report test results in sonar. I think the code and the solution can be adapted to resolve your problem with karma and gulp: solution with mocha and grunt – Laurentium

Answer 1

For anyone reading this and using Grunt (instead of Gulp) there is a plugin for grunt and karma to convert unit test results into the appropriate format to import into SonarQube. Install the plugin and set the Grunt configuration. For example:

        my_target: {
            project: {
                key: 'projectKey',
                name: 'projectName',
                version: package.version
            },
            paths: [{
                cwd: '.',
                src: './src', // source being tested
                test: './tests/unit', // tests for the source
                reports: {
                    // karma output for unit test results
                    unit: './tests/reports/dev/completion/unit.xml',
                    // karma output for unit test coverage
                    coverage: './tests/reports/dev/coverage/lcov/lcov.info'
                }
            }]
        }

By default this will take the files karma created and convert into ./tmp/sonar/results

Then in your sonar properties add something like

sonar.tests=./tests/unit
sonar.sources=./src
sonar.javascript.jstestdriver.reportsPath=.tmp/sonar/results/
sonar.javascript.lcov.reportPath=.tmp/sonar/results/coverage_report.lcov

Note: One thing that bit me at first was that tests named like test.spec.js will get converted to test_spec when Sonar runs, so then when it goes to map back the results to the files, it still looks for test_spec.js instead of test.spec.js. I fixed that just by changing my test names to us the underscore off the bat.

Another Note: When using this approach with Jenkins, I used it just for the conversion of the test results, not for anything else. Because of this, the Grunt task for karmaSonar would fail (since there was no instance of SonarQube configured or found on the box), so if you don't want your whole build to fail make sure to use the --force flag with Grunt. I found that the files were converted just fine even though the task failed. Then I used Invoke Standalone SonarQube Analysis step in my job to run the static code analysis and pick up the test results.

Hope this helps!

Exorbitant