Using gtest in jenkins

Question

I successfully run my unit test with google test in Jenkins, but I don't know how to show the .xml file generated by gtest. It is said that gtest satisfies the JUnit format, so what I set is as follows:

But it ends up with errors after a building.

No test report files were found. Configuration error?
Build step 'Publish JUnit test result report' changed build result to FAILURE
Finished: FAILURE

Aceous

Is result.xml located in the workspace root? If you browse your workspace you should be able to find the result.xml and which exact path it's located at. – Suwannee

@sti:I added the exact errors, and where is the workspace root of Jenkins? – Aceous

I know it might be late but maybe somebody comes across this question again, Fortunately gtest is now compatible with JUnit in Jenkins (I read there are still some issues but for me it was all good) I used the latest gtest from their master branch, the generated xml was compatible with JUnit and that was simply picked up and handled by Jenkins. – Idaline

Answer 1

Fraser's answer is good and you need some extra processing to convert the gtest XML to proper JTest format.

First you ask gtest to output the result to XML using:

mygtestapp --gtest_output=xml:gtestresults.xml

Then in a script you need to add extra elements to properly flag skipped tests as such. Jenkin's JTest processor requires that a skipped test contains the <skipped> element and not just setting status to "notrun":

awk '{ if ($1 == "<testcase" && match($0, "notrun")) print substr($0,0,length($0)-2) "><skipped/></testcase>"; else print $0;}' gtestresults.xml > gtestresults-skipped.xml
mv gtestresults.xml gtestresults.off

If running this on a windows batch file, put the awk action inside a file to avoid problems with the quotes. awk.progfile:

{ if ($1 == "<testcase" && match($0, "notrun")) print substr($0,0,length($0)-2) "><skipped/></testcase>"; else print $0;}

And create add in your bat file:

awk -f awk.progfile gtestresults.xml > gtestresults-skipped.xml

Lastly you point the JTest processor as a Post-Build step to read the converted XML:

# Publish JUnit Test Result Report
Test Report XMLs: gtestresults-skipped.xml

Protohistory

Now there is also possible to parse googletest output with xUnit plugin, "googletest 1.6" option. It seems that it can deal with disabled tests without awk magic, and works properly with googletest 1.7 too. – Swaraj

Answer 2

Are you running your test executable with the correct flags (i.e. --gtest_output=xml[:DIRECTORY_PATH\|:FILE_PATH])?

From the --help output:

--gtest_output=xml[:DIRECTORY_PATH\|:FILE_PATH] Generate an XML report in the given directory or with the given file name. FILE_PATH defaults to test_details.xml.

Trafalgar

Answer 3

The error on the Jenkins configuration page is a bit of a red herring.

Essentially, what's happening is that the test report xml file hasn't been generated by the build job. So, you then get this error:

Recording test results
No test report files were found. Configuration error? 

Of course, the location must be configured correctly. For that, see this post:

How to configure test-reports on Jenkins CI working with grails?

So, how to fix the error? The key is to study the console output to check whether the tests did successfully run. Chances are they didn't, and that's why the error has happened.

Once you get the tests running successfully, assuming that you correctly configured the location, you should be ok.

You're using JUnit so it'll be a Java project. I'll note here in case it may help others, that we were running Xcode. The tests weren't being run.

Buried in Jenkins console output just above the error was this note:

 note: RunUnitTests exited without running tests because TEST_AFTER_BUILD was set to NO.

Popping back into Xcode, and setting the UnitTests target's Test After Build flag to YES did the trick. That's under the Unit Testing section. You can also of course set the flag at the Project level, and change the target's to 'Other', setting a value of $(inherited).

Gebelein

Answer 4

Your results file is not stored at correct location and Jenkins plugin cannot find it. After your tests are executed and XML file is generated do you store it anywhere?

I suggest try make it working by replacing result.xml with '*' (assuming this is the only XML file that is supposed to be stored there) and if this is going to work then start working on correct file name.

We had the same issue in our configuration. Making sure that generated result XML is stored where the plugin expect it was the key. You can determine workspace root from your project config.

Wraparound

Answer 5

Jenkins has xunit plugin that converts googletest xml to junit format: https://plugins.jenkins.io/xunit/.

Example of pipeline

pipeline {
    agent any
    stages {
        stage('Test'){
            steps {
                sh "run_tests.bash"
            }
        }
    }
    post {
        always{
            xunit (
                thresholds: [ skipped(failureThreshold: '0'), failed(failureThreshold: '0') ],
                tools: [ GoogleTest(pattern: 'reports/*.xml') ]
            )
        }
    }
}

Other useful links:

• https://jenkins.io/doc/pipeline/steps/xunit/
• https://jenkins.io/blog/2016/10/31/xunit-reporting/

Goldcrest

It seems like there is a bracket ')' too much in your code. – Homelike

removed bracket – Goldcrest

Answer 6

Here is a windows batch version for converting the google-test "notRun" to junit "skipped" via windows batch. I know that there are more elegant ways, but this one only requires windows batch and does the job

rem convert gtest xml to junit compatible format (replace notRun by skipped)
IF EXIST  %INTEXTFILE% (
    IF EXIST %OUTTEXTFILE% (
        del %OUTTEXTFILE%
        waitfor fileSystemToDoItsStuff /t 1
    )
    FOR /f "tokens=1,* delims=¶" %%A IN ( '"type %INTEXTFILE%"') DO (
        ECHO."%%A" | findstr /C:"DISABLED_">nul & IF ERRORLEVEL 1 (
            SET modified=%%A
        ) ELSE (
            SET string=%%A
            SET modified=!string:/^>=^>^<skipped /^>^</testcase^>!
        )
        ECHO !modified!>> %OUTTEXTFILE%
    )
    del %INTEXTFILE%
    waitfor fileSystemToDoItsStuff /t 1
    move %OUTTEXTFILE% %INTEXTFILE%
)

Cavit