How to fix "Test reports were found but none of them are new. Did tests run?" in Jenkins

Question

I am getting the error "Test reports were found but none of them are new. Did tests run?" when trying to send unit test results by email. The reason is that I have a dedicated Jenkins job that imports the artifacts from a test job to itself, and sends the test results by email. The reason why I am doing this is because I don't want Jenkins to send all the developers email during the night :) so I am "post-poning" the email sending since Jenkins itself does not support delayed email notifications (sadly).

However, by the time the "send test results by email" job executes, the tests are hours old and I get the error as specified in the question title. Any ideas on how to get around this problem?

Renal

Answer 1

You could try updating the timestamps of the test reports as a build step ("Execute shell script"). E.g.

cd path/to/test/reports
touch *.xml

Crabber

Note that this may lead Jenkins to believe that removed tests have been executed, if a test is removed but the test reports are kept. – Prodrome

@DavidPärsson - solution? Clean up tests output dir ? – Evesham

I actually haven't found a good solution to this. Cleaning up the test output dir will force the tests to re-run when they could have been up to date. This could be acceptable if the feedback time is not an issue. – Prodrome

Just add you should execute the "send test results by email" job quickly after you update the XML files timestamps. Otherwise, Jenkins will still complain they are not new. – Goldplate

How should we achieve the same in case of Windows server cd path/to/test/reports touch *.xml – Azo

This answer can be improved by adding "why" this happens. – Restorative

Anybody get ` Permission denied` with this? – Challah

@DavidPärsson Please consider that in some cases it is better not to touch the files but to delete them. There might be tests (acceptance or integration tests) which depend on other systems or environment conditions and might return other results now. – Peck

Answer 2

mvn clean test 

via terminal or jenkins. This generates new tests reports.

The other answer that says cd path/to/test/reports touch *.xml didn't work for me, but mvn clean test yes.

Scutate

It could be the correct answer, it's always a good practice to clean before running tests – Tunisia

Answer 3

Updating the last modified date can also be achieved in gradle itself is desired:

task jenkinsTest{
    inputs.files test.outputs.files
    doLast{
        def timestamp = System.currentTimeMillis()
        test.testResultsDir.eachFile { it.lastModified = timestamp }
    }
}

build.dependsOn(jenkinsTest)

As mentioned here: http://www.practicalgradle.org/blog/2011/06/incremental-tests-with-jenkins/

Vandavandal

Also, as gradle warns, testResultsDir is deprecated and will be removed in Gradle 2.0. Use getReports().getJunitXml().getDestination() instead. – Teofilateosinte

Answer 4

Here's an updated version for Jenkinsfile (Declarative Pipeline):

pipeline {

    agent any

    stages {
        stage('Build') {
            steps {
                sh 'make build'
            }
        }
        stage('Test') {
            steps {
                sh 'make test'

                script {
                    def testResults = findFiles(glob: 'build/reports/**/*.xml')
                    for(xml in testResults) {
                        touch xml.getPath()
                    }
                }
            }
        }
    }

    post {
        always {
            archiveArtifacts artifacts: 'build/libs/**/*.jar', fingerprint: true
            junit 'build/reports/**/*.xml'
        }
    }
}

Phosgenite

I believe this only works if you have the 'Pipeline Utility Steps' plugin installed on your Jenkins server. I don't and the admin doesn't like to add plugins without vetting them first (takes a while) – Magna

This works, but creates a lot of garbage logs like [Pipeline] touch – Dehydrogenate

Answer 5

Because gradle caches results from previous builds I ran into the same problem.

I fixed it by adding this line to my publish stage:

sh 'find . -name "TEST-*.xml" -exec touch {} \\;'

So my file is like this:

....
stage('Unit Tests') {

    sh './gradlew test'

}

stage('Publish Results') {

    // Fool Jenkins into thinking the tests results are new
    sh 'find . -name "TEST-*.xml" -exec touch {} \\;'

    junit '**/build/test-results/test/TEST-*.xml'

}

Magna

Not working for me. Using xUnit plugin to have more control over passed / failed test thresholds. – Laryngoscope

Answer 6

Had same issue for jobs running repeatedly (every 30 mins).

For the job, go to Configure, Build, Advanced and within the Switches section add: --stacktrace --continue --rerun-tasks

Plowman

Answer 7

This worked for me

1. Navigate to report directory cd /report_directory
2. Delete all older report rm *.xml
3. Add junit report_directory/*.xml in pipeline
4. Rerun the test script , navigate to Build Number → Test Result

Make sure you have one successful build without any failure, only after this you can able to see the reports

Deraign

Answer 8

Make sure that you have mentioned the correct path against "Test report XMLs" under jenkins configuration, such as "target/surefire-reports/*.xml" There is no need to touch *.xml as jenkins won't complain even though test results xml file does not change.

Bake

Answer 9

if you use Windows slave, you can 'touch' results using groovy pipeline stage with powershell:

powershell 'ls "junitreports\\*.*" | foreach-object { $_.LastWriteTime = Get-Date }'

Agronomics

Answer 10

It happens if you are using a test report which is not modified by that job in that run.

In case for test purpose if you are testing with already created file then, add below command inside jenkins job under Build > Execute Shell

chmod -R 775 /root/.jenkins/workspace/JmeterTest/output.xml

echo " " >> /root/.jenkins/workspace/JmeterTest/output.xml

Above command changes timestamp of file hence error wont display.

Note: To achieve same in Execute Shell instead of above, do not try renaming file using move mv command etc. it won't work , append and delete same for change file timestamp only works.

Accalia

Answer 11

For me commands like chmod -R 775 test-results.xml or touch test-results.xml does not work due to permission error. As work around use is to set new file in test report settings and command to copy old xml report file to new file.

Challah

Answer 12

you can add following shell command to your "Pre Steps" section when configure your job on Jenkins

mvn clean test

this will clean the test

Tito

Answer 13

Here's an updated version of the gradle task that touch each test result files. From Jenkins pipeline script, just call "testAndTouchTestResult" task instead of "test" task.

The code below is with Kotlin syntax:

tasks {
register("testAndTouchTestResult") {
    setGroup("verification")
    setDescription("touch Test Results for Jenkins")
    inputs.files(test.get().outputs)
    doLast {
        val timestamp = System.currentTimeMillis()
        fileTree(test.get().reports.junitXml.destination).forEach { f ->
            f.setLastModified(timestamp)
        }
    }
}

}

Dogwatch

Answer 14

The solution for me was delete node_modules and change node version (from 7.1 to 8.4) on jenkins. That's it.

Obese