Login/Register
Jenkins Maven build reports success despite failed unit test
Asked Answered
Solved mavenjenkinsjunitmaven-surefire-plugin
D

1

9

I’m using Maven 3.2.3, JUnit 4.12, and SureFire 2.18.1 within Jenkins. I have a multi-module Maven job set up with the following Maven options:

clean install -U -e -P qa -Dci -DskipITs

However, when I run my Maven build, although one of my JUnit tests fail, the Maven module itself reports "SUCCESS" (output after) ... 

Executing Maven:  -B -f /var/lib/jenkins/workspace/subco/subco/pom.xml clean install -U -e -P qa -Dci -DskipITs


…

———————————————————————————
 T E S T S
-------------------------------------------------------
Running org.mainco.subco.user.domain.UserTest
Tests run: 24, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec <<< FAILURE! - in org.mainco.subco.user.domain.UserTest
testHasRoleNullRoles(org.mainco.subco.user.domain.UserTest)  Time elapsed: 0.006 sec  <<< FAILURE!
java.lang.AssertionError: Failed to indicate we found right role for user.
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.assertTrue(Assert.java:41)
    at 

…

[INFO] Tests are skipped.
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ othersubmodule ---
[INFO] Installing /var/lib/jenkins/workspace/subco/subco/othersubmodule/target/othersubmodule.war to /var/lib/jenkins/.m2/repository/org/mainco/subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.war
[INFO] Installing /var/lib/jenkins/workspace/subco/subco/othersubmodule/pom.xml to /var/lib/jenkins/.m2/repository/org/mainco/subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] subco ....................................... SUCCESS [4.849s]
[INFO] database .......................................... SUCCESS [18.340s]
[INFO] session ........................................... SUCCESS [6.601s]
[INFO] main .............................................. SUCCESS [8:00.887s]
[INFO] mainclient ........................................ SUCCESS [4:01.490s]
[INFO] orders ............................................ SUCCESS [50.831s]
[INFO] SBD myproject Webapp .................................. SUCCESS [1:45.484s]
[INFO] subco Admin Module .......................... SUCCESS [1:01.318s]
[INFO] Other Submodule ................................ SUCCESS [6.162s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16:20.351s
[INFO] Finished at: Thu Aug 20 21:17:57 UTC 2015
[INFO] Final Memory: 76M/748M
[INFO] ------------------------------------------------------------------------
Waiting for Jenkins to finish collecting data
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orgsclient/pom.xml to org.mainco.subco/orgsclient/83.0.0-SNAPSHOT/orgsclient-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orgsclient/target/orgsclient.war to org.mainco.subco/orgsclient/83.0.0-SNAPSHOT/orgsclient-83.0.0-SNAPSHOT.war
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/pom.xml to org.mainco.subco/subco/83.0.0-SNAPSHOT/subco-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/myproject/pom.xml to org.mainco.subco/myproject/83.0.0-SNAPSHOT/myproject-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/myproject/target/myproject.war to org.mainco.subco/myproject/83.0.0-SNAPSHOT/myproject-83.0.0-SNAPSHOT.war
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/database/pom.xml to org.mainco.subco/database/83.0.0-SNAPSHOT/database-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/database/target/database-83.0.0-SNAPSHOT.jar to org.mainco.subco/database/83.0.0-SNAPSHOT/database-83.0.0-SNAPSHOT.jar
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/othersubmodule/pom.xml to org.mainco.subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/othersubmodule/target/othersubmodule.war to org.mainco.subco/othersubmodule/83.0.0-SNAPSHOT/othersubmodule-83.0.0-SNAPSHOT.war
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/sbadmin/pom.xml to org.mainco.subco/sbadmin/83.0.0-SNAPSHOT/sbadmin-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/sbadmin/target/sbadmin.war to org.mainco.subco/sbadmin/83.0.0-SNAPSHOT/sbadmin-83.0.0-SNAPSHOT.war
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/pom.xml to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/target/main.jar to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT.jar
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/main/target/main-tests.jar to org.mainco.subco/main/83.0.0-SNAPSHOT/main-83.0.0-SNAPSHOT-tests.jar
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orders/pom.xml to org.mainco.subco/orders/83.0.0-SNAPSHOT/orders-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/orders/target/orders.jar to org.mainco.subco/orders/83.0.0-SNAPSHOT/orders-83.0.0-SNAPSHOT.jar
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/session/pom.xml to org.mainco.subco/session/83.0.0-SNAPSHOT/session-83.0.0-SNAPSHOT.pom
[JENKINS] Archiving /var/lib/jenkins/workspace/subco/subco/session/target/session.jar to org.mainco.subco/session/83.0.0-SNAPSHOT/session-83.0.0-SNAPSHOT.jar
Sending e-mails to: [email protected] [email protected] [email protected] [email protected] [email protected]

channel stopped Sending e-mails to: [email protected] Warning: you have no plugins providing access control for builds, so falling back to legacy behavior of permitting any downstream builds to be triggered Finished: UNSTABLE

How do I get the Maven module to report FAILURE if one of the unit tests fails? I realize the overall build reports as UNSTABLE in Jenkins, but for readability, i would prefer SUCCESS to show only if everything passed.

Edit: Here is the surefire plugin configuration

                    <plugin>
                            <groupId>org.apache.maven.plugins</groupId>
                            <artifactId>maven-surefire-plugin</artifactId>
                            <version>2.18.1</version>
                            <configuration>
                                    <reuseForks>true</reuseForks>
                                    <argLine>-Xmx2048m -XX:MaxPermSize=512M -noverify -XX:-UseSplitVerifier ${argLine}</argLine>
                                    <skipTests>${skipAllTests}</skipTests>
                            </configuration>
                    </plugin>
Disclimax answered 21/8, 2015 at 13:33 Comment(6)
So you get Unstable and you want Failure, is that correct? – Lelialelith
I think he wants maven to fail the build instead of showing success. Out of curiosity, have you tried manually setting the test failure ignore (maven.apache.org/surefire/maven-surefire-plugin/…) property to false? – Tenishatenn
Hi, I want what spoonybard896 described -- the Maven build to fail instead of showing success. I haven't set the property you mention explicitly. Including my surefire config in case that adds any clarity to this. – Disclimax
It's difficult to tell without seeing your whole setup. Have you actually verified that the maven.test.failure.ignore user property is false? – Cetinje
This can only be set by the plugin, right? Is there a way to output what this value is as Maven runs, (e.g. adding "-X" or somethign similar)? – Disclimax
Possible duplicate of How do I make Jenkins build fail when Maven unit tests fail? – Kathrynekathy
K
12

This answer is basically what @spoonybard896 suggested in a comment.

Add -Dmaven.test.failure.ignore=false to MAVEN_OPTS. That can be added in the Build -> Advanced section of your Jenkins Job.

Normally this should be the default value, but with Jenkins and Maven jobs, it isn't so. For those, this property is actually set to true unless you set it yourself.

(disclaimer: looked up the exact Jenkins steps from this answer).

Kathrynekathy answered 7/9, 2015 at 21:31 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.