What is the difference between the Maven Surefire and Maven Failsafe plugins?

Question

What is the difference between Maven Surefire and Maven Failsafe plugins?
I have searched all over web, but did not get the answer.

Bouie

Answer 1

In simple words, the Failsafe plugin is designed to run integration tests while Surefire to run unit tests.

This is further explained in Maven FAQ:

• maven-surefire-plugin is designed for running unit tests and if any of the tests fail then it will fail the build immediately.

• maven-failsafe-plugin is designed for running integration tests, and decouples failing the build if there are test failures from actually running the tests.

  The name "failsafe" was chosen both because it is a synonym of surefire and because it implies that when it fails, it does so in a safe way.

  The Failsafe Plugin has two goals:

  • failsafe:integration-test runs the integration tests of an application,
  • failsafe:verify verifies that the integration tests of an application passed.

See also:

• Benefits of Maven FailSafe Plugin at SO

Jordonjorey

The interesting question is why. Many build tools work perfectly without making this arguably rather artificial and limited distinction. Configuring both plugins together and absorbing their reports in consistent ways is quite the PITA. – Sunshine

And no, the differences stated in the other answers do not provide sufficient reason for having multiple tools for what should be filters and flags. – Sunshine

I think this answer fails to make a specific case for Failsafe. It also uses two more lifecycle phases within maven, pre-integration-test and post-integration-test. This is generally used to bring up an environment, or do some sort of external setup. If a test fails when running in failsafe, it is guaranteed that post-integration-test will run and tear things down. This is because, as stated in the answer, it does not fail until the verify phase. That is the real benefit of the plugin, it will ALWAYS run post-integration-test – Corbet

Technically, the plugin itself does not execute these phases, but it does adhere to the convention. Surefire will fail immediately when a test failure is detected and no subsequent maven lifecycle phases are executed. – Corbet

Answer 2

From https://maven.apache.org/surefire/maven-failsafe-plugin/, I would say that the difference between Surefire and Failsafe is the way they fail:

If you use the Surefire Plugin for running tests, then when you have a test failure, the build will stop at the integration-test phase and your integration test environment will not have been torn down correctly.

The Failsafe Plugin is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. The Failsafe Plugin will not fail the build during the integration-test phase, thus enabling the post-integration-test phase to execute.

Lund

Then the question becomes: why ever use Surefire if they are telling us it can leave stuff in an incorrect state? – Tameshatamez

Answer 3

In my country its the second google result when searching for "maven failsafe maven surefire" to get to this FAQ: Difference between maven-failsafe-plugin and maven-surefire-plugin which states:

maven-surefire-plugin is designed for running unit tests and if any of the tests fail then it will fail the build immediately.

maven-failsafe-plugin is designed for running integration tests, and decouples failing the build if there are test failures from actually running the tests."

Smriti