Benefits of Maven FailSafe Plugin

Question

I read Maven Failsafe plugin is designed specifically to run integration tests. Currently I'm working on a multi-module project and integration tests are in its own separate module, written in TestNg and run using Surefire plugin. We don't have conflicts with unit tests since only integration tests are run in the test phase in that module. And to set up the environment before the tests, and clean it after tests are run, @BeforeSuite @AfterSuite TestNg annotations are used. So there's no need to make use of pre-integration-test phase, integration-test phase, post-integration-test phase utilized by Failsafe plugin.

• Are there any more benefits I'm missing out on, by not using the Failsafe plugin?
• Are there better ways to do my current requirement using Failsafe plugin?
• Can I do my server startup, shut down, file unzipping etc. in the pre-integration-test, post-integration-test phases without writing a maven plugin?

Contreras

Answer 1

If you already have your own test setup/teardown in your suites, which from the looks of it you do, there is not much you can gain from the FailSafe plugin.

The FailSafe plugin is useful in situations where the Setup of your System Under Test is costly or takes a long time such as starting up a Servlet or a distributed system. The way the FailSafe plugin comes handy in these situations is that you can set up this environment in the pre-integration-test phase. This plugin also doesn't stop the execution of the Maven build when a test fails, which allows you to clean up all of your artifacts during the post-integration-test phase, after which it checks the status of your tests and passes or fails the build accordingly during the verify phase.

Latreese

What do you mean? If you don't use skipAfterFailureCount=1, whjch is not a default option, imvn test does not abort if it fails a test – Riebling

Answer 2

Failsafe has one big feature vs Surefire: When a test fails, it does not immediately abort. Instead it lets the clean-up code run (which typically takes down the Jetty server).

Nidanidaros

What do you mean? If you don't use skipAfterFailureCount=1, whjch is not a default option, imvn test does not abort until the end – Riebling

@Riebling when at least one test fails with surefire, Maven will by default stop the build after the test phase. With the failsafe plugin, the integration-test phase will always succeed, then post-Integration-test will run, and then the build will fail in the verify phase – Mincing

I had the same confusion as well. Real difference is that failsafe will continue with failed tests ( this is not a big deal as surefire can also be configured to do this ), but failsafe has the additional goal called verify that gets fired later in the verify phase and this will fail the build if any of the earlier tests had failed. This allows you to undo/teardown all the setup you did before the tests began in the post-integration-test phase – Lemoine

Answer 3

Addressing your third question as it isn't really answered, imho.

Can I do my server startup, shut down, file unzipping etc. in the pre-integration-test, post-integration-test phases without writing a maven plugin?

Taken from this answer to "Maven Failsafe Plugin: how to use the pre- and post-integration-test phases"

It boils down to: pre-integration-test and post-integration-test do nothing per default. You can bind a plugin specific for your task to those phases. Finding a specific plugin depends on what you're trying to do.

Another important thing to point out is default naming conventions used by the maven-failsafe-plugin: It runs test-classes with names starting or ending with IT (as integration test class)

Everyone