How can I run integration tests after building all modules in a multi-module Maven project?

Question

I am working on a large multi-module Maven project. Each module has fast unit tests (using the Surefire plugin), many modules have slow integration tests (using the Failsafe plugin).

I would like to speed up the feedback for "simple" build failures (compilation errors and unit test failures) by running integration tests from all modules after all modules have been built and unit-tested.

Can you suggest a good way of achieving this?

Panchito

Answer 1

Here's a solution that does exactly what I want: using the Failsafe plugin to run just the integration tests without having to move tests out of their natural places or recompile in order to run the tests.

mvn install -DskipITs
mvn failsafe:integration-test

We get Maven to publish the artifacts for the project into a local directory. We can then use that as a repository for the second step. In that second step we then invoke exactly the goal that we need and it grabs the artifacts from the local repository.

The same thing works for Surefire (mvn surefire:test) if you need to run unit tests separately.

I have to admit that I don't fully understand the Maven model and hence why this works. The internet seems unanimously of the view that this kind of thing can't be done (see other answers here), but it's definitely working for me.

Panchito

This doesn't execute unit tests either, right? If you're doing it this way, it might be better to skip integration tests by default (set plugin config in parent POM) and manually execute them any time you want. – Scummy

@VivekChavda Thank you. I've fixed the instructions to use -DskipITs instead. – Panchito

Answer 2

running integration tests from all modules after all modules have been built and unit-tested

To meet this requirement a possible solution is to have an additional (optional) module providing integration tests, that is, all integration tests should be moved to this module, which could be added to the default build via a profile.

<profiles>
   <profile>
      <id>it</id>
      <modules>
        ...
        <module>it-module</module>
      </modules>
   </profile>
</profiles>

In order to have it as last step of the build for the Maven Reactor, this module should also depend on all other modules (probably it would implicitly already be the case).

This module would actually be the only one providing maven-failsafe-plugin configuration and settings. Moreover, it might become more meaningful in case of the usage of an embedded server to test against (i.e. Jetty): the server will be created and shutted down only during the build of this module and not in each and every module requiring it, speeding up this integration phase.

Being the last module of the build, you will make sure that it would be reached only in case of no unit test failures on previous modules and only when the it profile would be active (e.g. during CI builds).

Ronn

Thank you. I'm not a big fan of this idea as most of our integration tests are naturally scoped to a single module (or a small subset of them), rather than being end-to-end tests which exercise all modules. We could, I suppose, introduce parallel modules -- one integration test module for each existing module; but I'd be reluctant to add that much complexity to the build. – Panchito

Answer 3

The short answer is probably to move the failsafe-plugin into a profile.

Since maven itself does not come with a default mapping for the failsafe plugin I assume it is enabled in your parent pom or (worse?) the company parent pom.

The way would be to create a profile to enable the integration tests and only enable that profile when needed. To avoid waste this profile could disable the surefire-tests to not re-execute them. If they are fast anyway that might not be an issue. In that case re-execute them for starting.

I think its ok to extract this into a profile. My general rule for maven profiles is that they are not allowed to change the resulting artifact (they can, but it usually gets messy). In this case you add another execution step. So that should not hurt. Just make sure the profile is enabled when cutting a release :)

Rupe

But in this case we would recompile when running the integration tests -- is that correct (or, at least run through all the checks to see if compilation and dependency download are needed)? I'll check to see how much time this adds to the build. Thanks. – Panchito

yes all phases up to integration-test will be run again. so also compile (code generation, resource handling). If that takes a while for whatever reason I would go with the other answer :) (remember that measuring code-coverage for IT is a bit cumbersome when tests are somewhere else than the code itself) – Rupe