How can I stop maven from running TEST twice?

Question

I have a big project with around 5000 test cases. When running mvn clean install it will run the test goal twice (one as part of the install and the second time as part of the surefire plugin).

Why it needs to run the test for the second time? And is there away to force surefire to use the test goal results instead of re-invoking it's own? I see it as a waste of time and machine resource, especially recently the second round of running the test caused a PermGen build error and no matter how much heap I pump into maven runner it still dies in the second testing round.

This is my surefire pluging configuration:

 <plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <executions>
       <execution>
         <id>default-test</id>
         <phase>test</phase>
         <goals>
             <goal>test</goal>
         </goals>
         <configuration>
            <parallel>classes</parallel>
            <threadCount>3</threadCount>
         </configuration>
      </execution>
   </executions>
</plugin>

Is there a way to tweak the plugin to better handle machine resources?

Here is the full default maven profile which gets executed:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-war-plugin</artifactId>
            <version>2.0.2</version>
            <configuration>
                <archive>
                    <manifest>
                        <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
                    </manifest>
                    <manifestEntries>
                        <Build-Number>${build.number}</Build-Number>
                        <Job-Name>${job.name}</Job-Name>
                        <Build-Url>${build.url}</Build-Url>
                        <Git-Commit>${git.commit}</Git-Commit>
                        <Git-Branch>${git.branch}</Git-Branch>
                        <Timestamp>${maven.build.timestamp}</Timestamp>
                        <StyleGuide-Version>${styleguide.version}</StyleGuide-Version>
                    </manifestEntries>
                </archive>
                <warName>pss</warName>
            </configuration>
        </plugin>
        <plugin>
            <groupId>com.cj.jshintmojo</groupId>
            <artifactId>jshint-maven-plugin</artifactId>
            <version>1.3.0</version>
            <executions>
                <execution>
                    <goals>
                        <goal>lint</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <options>maxparams:5,camelcase,eqeqeq,forin,immed,latedef,noarg,noempty,nonew,expr</options>
                <directories>
                    <directory>src/main/webapp/js/page</directory>
                </directories>
                <excludes>
                    <exclude>src/main/webapp/js/page/marketingPreferences.js</exclude>
                    <exclude>src/main/webapp/js/page/changeCarParkingDetails.js</exclude>
                    <exclude>src/main/webapp/js/page/angularjs-app.js</exclude>
                    <exclude>src/main/webapp/js/page/content-cover.js</exclude>
                    <exclude>src/main/webapp/js/page/amendmentConfirm.js</exclude>
                </excludes>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.lesscss</groupId>
            <artifactId>lesscss-maven-plugin</artifactId>
            <version>1.3.3</version>
            <executions>
                <execution>
                    <id>bingleless</id>
                    <configuration>
                        <sourceDirectory>${project.basedir}/src/main/webapp/app-resources/</sourceDirectory>
                        <outputDirectory>${project.basedir}/src/main/webapp/app-resources/</outputDirectory>
                        <includes>
                            <include>**\/policy-self-service\/**\/*pss-sg.less</include>
                        </includes>
                        <compress>true</compress>
                    </configuration>
                    <goals>
                        <goal>compile</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>net.alchim31.maven</groupId>
            <artifactId>yuicompressor-maven-plugin</artifactId>
            <version>1.1</version>
            <executions>
                <execution>
                    <goals>
                        <goal>compress</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <excludes>
                    <exclude>**/*.min.js</exclude>
                    <exclude>**/*.min.css</exclude>
                    <exclude>**/style-guide/**</exclude>
                    <exclude>**/generated/**</exclude>
                    <exclude>**/app-resources/common/**</exclude>
                    <exclude>**/app-resources/bower_components/**</exclude>
                    <exclude>**/app-resources/policy-self-service/**</exclude>
                </excludes>
                <nosuffix>true</nosuffix>
                <jswarn>false</jswarn>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-checkstyle-plugin</artifactId>
            <configuration>
                <logViolationsToConsole>true</logViolationsToConsole>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven-surefire-plugin.version}</version>
            <configuration>
                <parallel>classes</parallel>
                <threadCount>3</threadCount>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-report-plugin</artifactId>
            <version>${maven-surefire-report-plugin.version}</version>
            <executions>
                <execution>
                    <phase>install</phase>
                    <goals>
                        <goal>report-only</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Banff

Do you tried to remove the executions part? – Penicillin

Please add (trimmed) the output of mvn clean install, especially the lines starting with [INFO] --- maven-surefire-plugin - that should help track when and how many times the plugin is executed. – Sovran

It only runs once, do you have other plugins? – Toluol

@Penicillin Yes I tried removing the execution part, still calling it twice. – Banff

@Toluol The only other plugin is maven-surefire-report-plugin. You can see the full maven <build> above. – Banff

@AdamMichalik Could you please explain what do you mean by 'adding trimmed'? Can you suggest a revised configuration? Thanks. – Banff

Trimmed output means a short copy of your log when you run mvn clean install. It helps to see when the maven test is invoked – Strenuous

Answer 1

I think the <execution> in your pom.xml causes the 2nd test run. Maven sees it as another goal to execute in addition to the default goal in the test phase.

Since maven-surefire-plugin is a plugin used by default in Maven for test phase, you only need to provide the <configuration> part outside of <execution>. Modify your pom.xml as below

<plugins>
  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <configuration>
        <parallel>classes</parallel>
        <threadCount>3</threadCount>
    </configuration>
  </plugin>
</plugins>

Strenuous

This is correct, cause maven-surefire-plugin is by default already bound to the life cycle. Best is to give configuration via pluginManagement instead of in build.. – Psittacosis

@Strenuous Tried that, didn't work either. Still calling TESTS twice. – Banff

@Banff Did you ever find the solution? – Hierogram

A good way of finding out what is going on is to use: mvn help:effective-pom – Mcleroy

Answer 2

If in your pom both cobertura and surefire plugin are there tests will execute twice. To avoid that add execution phase as none in cobertura plugin like mentioned below:

<plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>cobertura-maven-plugin</artifactId>
            <executions>
                <execution>
                    <phase>none</phase>
                </execution>
            </executions>
</plugin>

Gynaecomastia

Answer 3

In my case, I found that:
because of using rerunFailingTestsCount of maven-surefire-plugin, so in case of there is a failed test, it will be run again.

By the way:
I am against trying to re-run flaky test multiple times till success, instead we can resolve the root cause of it.

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        ...
        <!-- Workaround to make builds more stable while having flaky tests -->
        <rerunFailingTestsCount>1</rerunFailingTestsCount>
    </configuration>
</plugin>

Conni

Answer 4

I was able to do it by only running mvn surefire-report:report -DskipTests. it ran the whole soapui project- in my soapuimaven plugin configuration and then generated the surefire html report.

Judoka