How do you create nested TestSuites in JUnit 4.x?

Question

I'm trying to add JUnit to a large project and am having difficulties nesting test suites. For example:

@RunWith(Suite.class)
@Suite.SuiteClasses({Test2.class, .....})
public class Test1{
}

@RunWith(Suite.class)
@Suite.SuiteClasses({Test3.class, .....})
public class Test2{
  //might have tests
}

@RunWith(Suite.class)
public class Test3{
  //tests here
}

Each class runs the tests in it's own package, as well as all sub-package TestSuites

Running Test1 results in java.lang.Exception: No runnable methods. How do I get this to work? Is there a better way to organize tests without a without a huge list somewhere?

Tupelo

Removing @RunWith(Suite.class) on all the Test3s and adding a missed annotation one one of the Test2s fixed all the issues. However this doesn't seem ike the best way to do it. Is there a better way to run all tests under a specific directory? – Tupelo

See also https://mcmap.net/q/183740/-how-to-add-test-cases-to-a-suite-using-junit. – Flowage

Answer 1

The first problem is that Test3 uses @RunWith(Suite.class), but doesn't contain @Suite.SuiteClasses({Test3.class, .....}). This produces an IntializationError: class 'Test3' must have a SuiteClasses annotation. Since you aren't intending there to be any classes underneath Test3, this annotation should be removed.

The second problem of Exception: No runnable methods is almost always due to forgetting to add @Test to a test. You didn't put the tests in your sample, so I don't know if that's actually the case or not, but it's the most likely cause.

The following is a working version of your code that allows tests to be run from any class:

Test1.java

import org.junit.runner.*;
import org.junit.runners.*;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses({Test2.class})
public class Test1 {

}

Test2.java

import org.junit.runner.*;
import org.junit.runners.*;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses({Test3.class})
public class Test2 {

}

Test3.java

import static org.junit.Assert.*;

import org.junit.*;

public class Test3 {

    @Test
    public void testTrue(){
        assertTrue(true);
    }
}

As for whether or not there is a better way than to organize things, I guess it just depends on how you decide to create the classes. Since you can add suites to suites, as this demonstrates, you can create smaller chunks of suites that depend on everything, like a tree. For instance, what I generally do is:

AllTestSuite
    TextParsingSuite
    GuiSuite
        SwingSuite
        JavaFXSuite
    FileIOSuite

A test is added to the most relevant suite. In the end, I don't think I have any suite with more than 10 test classes/suites or so. If I do, it's time to make a new sub-suite. In other words, there is no "huge list somewhere", just a lot of smaller lists that are combined together into another list in order to effectively make one big list.

I suppose you could use some tool to dynamically find all of the Java classes containing tests, but JUnit itself doesn't support this behavior (it only runs the tests you tell it to, which I personally think is a good thing).

Buffalo

Answer 2

Personally, I use maven/ant/Eclipse to run sets of tests.

If you're using maven, look at surefire. To run all junit tests for a module, add the following to the pom:

<build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.11</version>
        </plugin>
      </plugins>
    </pluginManagement>
</build>

For Ant, look at the JUnit task:

<junit printsummary="yes" haltonfailure="yes">
  <classpath>
    <pathelement location="${build.tests}"/>
    <pathelement path="${java.class.path}"/>
  </classpath>

  <formatter type="plain"/>

  <test name="my.test.TestCase" haltonfailure="no" outfile="result">
    <formatter type="xml"/>
  </test>

  <batchtest fork="yes" todir="${reports.tests}">
    <fileset dir="${src.tests}">
      <include name="**/*Test*.java"/>
      <exclude name="**/AllTests.java"/>
    </fileset>
  </batchtest>
</junit>

For Eclipse, right click on the package and select 'Run as JUnit'. This will run all of the tests it can find in those packages. There will be similar functionality in Intellij.

Deejay