How to parameterize junit Test Suite

Question

Is it possible to parameterize a TestSuite in junit 4 ?

For declaring a class as a test suite I need the annotation @RunWith(Suite.class), but the same annotation is also needed to declare the test as parameterized: @RunWith(Parameterized.class) so I cannot add both to the same class.

I found a similar question in this site that did not help much. So far, all the examples I have found explain how to parameterize simple unit tests, not a complete test tuite.

Filature

Answer 1

I believe the basic answer is No, because as you said, the @RunsWith only take one parameter. I found a blog posting that got a bit creative in how to handle this situation.

We don't use the parameterized tests, but may you could create a separate suite like we do that only lists the test classes and the parameterized test could be part of that. I modified our test suite to include a parameterized test class to part of the suite and it ran fine. We create our suite like below where PrimeNumberCheckerTest was a simple I pulled from the web.

package com.jda.portfolio.api.rest.server;

import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses({  com.mycompany.api.rest.server.resource.TestCartResourceJava.class, 
                 com.mycompany.api.rest.server.resource.TestCustomerResource.class,
                 com.mycompany.api.rest.server.resource.TestWizardProfileResource.class,
                 com.mycompany.api.rest.server.interceptor.TestBaseSearchInterceptor.class, 
                 com.mycompany.api.rest.server.resource.TestQueryParameters.class, 
                 com.mycompany.api.rest.server.expression.TestCartExpressionGenerator.class, 
                 com.mycompany.api.rest.server.expression.TestPreferenceExpressionGenerator.class, 
                 com.mycompany.api.rest.server.PrimeNumberCheckerTest.class, 
                 })
public class AllTests {}

Here's the source for the parameterized test case;

package com.jda.portfolio.api.rest.server:

import static org.junit.Assert.*;
import java.util.Arrays;
import java.util.Collection;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Parameterized.class)
@SuiteClasses({PrimeNumberCheckerTest.class})
public class PrimeNumberCheckerTest {
  private Integer inputNumber;
  private Boolean expectedResult;
  private PrimeNumberChecker primeNumberChecker;

  @Before
  public void initialize() {
     primeNumberChecker = new PrimeNumberChecker();
  }

  // Each parameter should be placed as an argument here
  // Every time runner triggers, it will pass the arguments
  // from parameters we defined in primeNumbers() method
  public PrimeNumberCheckerTest(Integer inputNumber, 
     Boolean expectedResult) {
     this.inputNumber = inputNumber;
     this.expectedResult = expectedResult;
  }

  @Parameterized.Parameters
  public static Collection primeNumbers() {
     return Arrays.asList(new Object[][] {
        { 2, true },
        { 6, false },
        { 19, true },
        { 22, false },
        { 23, true }
     });
  }

  // This test will run five times since we have as many parameters defined
  @Test
  public void testPrimeNumberChecker() {
     System.out.println("Parameterized Number is : " + inputNumber);
     assertEquals(expectedResult, 
     primeNumberChecker.validate(inputNumber));
  }

Gasworks

-1 because I think you missed the point. Sergio wanted to parameterise the suite itself, not only execute a single, parameterised test – Jujutsu

Answer 2

I was able to parameterize a test suite and use its data in a test class member of the suite as follows:

In JUTsuite:

@RunWith(Suite.class)
@Suite.SuiteClasses({ 
    JUT_test1.class,
})

public class JUTSuite{  
    // Declare all variables/objects you want to share with the test classes, e.g.
    protected static List<Fx> globalFxs;
    // This is the data list we'll use as parameters
    protected static List<Dx> globalDxs;

    @Parameters
    public static Collection<Object[]> data(){
        // Instantiate object list for parameters.  
        // Note: you must do it here and not in, say, @BeforeClass setup()
        // e.g.
        globalDxs=new ArrayList<Dx>(serverObj.values());

        Collection<Object[]> rows=new ArrayList<Object[]>();
        for(Dx d:globalDxs) {
            rows.add(new Object[]{d});
        }
        return rows;
    }

    @BeforeClass
    public static void setUp() throws Exception {
        // Instantiate/initialize all suite variables/objects to be shares with test classes
        // e.g. globalFxs=new ArrayList<Fx>();
    }

    @AfterClass
    public static void tearDown() throws Exception {
        // Clean up....
    }
}

Next, in test class:

@RunWith(Parameterized.class)
public class JUT_test1 {
    // declare local names (if desired) for suite-wide variable/objects 
    // e.g. 
    private static List<Fx> globalFxs;

    // This is the test parameter:      
    private Dx d;

    public JUT_test1(Dx d){
        this.d=d;
    }

    @Parameters
    public static Collection<Object[]> data(){
    // Note: we're calling the suite's data() method which has already executed.
        return JUTSuite.data();
    }

    @BeforeClass
    public static void setUpBeforeClass() throws Exception {
    // (If desired)initialize local variables by referencing suite variables.
    // e.g.globalFxs=JUTSuite.globalFxs;
    }
}

Aubyn

Can you please explain if JUT_KB_integrity is supposed to be a constructor or a setter? – Contrecoup

Never mind, I corrected it. Thank you for the detailed explanation. – Contrecoup

Answer 3

I agree, it's not possible with the provided classes, but there are workarounds that will get you most of the way there, like @mikemil's.

I've spent some time extending Suite and delegating to Parameterized, with partial success; it is possible to build runner that does what you want, and the code is more-or-less written for you in those two classes. The way those classes interact (in particular, the definition of Parameterized#getChildren()) makes it difficult to extend or delegate to those classes to accomplish what you need, but creating a whole new class than extends ParentRunner and lifts code from the other two would be fairly easy.

I'll try to get more time to come back to this later. If you do build a new runner before I get around to it, please post it as an answer, I'd love to use it myself.

Heddle

Did you find some time to do so in the meantime, Paul? :-) – Jujutsu

Answer 4

You're right: Both Suite and Parameterized are Runners and only one Runner may be used to run a test at a time. Standard JUnit 4 doesn't provide a combined Runner.

You can either implement your own Runner or have a look at this ready-to-use library which provides a ParameterizedSuite Runner: https://github.com/PeterWippermann/parameterized-suite

A parameterized test suite looks like this:

@RunWith(ParameterizedSuite.class)
@SuiteClasses({OneTest.class, TwoTest.class})
public class MyParameterizedTestSuite {
    @Parameters(name = "Parameters are {0} and {1}")
    public static Object[] params() {
        return new Object[][] {{'A',1}, {'B',2}, {'C',3}};
    }

Jujutsu

Answer 5

the best solution will be, keep suit classes separately in a blank class. For example, I am testing logins as Parameterized tests and putting in a suit (for navigation performance measurement)

     @RunWith(Suite.class)
@Suite.SuiteClasses({
            LoginPageTest.class,
            HomePageTests.class})
    public class PerformanceTests {
    }

and LoginPageTest is actually Parameterizedtests

@RunWith(Parameterized.class)
public class LoginPageTest
{...}

Fults

Answer 6

As already stated multiple times, it's not possible to parameterize a test suite with the runners provided by JUnit 4.

Anyway, I wouldn't recommend to make your testclasses dependent from some externally provided state. What if you want to run a single testclass?

I would recommend to make your separate test classes @Parameterized and use a utility class to provide the parameters:

@RunWith(Suite.class)
@SuiteClasses({ Test1.class, Test2.class })
public class TestSuite {
    // suite
}

@RunWith(Parameterized.class}
public class Test1 {
    public Test1(Object param1) { /* ... */ }

    @Parameters
    public static Collection<Object[]> data() {
        return TestParameters.provideTestData()
    }

    @Test
    public void someTest() { /* ... */ }
}

@RunWith(Parameterized.class}
public class Test2 {
    public Test2(Object param1) { /* ... */ }

    @Parameters
    public static Collection<Object[]> data() {
        return TestParameters.provideTestData()
    }

    @Test
    public void someOtherTest() { /* ... */ }
}

class TestParameters {
    public static Collection<Object[]> provideTestData() {
        Collection<Object[]> data = new ...;
        // build testdata
    return data;
}

Rib

Answer 7

Maybe this answer helps: Parameterized unit test suites

It uses @RunWith(Enclosed.class) and seems to solve the problem.

Trajan