Create multiple parameter sets in one parameterized class (junit)

Question

Currently I have to create a parameterized test class for every method that I want to test with several different inputs. Is there a way to add this together in one file?

Right now there's CalculatorTestAdd.java which has a set of parameters that are used to check if the Add() function works properly. Is there a possbility for me to 'connect' this set to the Add() function and create an additional set meant for the Subtract() method and add this method in the same test class, resulting in one file called CalculatorTest.java?

Willumsen

Too bad you use JUnit... TestNG has exactly what you want with its @DataProvider. – Penholder

@Penholder yeah, but it isn't too hard with JUnit - I've done it before. You just have to jump though a couple of hoops. – Wunder

github.com/junit-team/junit/wiki/Parameterized-tests – Bistort

@DataProvider can be used with JUnit thanks to github.com/TNG/junit-dataprovider , better than JUnitParams. – Underworld

Answer 1

Yes. There's nothing special you have to do. For every set of value(s) of the parameters, each @Test method is run once, so just have one method test add() and another method test subtract().

May I also add that the person who is dictating this requirement is misguided. There is little value in dictating certain design patterns "for all cases" - might as well hire trained monkeys.

Wunder

This seems to work, but it also means half of my tests fail. While it does what I want to do, am I to assume this is as good as it gets? – Willumsen

Put you as solved since it solves my problem indeed, but if anyone has a more elegant solution which can bypass the side-effect that causes me to fail a lot of tests with parameters that aren't meant for that test (and thus don't give me a nice, green bar), I'd be happy to hear it. – Willumsen

This is why I added my final paragraph. Dictating restrictions like this can only lead to problems. The best solution is to have multiple test classes; one that uses once set of parameters, another for a second set, maybe another that doesn't use parameters. Basically do what you need to do and tell your team leader why you did it. Hopefully he will see the light. If not, go over his head and explain why his "coding rules" are causing problems and will both harm the quality of the software and delay the project. I gather you are not using Continuous Integration... you should use it. – Wunder

@JeroenVannevel if you can somehow distinguish the parameters which are not meant for a specific test method, you can use the assume* set of methods to omit the running of those parameters for those methods. Those will then be counted as "skipped", just like when using @Ignore (but they can run for other parameters). – Trio

@PaŭloEbermann i have these distinguish it will be great help if you can share how assume can be used – Harbert

@arjun what do mean exactly? More detail please – Wunder

@Wunder i am having same problem like you i have many method in one class i need to write single testcase class where i can have test method for each and every method. but how can i manage different parameter for each and every method available in Testclass – Harbert

Answer 2

This answer is similar to Tarek's one (the parametrized part), although I think it is a bit more extensible. Also solves your problem and you won't have failed tests if everything is correct:

@RunWith(Parameterized.class)
public class CalculatorTest {
    enum Type {SUBSTRACT, ADD};
    @Parameters
    public static Collection<Object[]> data(){
        return Arrays.asList(new Object[][] {
          {Type.SUBSTRACT, 3.0, 2.0, 1.0},
          {Type.ADD, 23.0, 5.0, 28.0}
        });
    }

    private Type type;
    private Double a, b, expected;

    public CalculatorTest(Type type, Double a, Double b, Double expected){
        this.type = type;
        this.a=a; this.b=b; this.expected=expected;
    }

    @Test
    public void testAdd(){
        Assume.assumeTrue(type == Type.ADD);
        assertEquals(expected, Calculator.add(a, b));
    }

    @Test
    public void testSubstract(){
        Assume.assumeTrue(type == Type.SUBSTRACT);
        assertEquals(expected, Calculator.substract(a, b));
    }
}

Slipcover

An elegant workaround. This is like saying "this test method applies only to this data set". This allows the flexibility to have e.g. 2 test methods that use data set 1 and 3 test methods that use data set 2. – Shaveling

Nice solution - but what happens if each test case requires a different constructor? – Gravois

@robjob27 I think you could have inner classes for the parameters: { Type.SUBSTRACT, new ParamsSubstract(3,2), 1}, { Type.SQROOT, new ParamsSquareRoot(4), 2}}); With the Params classes inheriting from a common class Params that is part of the constructor: public CalculatorTest(Type type, Params params, Double expected){..} But it might complicate everything too much, at that point I would consider whether it is worth to have one class for everything. – Slipcover

I didn't like this approach and it has assumptions and basically we are running both the test cases each time hence 4 test cases are run while i wanted only two test cases. – Livvy

OR use switch-case statements in the same Test method that pattern match with the Enum. That way you only have to run a single test – Mors

Answer 3

Another pure JUnit but yet elegant solution in my view is to encapsulate each parameterized test(s) in their own inner static class and use the Enclosed test runner on the top level test class. This allows you not only to use different parameter values for each test independently of each other but also to test methods with completely different parameters.

This is how it would look like:

@RunWith(Enclosed.class)
public class CalculatorTest {

  @RunWith(Parameterized.class)
  public static class AddTest {

    @Parameters
    public static Collection<Object[]> data() {
      return Arrays.asList(new Object[][] {
          { 23.0, 5.0, 28.0 }
      });
    }

    private Double a, b, expected;

    public AddTest(Double a, Double b, Double expected) {
      this.a = a;
      this.b = b;
      this.expected = expected;
    }

    @Test
    public void testAdd() {
      assertEquals(expected, Calculator.add(a, b));
    }
  }

  @RunWith(Parameterized.class)
  public static class SubstractTest {

    @Parameters
    public static Collection<Object[]> data() {
      return Arrays.asList(new Object[][] {
          { 3.0, 2.0, 1.0 }
      });
    }

    @Parameter(0)
    private Double a;
    @Parameter(1)
    private Double b;
    @Parameter(2)
    private Double expected;

    @Test
    public void testSubstract() {
      assertEquals(expected, Calculator.substract(a, b));
    }
  }

  @RunWith(Parameterized.class)
  public static class MethodWithOtherParametersTest {

    @Parameters
    public static Collection<Object[]> data() {
      return Arrays.asList(new Object[][] {
          { 3.0, 2.0, "OTHER", 1.0 }
      });
    }

    private Double a;
    private BigDecimal b;
    private String other;
    private Double expected;

    public MethodWithOtherParametersTest(Double a, BigDecimal b, String other, Double expected) {
      this.a = a;
      this.b = b;
      this.other = other;
      this.expected = expected;
    }

    @Test
    public void testMethodWithOtherParametersTest() {
      assertEquals(expected, Calculator.methodWithOtherParametersTest(a, b, other));
    }
  }

  public static class OtherNonParameterizedTests {

    // here you can add any other test which is not parameterized

    @Test
    public void otherTest() {
      // test something else
    }
  }
}

Note the usage of the @Parameter annotation in the SubstractTest, which I consider more readable. But this is more a matter of taste.

Lade

Answer 4

Well, now JUnit-5 offers you a solution for this - by redefining a way to write parameterized tests. Now a parameterized test can be defined at a method level using @ParameterizedTest and can be given a method source using @MethodSource.

So in your case you can have 2 separate data source methods for providing input data for your add() and subtract() test methods, both in the same class. Your code should go something like this:

public class CalculatorTest{
    public static int[][] dataSetForAdd() {
        return new int[][] { { 1 , 2, 3 }, { 2, 4, 6 }, { 121, 4, 125 } };
    }
    
    public static int[][] dataSetForSubtract() {
        return new int[][] { { 1 , 2, -1 }, { 2, 4, -2 }, { 121, 4, 117 } };
    }

    @ParameterizedTest
    @MethodSource(value = "dataSetForAdd")
    void testCalculatorAddMethod(int[] dataSetForAdd) {
        Calculator calculator= new Calculator();
        int m1 = dataSetForAdd[0];
        int m2 = dataSetForAdd[1];
        int expected = dataSetForAdd[2];
        assertEquals(expected, calculator.add(m1, m2));
    }

    @ParameterizedTest
    @MethodSource(value = "dataSetForSubtract")
    void testCalculatorAddMethod(int[] dataSetForSubtract) {
        Calculator calculator= new Calculator();
        int m1 = dataSetForSubtract[0];
        int m2 = dataSetForSubtract[1];
        int expected = dataSetForSubtract[2];
        assertEquals(expected, calculator.subtract(m1, m2));
    }
}

Olympian

I think, the annotation @MethodSource has [ no longer | not ] the field names but just the field value – Zuckerman

Change incorporated. – Olympian

Answer 5

I'm sure you are not having this problem anymore, but I thought of 3 ways you can do this, each with its pros and cons. With the Parameterized runner, you'll have to use a workaround.

- Using more parameters with Parameterized

In case you have to load the parameters externally, you simply add a parameter for the expected results.

Pros: less coding, and it runs all the tests.

Cons: new parameters for each different set of tests.

@RunWith(Parameterized.class)
public class CalculatorTest extends TestCase {
    private Calculator calculator;
    private int operator1;
    private int operator2;
    private int expectedSum;
    private int expectedSub;

    public CalculatorTest(int operator1, int operator2, int expectedSum, int expectedSub) {
        this.operator1 = operator1;
        this.operator2 = operator2;
    }

    @Params
    public static Collection<Object[]> setParameters() {
        Collection<Object[]> params = new ArrayList<>();
        // load the external params here
        // this is an example
        params.add(new Object[] {2, 1, 3, 1});
        params.add(new Object[] {5, 2, 7, 3});

        return params;
    }

    @Before
    public void createCalculator() {
        calculator = new Calculator();
    }

    @Test
    public void addShouldAddTwoNumbers() {
        assertEquals(expectedSum, calculator.add(operator1, operator2));
    }

    @Test
    public void subtractShouldSubtractTwoNumbers() {
        assertEquals(expectedSub, calculator.subtract(operator1, operator2));
    }

    @After
    public void endTest() {
        calculator = null;
        operator1 = null;
        operator2 = null;
        expectedSum = null;
        expectedSub = null;
    }
}

- Not using Parameterized runner

This works fine if you set your parameters programatically.

Pros: You can have as many tests as you want without having to set a huge set of parameters.

Cons: More coding, and it stops at the first failure (which might not be a con).

@RunWith(JUnit4.class)
public class CalculatorTest extends TestCase {
    private Calculator calculator;

    @Before
    public void createCalculator() {
        calculator = new Calculator();
    }

    @Test
    public void addShouldAddTwoNumbers() {
        int[] operator1 = {1, 3, 5};
        int[] operator2 = {2, 7, 9};
        int[] expectedResults = {3, 10, 14};

        for (int i = 0; i < operator1.length; i++) {
            int actualResult = calculator.add(operator1[i], operator2[i]);
            assertEquals(expectedResults[i], actualResult);
        }
    }

    @Test
    public void subtractShouldSubtractTwoNumbers() {
        int[] operator1 = {5, 8, 7};
        int[] operator2 = {1, 2, 10};
        int[] expectedResults = {4, 6, -3};

        for (int i = 0; i < operator1.length; i++) {
            int actualResult = calculator.subtract(operator1[i], operator2[i]);
            assertEquals(expectedResults[i], actualResult);
        }
    }

    @After
    public void endTest() {
        calculator = null;
    }
}

- Using JUnitParams

I have no affiliation with Pragmatists, I just found this a few days ago. This framework runs on top of JUnit and handles parameterized tests differently. Parameters are passed directly to the test method, so you can have in the same class different params for different methods.

Pros: achieves the same results as the solutions above without workarounds.

Cons: maybe your company doesn't allow you add a new dependency to the project or forces you to use some bizarre coding rule (like using the Parameterized runners exclusively). Let's face it, it happens more than we'd like to.

Here's a fine example of JUnitParams in action, and you can get the project/check the code on this Github page.

Clapperclaw

this approach fails or will have redundant empty data in case i have more cases for add than substract. – Livvy

yes, with JUnit's 4 Parameterized runner you'd need to have the same number of cases for each test, or you'd need to create two different test classes with their own parameter. Junit 5 finally has a new @ParameterizedTest that allows you to provide values per test, not per class. – Clapperclaw

Answer 6

you can use parameters with https://github.com/piotrturski/zohhak:

@TestWith({
   "1, 7, 8",
   "2, 9, 11"
})
public void addTest(int number1, int number2, int expectedResult) {
    BigDecimal result = calculator.add(number1, number2);
    assertThat(result).isEqualTo...
}

if you want to load parameters from file, you can use http://code.google.com/p/fuzztester/ or http://code.google.com/p/junitparams/

and if you need real flexibility you can use junit's @Parameterized but it clutters your code. you can also use junit's Theories - but it seems an overkill for calculator tests

Efflux

Since Java8, it is possible to repeat the same annotation. Do you know whether this feature exists? – Alphonsoalphonsus

i don't understand your question. how do you want to use repeating annotation to solve the OP question using any of above libs? – Efflux

I would like something like @TestWith("1, 7, 8") @TestWith("2, 9, 11") public void addTest(int number1, int number2, int expectedResult) { BigDecimal result = calculator.add(number1, number2); assertThat(result).isEqualTo... } – Alphonsoalphonsus

nope, zohhak doesn't support it. others, i'm not sure but i don't think so – Efflux

Answer 7

With Junit Jupiter: https://www.petrikainulainen.net/programming/testing/junit-5-tutorial-writing-parameterized-tests/

import intf.ICalculator;

public class Calculator implements ICalculator {
    @Override
    public int plus(int a, int b) {return a + b; }

    @Override
    public int minuis(int a, int b) {return a - b;}

    @Override
    public int multy(int a, int b) {return a * b;}

    @Override  // check in junit byZero
    public int divide(int a, int b) {return a / b;}

}

Test class:

import static org.junit.Assert.assertEquals;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;

class CalculatorJupiter5Test {

    Calculator calculator = new Calculator();

    @DisplayName("Should calculate the correct sum")
    @ParameterizedTest(name = "{index} => a={0}, b={1}, sum={2}")
    @CsvSource({
            "5, 3, 8",
            "1, 3, 4",
            "6, 6, 12",
            "2, 3, 5"
    })
    void sum(int a, int b, int sum) {
        assertEquals(sum, calculator.plus(a, b) );
    }

    @DisplayName("Should calculate the correct multy")
    @ParameterizedTest(name = "{index} => a={0}, b={1}, multy={2}")
    @CsvSource({
        "5, 3, 15",
        "1, 3, 3",
        "6, 6, 36",
        "2, 3, 6"
    })
    void multy(int a, int b, int multy) {
        assertEquals(multy, calculator.multy(a, b) );
    }

    @DisplayName("Should calculate the correct divide")
    @ParameterizedTest(name = "{index} => a={0}, b={1}, divide={2}")
    @CsvSource({
        "5, 3, 1",
        "14, 3, 4",
        "6, 6, 1",
        "36, 2,  18"
    })
    void divide(int a, int b, int divide) {
        assertEquals(divide, calculator.divide(a, b) );
    }

   @DisplayName("Should calculate the correct divide by zero")
   @ParameterizedTest(name = "{index} => a={0}, b={1}, divide={2}")
   @CsvSource({
      "5, 0, 0",
   })
    void divideByZero(int a, int b, int divide) {
     assertThrows(ArithmeticException.class,
         () -> calculator.divide(a , b),
         () -> "divide by zero");
    }

    @DisplayName("Should calculate the correct minuis")
    @ParameterizedTest(name = "{index} => a={0}, b={1}, minuis={2}")
    @CsvSource({
        "5, 3, 2",
        "1, 3, -2",
        "6, 6, 0",
        "2, 3, -1"
    })
    void minuis(int a, int b, int minuis) {
        assertEquals(minuis, calculator.minuis(a, b) );
    }
}

Foretop

Answer 8

I use junitparams, which permits me to pass distinct set of params in each tests. JunitParams uses methods to return set of params, and in the test, you provide the method names as source of param input, so changing the method name will change data set.

import com.xx.xx.xx.Transaction;
import junitparams.JUnitParamsRunner;
import junitparams.Parameters;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;

import javax.validation.ConstraintViolation;
import javax.validation.Validation;
import javax.validation.Validator;
import javax.validation.ValidatorFactory;
import java.util.Arrays;
import java.util.List;
import java.util.Set;

import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;


@RunWith(JUnitParamsRunner.class)
public class IpAddressValidatorTest {

    private Validator validator;

    @Before
    public void setUp() {
        ValidatorFactory factory = Validation.buildDefaultValidatorFactory();
        validator = factory.getValidator();

    }

    public static List<String> goodData() {
        return Arrays.asList(
                "10.10.10.10",
                "127.0.0.1",
                "10.136.182.1",
                "192.168.1.1",
                "192.168.1.1",
                "1.1.1.1",
                "0.0.0.0"
        );
    }

    public static List<String> badData() {
        return Arrays.asList(
                "01.01.01.01",
                "255.255.255.256",
                "127.1",
                "192.168.0.0"
        );
    }

    @Test
    @Parameters(method = "goodData")
    public void ipAddressShouldBeValidated_AndIsValid(String ipAddress) {
        Transaction transaction = new Transaction();
        transaction.setIpAddress(ipAddress);
        Set<ConstraintViolation<Transaction>> violations = validator.validateProperty(transaction, "ipAddress");
        assertTrue(violations.isEmpty());
    }

    @Test
    @Parameters(method = "badData")
    public void ipAddressShouldBeValidated_AndIsNotValid(String ipAddress) {
        Transaction transaction = new Transaction();
        transaction.setIpAddress(ipAddress);
        Set<ConstraintViolation<Transaction>> violations = validator.validateProperty(transaction, "ipAddress");
        assertFalse(violations.isEmpty());
    }


}

Fuel

Answer 9

JUnitParamsRunner is elegant on this

@RunWith(JUnitParamsRunner.class)
public class AccountKeyTest {

public static Object[][] parametersForTestCompareTo() {
    final int id = anInt();
    final int greaterId = anIntGreaterThan(id);
    final BookyearKey bookyearKey = mockBookyearKey();
    return new Object[][] {
        {id, id, null, null, 0},
        {id, greaterId, null, null, -1},
        {greaterId, id, null, null, 1},
        {id, id, bookyearKey, bookyearKey, 0},
        {id, greaterId, bookyearKey, bookyearKey, -1},
        {greaterId, id, bookyearKey, bookyearKey, 1}
    };
}

@Test
@Parameters
public void testCompareTo(final int id1, final int id2, final BookyearKey bookyearKey1, final BookyearKey bookyearKey2, final int expected) {
    // GIVEN
    final AccountKey key1 = new AccountKey(id1, bookyearKey1);
    final AccountKey key2 = new AccountKey(id2, bookyearKey2);

    // WHEN
    final int actual = key1.compareTo(key2);

    // THEN
    assertThat(actual).isEqualTo(expected);
}

public static Object[][] parametersForTestEquals() {
    final int id = anInt();
    final int greaterId = anIntGreaterThan(id);
    final BookyearKey bookyearKey = mockBookyearKey();
    return new Object[][] {
        {id, id, null, null, true},
        {id, greaterId, null, null, false},
        {greaterId, id, null, null, false},
        {id, id, bookyearKey, bookyearKey, true},
        {id, greaterId, bookyearKey, bookyearKey, false},
        {greaterId, id, bookyearKey, bookyearKey, false}
    };
}

@Test
@Parameters
public void testEquals(final int id1, final int id2, final BookyearKey bookyearKey1, final BookyearKey bookyearKey2, final boolean expected) {
    // GIVEN
    final AccountKey key1 = new AccountKey(id1, bookyearKey1);
    final AccountKey key2 = new AccountKey(id2, bookyearKey2);

    // WHEN
    final boolean actual = key1.equals(key2);

    // THEN
    assertThat(actual).isEqualTo(expected);
}

}

Amine