Writing a single unit test for multiple implementations of an interface

Question

I have an interface List whose implementations include Singly Linked List, Doubly, Circular etc. The unit tests I wrote for Singly should do good for most of Doubly as well as Circular and any other new implementation of the interface. So instead of repeating the unit tests for every implementation, does JUnit offer something inbuilt which would let me have one JUnit test and run it against different implementations?

Using JUnit parameterized tests I can supply different implementations like Singly, doubly, circular etc but for each implementation the same object is used to execute all the tests in the class.

Boondoggle

what do you mean "the same object is used to execute all the tests"? – Motherhood

As an former junit addict, I'd just like to say that you should look at groovy/spock. Spock is cool and groovy gives you some abilities you just can't do with junit. One of my favorite things is accessing private data members so you don't have to expose something just to create a proper unit test. – Treasury

@Treasury Why do you want to access private data members? – Heroism

@DávidHorváth I was tasked with increasing our code coverage. I'm trying to avoid refactoring classes under test as much as possible. If they had written the tests, it would be easy to write tests for. :( – Treasury

Answer 1

With JUnit 4.0+ you can use parameterized tests:

• Add @RunWith(value = Parameterized.class) annotation to your test fixture
• Create a public static method returning Collection, annotate it with @Parameters, and put SinglyLinkedList.class, DoublyLinkedList.class, CircularList.class, etc. into that collection
• Add a constructor to your test fixture that takes Class: public MyListTest(Class cl), and store the Class in an instance variable listClass
• In the setUp method or @Before, use List testList = (List)listClass.newInstance();

With the above setup in place, the parameterized runner will make a new instance of your test fixture MyListTest for each subclass that you provide in the @Parameters method, letting you exercise the same test logic for every subclass that you need to test.

Noelyn

Damn! Why did I not think of it. Thanks. – Boondoggle

How about doing List testList = (List)listClass.newInstance(); in the setUp method instead of every test method? – Boondoggle

@Boondoggle Yes, that would work as well. You could also make a helper makeList() method in case some of your test methods must create several instances of the list class. – Noelyn

Works great. Thanks. Plus see this post when your constructor needs parameters. – Microeconomics

One problem with this solution is where the object being passed requires surrounding logic, e.g. to release resources. It's not clear where this code would lie, because there's no equivalent to @After or @AfterClass. Furthermore, I wonder if this is an abuse of @Parameterized - I wonder if it should be used strictly for simple parameters to be passed to the class under test, rather than the being the class itself. I personally do use this approach sometimes, but I think it is a bit limited. – Commode

That was such a great mini-tutorial. Thanks :D – Vigesimal

Answer 2

I'd probably avoid JUnit's parameterized tests (which IMHO are pretty clumsily implemented), and just make an abstract List test class which could be inherited by tests implementations:

public abstract class ListTestBase<T extends List> {

    private T instance;

    protected abstract T createInstance();

    @Before 
    public void setUp() {
        instance = createInstance();
    }

    @Test
    public void testOneThing(){ /* ... */ }

    @Test
    public void testAnotherThing(){ /* ... */ }

}

The different implementations then get their own concrete classes:

class SinglyLinkedListTest extends ListTestBase<SinglyLinkedList> {

    @Override
    protected SinglyLinkedList createInstance(){ 
        return new SinglyLinkedList(); 
    }

}

class DoublyLinkedListTest extends ListTestBase<DoublyLinkedList> {

    @Override
    protected DoublyLinkedList createInstance(){ 
        return new DoublyLinkedList(); 
    }

}

The nice thing about doing it this way (instead of making one test class which tests all implementations) is that if there are some specific corner cases you'd like to test with one implementation, you can just add more tests to the specific test subclass.

Nosepiece

Thanks for the answer, its more elegant than Junit Paramterized tests and I am probably gonna use it. But I have to stick with dasblinkenlight's answer as I was looking for a way out using Junit's Parameterized tests – Boondoggle

You could do it this way, or use parameterized tests and use Assume. If there is a test method which only applies to one (or more) particular class, then you'd have an assume at the beginning of the test, and that test would get ignored for for other classes. – Hued

I think this a base for a great solution. It is important to be able to test all interface methods implemented by an object. But imagine a RepositoryGateway interface with methods like saveUser(user) and findUserById(id), which should be implemented for different databases. For findUserById(id), the testmethod specific setup will need to populate the specific database with the user to be found. For saveUser(user),the assert part should retrieve data from the specific database. Could this be solved by adding a hook (like prepareFindUser) in the testmethod, implemented by the specific testclass? – Imagery

@JopvanRaaij that's one way to do it, but you might as well make it a full-blown integration test instead and use the interface methods to create the test objects. – Nosepiece

instead of a public abstract class for the base test, i would actually use Test interfaces (junit.org/junit5/docs/current/user-guide/…) from junit5, exactly for such purpose – Halter

I have a working example exactly for such purpose using Test interfaces github.com/javieraviles/fibonacci-calculator/tree/master/src/… – Halter

Answer 3

I know this is old, but I learned to do this in a slightly different variation which works nicely wherein you can apply the @Parameter to a field member to inject the values.

It's just a little cleaner in my opinion.

@RunWith(Parameterized.class)
public class MyTest{

    private ThingToTest subject;

    @Parameter
    public Class clazz;

    @Parameters(name = "{index}: Impl Class: {0}")
    public static Collection classes(){
        List<Object[]> implementations = new ArrayList<>();
        implementations.add(new Object[]{ImplementationOne.class});
        implementations.add(new Object[]{ImplementationTwo.class});

        return implementations;
    }

    @Before
    public void setUp() throws Exception {
        subject = (ThingToTest) clazz.getConstructor().newInstance();
    }

Grandiloquence

Answer 4

Based on the anwser of @dasblinkenlight and this anwser I came up with an implementation for my use case that I'd like to share.

I use the ServiceProviderPattern (difference API and SPI) for classes that implement the interface IImporterService. If a new implementation of the interface is developed, only a configuration file in META-INF/services/ needs to be altered to register the implementation.

The file in META-INF/services/ is named after the fully qualified class name of the service interface (IImporterService), e.g.

de.myapp.importer.IImporterService

This file contains a list of casses that implement IImporterService , e.g.

de.myapp.importer.impl.OfficeOpenXMLImporter

The factory class ImporterFactory provides clients with concrete implementations of the interface.

---

The ImporterFactory returns a list of all implementations of the interface, registered via the ServiceProviderPattern. The setUp() method ensures that a new instance is used for each test case.

@RunWith(Parameterized.class)
public class IImporterServiceTest {
    public IImporterService service;

    public IImporterServiceTest(IImporterService service) {
        this.service = service;
    }

    @Parameters
    public static List<IImporterService> instancesToTest() {
        return ImporterFactory.INSTANCE.getImplementations();
    }

    @Before
    public void setUp() throws Exception {
        this.service = this.service.getClass().newInstance();
    }

    @Test
    public void testRead() {
    }
}

The ImporterFactory.INSTANCE.getImplementations() method looks like the following:

public List<IImporterService> getImplementations() {
    return (List<IImporterService>) GenericServiceLoader.INSTANCE.locateAll(IImporterService.class);
}

Redemptioner

Answer 5

You could actually create a helper method in your test class that sets up your test List to be an instance of one of your implementations dependent on an argument. In combination with this you should be able to get the behaviour you want.

Lexicology

Answer 6

Expanding on the first answer, the Parameter aspects of JUnit4 work very well. Here is the actual code I used in a project testing filters. The class is created using a factory function (getPluginIO) and the function getPluginsNamed gets all PluginInfo classes with the name using SezPoz and annotations to allow for new classes to be automatically detected.

@RunWith(value=Parameterized.class)
public class FilterTests {
 @Parameters
 public static Collection<PluginInfo[]> getPlugins() {
    List<PluginInfo> possibleClasses=PluginManager.getPluginsNamed("Filter");
    return wrapCollection(possibleClasses);
 }
 final protected PluginInfo pluginId;
 final IOPlugin CFilter;
 public FilterTests(final PluginInfo pluginToUse) {
    System.out.println("Using Plugin:"+pluginToUse);
    pluginId=pluginToUse; // save plugin settings
    CFilter=PluginManager.getPluginIO(pluginId); // create an instance using the factory
 }
 //.... the tests to run

Note it is important (I personally have no idea why it works this way) to have the collection as a collection of arrays of the actual parameter fed to the constructor, in this case a class called PluginInfo. The wrapCollection static function performs this task.

/**
 * Wrap a collection into a collection of arrays which is useful for parameterization in junit testing
 * @param inCollection input collection
 * @return wrapped collection
 */
public static <T> Collection<T[]> wrapCollection(Collection<T> inCollection) {
    final List<T[]> out=new ArrayList<T[]>();
    for(T curObj : inCollection) {
        T[] arr = (T[])new Object[1];
        arr[0]=curObj;
        out.add(arr);
    }
    return out;
}

Motorize

Answer 7

I had exactly the same problem and here is my approach with help of JUnit parameterized tests (based on @dasblinkenlight's answer).

1. Create a base class for the all test classes:

@RunWith(value = Parameterized.class)
public class ListTestUtil {
    private Class<?> listClass = null;

    public ListTestUtil(Class<?> listClass) {
        this.listClass = listClass;
    }

    /**
     * @return a {@link Collection} with the types of the {@link List} implementations.
     */
    @Parameters
    public static Collection<Class<?>> getTypesData() {
        return List.of(MySinglyLinkedList.class, MyArrayList.class);
    }

    public <T> List<Integer> initList(Object... elements) {
        return initList(Integer.class, elements);
    }

    @SuppressWarnings("unchecked")
    public <T> List<T> initList(Class<T> type, Object... elements) {
        List<T> myList = null;
        try {
            myList = (List<T>) listClass.getDeclaredConstructor().newInstance();
            for (Object el : elements)
                myList.add(type.cast(el));
        } catch (Exception e) {
            e.printStackTrace();
        }
        return myList;
    }
}

2. Classes the contain test cases extend ListTestUtil and you can just use initList(...) wherever you want:

public class AddTest extends ListTestUtil {
    public AddTest(Class<?> cl) {
        super(cl);
    }

    @Test
    public void test1() {
        List<Integer> myList = initList(1, 2, 3);
        // List<Integer> myList = initList(Strng.class, "a", "b", "c");
        ...
        System.out.println(myList.getClass());
    }
}

The output proves that the test is called twice - once for each implementation of the list:

class java.data_structures.list.MySinglyLinkedList
class java.data_structures.list.MyArrayList

Oecology