Cleaning up after all JUnit tests without explicit test suite classes declaration

Question

In Intelij and Eclipse IDEs (and probably some others too) it's possible to run all test classes from a package (or even all test classes in a project) without the need to put each of them explicitly in a test suite class (this is something I want to avoid). Just right click -> run all tests and voilà!

I've got one problem with that approach to testing though. I want to do some cleaning up after all the tests are done, but no matter what I do, nothing seems to work.

At first, I tried using RunListener and its testRunFinished() method, but it is called after every atomic test is done, so not what I want when running many of them.

Then I thought about finalizers and runFinalizersOnExit(true), unfortunatelly, it is deprecated and worked only on one of computers that tests are executed on.

Last thing I tried was to create a "listener" thread, that - given tests execution start and end time differences - would clean up, for instance, after five seconds of test completion. I used code below to test that solution:

import org.junit.Test;

public class Main {
    static {
        System.out.println("In a static block!");

        new Thread(new Runnable() {
            public void run() {
                System.out.println("Starting static thread!");

                try {
                    while (true) {
                        Thread.sleep(1000);
                        System.out.println("Static thread working...");
                    }
                } catch (InterruptedException e) {
                    System.err.println("Static thread interrupted!");
                    e.printStackTrace();
                } catch (Exception e) {
                    System.err.println("Static thread catches exception!");
                    e.printStackTrace();
                } finally {
                    System.err.println("Static thread in finally method.");
                    Thread.currentThread().interrupt();
                }

            }
        }).start();

        System.out.println("Exiting static block!");
    }

    @Test
    public void test() throws Exception {
        System.out.println("Running test!");
        Thread.sleep(3000);
        System.out.println("Stopping test!");
    }
}

With no luck. The thread is killed after the test is done. And even the finally block is never executed...

In a static block!
Exiting static block!
Running test!
Starting static thread!
Static thread working...
Static thread working...
Stopping test!
Static thread working...

Desired behavior would be:

1. right click
2. run all tests
3. TestA is running...
4. TestA done
5. TestB is running...
6. TestB done
7. ... more test classes...
8. cleanup

Vanpelt

Answer 1

I've found a solution to my problem. My colleague suggested "hey, can't you just count the test classes?" - and that's what I did.

A little bit of reflection magic is used here, so the code might not be portable:

public abstract class CleaningTestRunner extends BlockJUnit4ClassRunner {
    protected abstract void cleanupAfterAllTestRuns();

    private static long TEST_CLASSES_AMOUNT;
    private static long TEST_RUNS_FINISHED = 0;
    private static boolean CLASSES_COUNTED = false;

    static {
        while (!CLASSES_COUNTED) {
            try {
                Field f = ClassLoader.class.getDeclaredField("classes");
                f.setAccessible(true);

                Vector<Class> classes = (Vector<Class>) f.get(CleaningTestRunner.class.getClassLoader());
                TEST_CLASSES_AMOUNT = 0;
                for (Class<?> klass : classes) {
                    if (klass.isAnnotationPresent(RunWith.class)) {
                        if (CleaningTestRunner.class.isAssignableFrom(klass.getAnnotation(RunWith.class).value())) {
                            for (Method method : klass.getMethods()) {
                                if (method.isAnnotationPresent(Test.class)) {
                                    ++TEST_CLASSES_AMOUNT;
                                    break;
                                }
                            }
                        }
                    }

                }

                CLASSES_COUNTED = true;
            } catch (Exception ignored) {

            }
        }
    }

    public CleaningTestRunner(Class<?> klass) throws InitializationError {
        super(klass);
    }

    @Override
    public void run(RunNotifier notifier) {
        notifier.addListener(new TestCleanupListener());
        super.run(notifier);
    }

    private class TestCleanupListener extends RunListener {
        @Override
        public void testRunFinished(Result result) throws Exception {
            ++TEST_RUNS_FINISHED;

            if (TEST_RUNS_FINISHED == TEST_CLASSES_AMOUNT) {
                cleanupAfterAllTestRuns();
            }
        }
    }
}

Vanpelt

Answer 2

Not sure if I fully have your question right, but I think you want before, beforeClass, after and afterClass methods. i.e.

@BeforeClass
public void beforeClass() {
    // Do stuff before test class is run
}

@Before
public void before() {
    // Do stuff before each test is run
}

@After
public void after() {
    // DO stuff after each test is run
}

@AfterClass
public void afterClass() {
    // DO stuff after test class is run
}

You can do things on a more global level with some hacking or other frameworks. Spring's test suites for example. But I would try to keep such things within the scope of a single test class.

Bullate

Nope, it's still executed after single test class, not after all of them are done. I am running tests in multiple test classes. – Vanpelt

Ok. Then you need to start messing with suites or look into using other frameworks which can handle this. But I'm wondering if perhaps a better approach might be to refactor the tests so that clean up can be done after each test class. You might be trying to work at too larger a scale. – Bullate

Tests are reusing some static objects that require a long initialization. Said objects are only reinitialized on test failure, because otherwise they are in predictible state. Before their deletion, things have to be cleaned up though, otherwise a person who ran the tests have to do it manually. Suites are very inconvinient here, as there are a lot of test classes and most of the time only a handful is run at once, as the overall execution time is really long. – Vanpelt

won't run - @BeforeClass and @AfterClass require methods to be static. – Coverlet