JUnit: how to avoid "no runnable methods" in test utils classes

Question

I have switched to JUnit4.4 from JUnit3.8. I run my tests using ant, all my tests run successfully but test utility classes fail with "No runnable methods" error. The pattern I am using is to include all classes with name *Test* under test folder.

I understand that the runner can't find any method annotated with @Test attribute. But they don't contain such annotation because these classes are not tests. Surprisingly when running these tests in eclipse, it doesn't complain about these classes.

In JUnit3.8 it wasn't a problem at all since these utility classes didn't extend TestCase so the runner didn't try to execute them.

I know I can exclude these specific classes in the junit target in ant script. But I don't want to change the build file upon every new utility class I add. I can also rename the classes (but giving good names to classes was always my weakest talent :-) )

Is there any elegant solution for this problem?

Leafy

Does your tests work in Eclipse/NetBeans/your favourite IDE? – Congest

I use eclipse. Actually there is no problem there, somehow eclipse doesn't try to run these classes. I wonder how? – Leafy

I don't know if we understood your question. Please re-read your question and probably add some more information. – Congest

@guerda: The question seems pretty clear to me. His Ant task is finding classes which don't contain tests, because the filter is picking up the utility class. Hence my answer, which I still believe is entirely relevant. – Kidskin

LiorH: Thanks for clarification, so my answer is waste :) – Congest

Answer 1

Assuming you're in control of the pattern used to find test classes, I'd suggest changing it to match *Test rather than *Test*. That way TestHelper won't get matched, but FooTest will.

Kidskin

I don't think it would help, because he moved to JUnit 4.4 and that should not matter. – Congest

You seem to have missed the point of my answer. He has a name filter to determine the classes to be considered as tests. If he changes the filter, he can easily exclude the helper classes. – Kidskin

Your suggestion is valid, however I checked out my tests classes and some start with Test and some end with Test. no clear distinction between utility classes and real test classes. Do you think the convention you suggested is a good practice? (i.e. utils start with Test, and tests end with Test) – Leafy

It's almost a convention that you suffix the testcase classes with *Test. You might need to refactor by renaming test classes appropriately and also rename the helpers so they won't use that suffix convention. – Succuss

I agree with Spoike - if you can't tell from the name of the class whether it's a test or a helper, you should rename the class. The convention is more "the class is a test if and only if it ends with Test." Utility classes may or may not begin with Test - it doesn't matter. – Kidskin

we'll go with the convention. Thanks – Leafy

testPattern { include '**/*Test.java' } - added this but it does not seem to be working, am I missing something? – Accident

Answer 2

Annotate your util classes with @Ignore. This will cause JUnit not to try and run them as tests.

Cavity

Actually, no, it shouldn't. @Ignore is for temporarily disabling tests. – Micturition

Sorry but that's a bad idea. You want to start annotating your production code with test related annotations just because they might match a test pattern? The proper answer is to fix the class names if they are triggering the pattern matching for tests. And make sure the pattern finds only classes that END with Test. That's a commonly accepted pattern – Dragline

Yeah, this is bad and I didn't realize it until after contributing another upvote that i can't remove. Make your base class abstract, then JUnit will ignore it. See @gmoore's answer below. – Germiston

Answer 3

My specific case has the following scenario. Our tests

public class VenueResourceContainerTest extends BaseTixContainerTest

all extend

BaseTixContainerTest

and JUnit was trying to run BaseTixContainerTest. Poor BaseTixContainerTest was just trying to setup the container, setup the client, order some pizza and relax... man.

As mentioned previously, you can annotate the class with

@Ignore

But that caused JUnit to report that test as skipped (as opposed to completely ignored).

Tests run: 4, Failures: 0, Errors: 0, Skipped: 1

That kind of irritated me.

So I made BaseTixContainerTest abstract, and now JUnit truly ignores it.

Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

Motorbus

Much better than @Ignore – Jehad

I tried the @Ignore approach and thought, well that's good, then I read this answer and slapped myself in the forehead, "Of course!" – Uis

Answer 4

To prevent JUnit from instantiating your test base class just make it

public abstract class MyTestBaseClass { ... whatever... }

(@Ignore reports it as ignored which I reserve for temporarily ignored tests.)

Kassandrakassaraba

JUnit runners often try to instantiate abstract classes as well, and then fail with an instantiation error. – Esteban

Works perfectly for my for base test classes – Vichyssoise

This is working because of the name (it doesn't end in Test), not because of the abstract modifier. Change the class name to MyBaseClassTest and it will try to instantiate as mentioned by @HollyCummins (and fail) – Ambivalence

In my case, it should be protected abstract class. – Towhee

Answer 5

1. If this is your base test class for example AbstractTest and all your tests extends this then define this class as abstract
2. If it is Util class then better remove *Test from the class rename it is MyTestUtil or Utils etc.

Outworn

Answer 6

Be careful when using an IDE's code-completion to add the import for @Test.

It has to be import org.junit.Test and not import org.testng.annotations.Test, for example. If you do the latter, you'll get the "no runnable methods" error.

Schizo

This should be a comment rather than an answer. – Schoolfellow

I don't see why. It's a valid solution. – Schizo

Intellij Idea 2017 was messing with my mind by importing org.junit.jupiter.api.Test instead! but thanks to you it is solved now – Untold

Thank you very much, I was confused during getting problem "no runnable methods". – Ridings

Answer 7

Ant now comes with the skipNonTests attribute which was designed to do exactly what you seem to be looking for. No need to change your base classes to abstract or add annotations to them.

Tiemroth

It looks like the skipNonTests attribute is only available in ant 1.9+, which is a shame, since it looks incredibly useful. It will also exclude abstract test superclasses. – Esteban

Answer 8

What about adding an empty test method to these classes?

public void avoidAnnoyingErrorMessageWhenRunningTestsInAnt() {
    assertTrue(true); // do nothing;
}

Scaffold

but that falsely increases the number the tests we have :) not that its a big deal – Bynum

Answer 9

In your test class if wrote import org.junit.jupiter.api.Test; delete it and write import org.junit.Test; In this case it worked me as well.

Aldwon

amazingly, it worked. i executed manually in Windows command line. however, another problem is that @BeforeAll and @AfterAll are not run. – Serrell

seemingly, JUnit4 worked (with @BeforeClass and @AfterClass, but JUnit5's not. Reference: junit.org/junit5/docs/current/user-guide/#migrating-from-junit4 – Serrell

Answer 10

I was also facing a similar issue ("no runnable methods..") on running the simplest of simple piece of code (Using @Test, @Before etc.) and found the solution nowhere. I was using Junit4 and Eclipse SDK version 4.1.2. Resolved my problem by using the latest Eclipse SDK 4.2.2. I hope this helps people who are struggling with a somewhat similar issue.

Alliteration

Answer 11

I also faced the same issue once. In my case I was running my tests using Enclosed Runner of Junit. I created a class called SharedSetup to enable common features for my 2 test classes. But somehow facing the same issue.

@RunWith(Enclosed.class)
public class StructApprovalNodeTest {

abstract static class SharedSetup {

    StructApprovalNode sut;

    ExecutionContext ctx = mock(ExecutionContext.class);
    DTDDAOService dtd = mock(DTDDAOService.class);

    @Rule
    public ExpectedException expectedException = ExpectedException.none();

    @Before
    public void before() throws Exception {
        PowerMockito.mockStatic(ServiceHelper.class);

        when(ServiceHelper.getService("dtd")).thenReturn(dtd);
        when(ctx.getContextInstance()).thenReturn(mock(ContextInstance.class));

        when(dtd.getLatestStructures(Matchers.anyInt(), Matchers.anyString(), Matchers.anyString())).thenReturn(
                new ArrayList<Trade>());

        sut = new StructApprovalNode();
        spy(sut);
    }
}

@RunWith(PowerMockRunner.class)
@PrepareForTest({ ServiceHelper.class, StructApprovalNode.class })
@PowerMockIgnore("javax.management.*")
@PowerMockRunnerDelegate(Parameterized.class)
public static class ParamaterizedBatchTest extends SharedSetup {

    private String batchName;
    private String approvalStatus;

    public ParamaterizedBatchTest(String batchName, String approvalStatus) {
        this.batchName = batchName;
        this.approvalStatus = approvalStatus;
    }

    @Parameterized.Parameters
    public static Collection testValues() {
        return Arrays.asList(new Object[][] {
                { "SDC_HK_AUTOMATION_BATCH", Constants.APRVLSTATUS_APPROVED },
                { "SDC_PB_AUTOMATION_BATCH", Constants.APRVLSTATUS_APPROVED },
                { "SDC_FX_AUTOMATION_BATCH", Constants.APRVLSTATUS_APPROVED }
        });
    }

    @Test
    public void test1_SDCBatchSourceSystems() throws Exception {
        Trade trade = new Trade();
        String tradeXml = FileHelper.getResourceFromJar("/testdata/SDC_BATCH_TRADE_XML.xml");
        trade.setTradeXml(tradeXml);
        trade.setTradeDoc(XmlHelper.createDocument(trade.getTradeXml()));
        trade.setStatus(Constants.STATUS_LIVE);
        trade.setSourceSystem(this.batchName);

        when(ctx.getContextInstance().getVariable("trade")).thenReturn(trade);
        when(ctx.getContextInstance().getTransientVariable("prevTrade")).thenReturn(null);

        sut.execute(ctx);

        PowerMockito.verifyPrivate(sut, times(1)).invoke("resetApprovalDetails", trade);
        Assert.assertEquals(this.approvalStatus, trade.getApprovalStatus());
    }

}

@RunWith(PowerMockRunner.class)
@PrepareForTest({ ServiceHelper.class, StructApprovalNode.class })
@PowerMockIgnore("javax.management.*")
public static class NonParamaterizedBatchTest extends SharedSetup {

    @Test
    public void test2_PrevInvalidTrade() throws Exception {
        expectedException.expect(Exception.class);
        expectedException.expectMessage("External Id of STRUCTURE_TRADE cannot be changed.");

        Trade trade = new Trade();
        trade.setExternalId(123);
        PrevTrade prevTrade = new PrevTrade();
        prevTrade.setExternalId(1234);

        when(ctx.getContextInstance().getVariable("trade")).thenReturn(trade);
        when(ctx.getContextInstance().getTransientVariable("prevTrade")).thenReturn(prevTrade);

        sut.execute(ctx);
    }

    @Test
    public void test3_ValidPrevTrade() throws Exception {
        Trade trade = new Trade();
        String tradeXml = FileHelper.getResourceFromJar("/testdata/SDC_BATCH_TRADE_XML.xml");
        trade.setTradeXml(tradeXml);
        trade.setTradeDoc(XmlHelper.createDocument(trade.getTradeXml()));
        trade.setStatus(Constants.STATUS_LIVE);
        trade.setSourceSystem("BATCH");
        trade.setExternalId(1277402441);

        PrevTrade prevTrade = new PrevTrade();
        prevTrade.setExternalId(1277402441);

        when(ctx.getContextInstance().getVariable("trade")).thenReturn(trade);
        when(ctx.getContextInstance().getTransientVariable("prevTrade")).thenReturn(prevTrade);

        sut.execute(ctx);

        PowerMockito.verifyPrivate(sut, times(1)).invoke("resetApprovalDetails", trade);
        Assert.assertEquals("APPROVED", trade.getApprovalStatus());
    }

    @Test
    public void test4_ValidPrevTradeAutoApprpve() throws Exception {
        Trade trade = new Trade();
        String tradeXml = FileHelper.getResourceFromJar("/testdata/SDC_BATCH_TRADE_XML_AUTO_APPRV.xml");
        trade.setTradeXml(tradeXml);
        trade.setTradeDoc(XmlHelper.createDocument(trade.getTradeXml()));
        trade.setStatus(Constants.STATUS_LIVE);
        trade.setSourceSystem("BATCH");
        trade.setExternalId(1277402441);

        PrevTrade prevTrade = new PrevTrade();
        prevTrade.setExternalId(1277402441);
        prevTrade.setApprovalStatus(Constants.APRVLSTATUS_NOTAPPROVED);

        when(ctx.getContextInstance().getVariable("trade")).thenReturn(trade);
        when(ctx.getContextInstance().getTransientVariable("prevTrade")).thenReturn(prevTrade);

        sut.execute(ctx);

        PowerMockito.verifyPrivate(sut, times(1)).invoke("resetApprovalDetails", trade);
        Assert.assertEquals(prevTrade.getApprovalStatus(), trade.getApprovalStatus());
    }

    @Test
    public void test5_tradeStatusDraft() throws Exception {
        Trade trade = new Trade();
        String tradeXml = FileHelper.getResourceFromJar("/testdata/SDC_BATCH_TRADE_XML.xml");
        trade.setTradeXml(tradeXml);
        trade.setTradeDoc(XmlHelper.createDocument(trade.getTradeXml()));
        trade.setStatus(Constants.STATUS_DRAFT);
        trade.setSourceSystem("BATCH");
        trade.setExternalId(1277402441);

        when(ctx.getContextInstance().getVariable("trade")).thenReturn(trade);
        when(ctx.getContextInstance().getTransientVariable("prevTrade")).thenReturn(null);

        sut.execute(ctx);

        PowerMockito.verifyPrivate(sut, times(1)).invoke("resetApprovalDetails", trade);
        Assert.assertEquals(Constants.APRVLSTATUS_NONE, trade.getApprovalStatus());
    }

}

}

To solve the issue, I just removed the public modifier from the abstract super class SharedSetup and issue is fixed for good

Provinciality