Running Eclipse Junit Plugin tests with Junit 4.4 or newer -- why aren't tests detected?

Question

I need to use JUnit 4.4 (or newer) in a set of eclipse plugin tests, but I've run into the following problem:

Tests are not detected when running with the junit 4.4 or 4.5 bundles from springsource (junit44 and junit45). The org.junit4 bundle that can be obtained with eclipse supplies junit 4.3 (as of Ganymead / Eclipse 3.4). The org.junit4 bundle does work in that it identifies and runs the tests, but it is not compatible with the latest versions of JMock, and I need to use a mocking library.

Here is a sample test:

package testingplugin;

import static org.junit.Assert.*;
import org.junit.Test;

public class ActivatorTest {
   @Test
   public final void testDoaddTest() {
      fail("Not yet implemented");
   }
}

When running this test, I receive the following exception:

java.lang.Exception: No runnable methods
    at org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:33)
    at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)
    at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
    at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:45)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
    at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:62)
    at org.eclipse.pde.internal.junit.runtime.CoreTestApplication.run(CoreTestApplication.java:23)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:574)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:195)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:382)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1236)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1212)

However, if I switch the project dependencies from com.springsource.org.junit to org.junit4, then the test runs and fails (as expected).

I am running the test as a JUnit Plug-in Test in Eclipse, with the following program arguments:

-os ${target.os} -ws ${target.ws} -arch ${target.arch} -nl ${target.nl}

The following plug-ins selected during launch (selected by me, then I used "add required plugins" to get the rest of the dependencies.):

Workspace:
   testingPlugin
Target Platform:
   com.springsource.org.hamcrest.core (1.1.0)
   com.springsource.org.junit (4.5.0)
   ....and a bunch of others... (nothing related to testing was auto-selected)

Here is my MANIFEST.MF:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: TestingPlugin Plug-in
Bundle-SymbolicName: testingPlugin
Bundle-Version: 1.0.0
Bundle-Activator: testingplugin.Activator
Import-Package: org.osgi.framework;version="1.3.0"
Bundle-RequiredExecutionEnvironment: JavaSE-1.6
Require-Bundle: com.springsource.org.junit;bundle-version="4.5.0"

Switching the last line to:

Require-Bundle: org.junit4;bundle-version="4.3.1"

And updating the selected plugins at launch to:

Workspace:
   testingPlugin
Target Platform:
   org.junit4 (4.3.1)
   ...bunches of auto-selected bundles... (again, nothing else test related)

Causes the test to run properly (but with the wrong version of junit).

Quartana

I have the exact same problem and I can't seem to find any insights on the reason. Apparently, nor does anyone else :( – Lactoscope

For now I've fallen back on junit 4.3 (and an earlier version of jmock). I think the next step is probably to report a bug to Spring Source about their bundles, and/or try creating a new junit 4.4 bundle to see if it's a problem with the bundles, or with junit. – Quartana

How long will this stay at the top of the Unanswered questions queue? – Calve

ah, I (finally) figured out how to make this community editable... I don't have time to evaluate the suggestions below anymore, so if someone can, at least they can put that info into the question proper now (and if needed, I can award the accepted answer--I'm not sure how community questions work wrt. accepted answers) – Quartana

Had the same problem. Adding this line (as indicated above) worked for me: Require-Bundle: org.junit4;bundle-version="4.3.1" – Hiding

Answer 1

In my experience this happens if the plugin which contains the plugin tests does not depend on junit. After adding the junit 4.4 dependency to my MANIFEST.MF file the error went away and all tests were executed. The junit dependency should be optional because the plugin usually only needs it for the test code.

Squalid

Thanks for your answer this is exactly what solved my ptoblem. I struggled with running Junit tests since I expected that Eclipse's "Plugin Junit Test" configuration will add it automatically – Synchronize

Adding org.junit as an optional dependency fixed my "no runnable methods" problem too. – Dud

Answer 2

I cannot test this right now as I don't have an Eclipse 3.4 installation handy, but I've run across a similar problem a while ago in (I think) IntelliJ IDEA 7.0.x, and a workaround was to explicitly specify a test runner.

With JUnit 4.5:

import org.junit.runners.JUnit4;

@RunWith(JUnit4.class)
public class ActivatorTest {
    //...
}

If this does not work you may have more success with org.junit.runners.BlockJUnit4ClassRunner

For JUnit 4.4 I would try org.junit.internal.runners.JUnit4ClassRunner

EDIT : not too sure about the com.springsource. part as I don't use Springsource. From your question it seems that springsource repackages JUnit under com.springsource.org.junit but you use just org.junit in your code so I'll stick with that.

Probable

The repackaging is just to supply junit as an OSGi bundle. com.springsource.org.junit is just the name of the bundle, and has no relation to the actual packages provide by the bundle (other than the convention that you use <organization>.<library> for the naming. – Quartana

I'll try the @RunWith(..) idea next time I have a chance though -- we're just using junit 3.8 for now :(. I have some doubts that changing the test running will work, simply because the tests have to have a very specific plug-in environment to work, and I'm unsure if that includes the test runner. – Quartana

Had the same problem and @RunWith(Junit4.class) solved my problem as well. – Palmira

Answer 3

I'm wondering whether you might need to import the @Test tag from com.springsource.org.junit and not from org.junit.

Volker

Arriaga

Answer 4

I had some similar sounding problems with jMock, JUnit & Eclipse recently, although admittedly not with plugin tests.

I'm not sure if it's relevant, but I got it all working with the following versions :-

• jmock-2.5.1.jar
• hamcrest-core-1.1.jar
• hamcrest-library-1.1.jar
• jmock-junit4-2.5.1.jar

I also found I had to use the JMock test runner like this :-

  import org.junit.Test;
  import org.junit.runner.RunWith;

  import org.jmock.Mockery;
  import org.jmock.Expectations;
  import org.jmock.integration.junit4.JUnit4Mockery;
  import org.jmock.integration.junit4.JMock;

  @RunWith(JMock.class)
  public class PublisherTest {

    Mockery context = new JUnit4Mockery();

    @Test 
    public void oneSubscriberReceivesAMessage() {

Leavy

Answer 5

I don't know which version of JUnit it was, but to succesfully find test the test methods name must start with the word "test".

In newer version you can simply mark test with @Test, for me it works in this combination:

import static junit.framework.Assert.*;
...
@Test
    public void testDummy() throws Exception

Mazarin

It was all JUnit versions prior to JUnit 4. This "old style" still works, though. – Latoyia

...and if you look at my test case, it uses both (@Test and a v. <4 naming convention) – Quartana

Answer 6

Maybe your JUnit bundle is missing an entry in the MANIFEST.MF:

Dynamic-Import-Package: *

This is mandatory to load classes from other bundles.

Bebbo

Single

Answer 7

ActivatorTest needs to extend TestCase

Hamlani

You're absolutely right reppimp, I didn't realize he was using JUnit 4 at the time I posted. – Hamlani

Answer 8

@RunWith(Suite.class) @SuiteClasses( { UserServiceTest.class,ABCServiceTest.class })

public class AllTestSuite {

public static Test suite() {

     return new JUnit4TestAdapter(AllTestSuite .class);
 }

}

Porthole

Answer 9

I think the spring testing framework is not compatible with junit 4.4+

Marsipobranch