Eclipse Tycho: Testing plug-in's without using single test-bundles

Question

I'm looking for possibilities to test a set of eclipse plugins without having to use a single bundle for each plugin which is tested. Currently I have a PDE build running for an Eclipse product (about 70 plugins, features and a product). All the unit tests for all plugins are contained together in a single plain-java project, having an Eclipse-project reference to all plugins in order to be able to instantiate the classes and run the tests. This setup is no longer working as I converted the PDE build to maven tycho, since the plain java project misses all target platform projects. I'm not executing real OSGI plugin tests, but some tests require having the core eclipse classes like IProgressMonitor contained in the classpath, since I use those eclipse runtime interfaces also in own method signatures.

After setting up the new maven tycho build successfully I tried a few possibilities to get the tests running again:

1) Converting the plain-java test project into a plugin test project
Disadvantages:
- In order to be able to test classes in internal packages, I have to export every single package with x-friend: notation and have to repeat this procedure for each new tested package

2) Adding a second source folder in each plugin and move the tests into the corresponding plugin
Disadvantages:
- Tycho seems to use build.properties to include the necessary source folders for the compile step. Since both the src/main/java and src/test/java needs to be registered as source folder, the real classes and the test classes are mixed up in the target/classes output folder and finally contained in the plugin's JAR file. I did not find a way to configure tycho to use src/main/java as sourceDirectory and src/test/java as testSourceDirectory.
- Tycho executes the unit tests only, if the package type is "eclipse-test-plugin"
- Sonar seems not to recognize tests executed in this way (I did not spend a lot of time trying to solve this issue, maybe there is an easy solution for this point)

3) Add the necessary eclipse target platform plugins as plain-maven dependency to the plain-java test project
Disadvantages:
- The target platform information is duplicated, once in the target platform for the tycho build, and once in the maven dependency list of the test project (executed with plain maven-surefire)
- The target platform bundles are deployed twice in the artifactory, once as target platform p2 archive, and once as maven dependencies (plugin+POM)

4) Adding a test fragment for each plugin (this seems to be the usually chosen possibility)
Disadvantages:
- Needs a huge effort (> 70 plugins, >4500 unit tests), so I would need to add about 70 new fragments and split up all tests.

At the moment, possibility 3) seems the most reasonable for me... any suggestions? otherideas?

Pettit

Just another idea: You could also just merge the 70 bundles together... But if there is a good reason to separate the productive code into bundles, it probably also makes sense to separate the tests. I don't see why tests should be treated separately (other than migration efforts). – Heydon

It definetly would make sense to split the test project into single bundles, but this would need weeks of work which we don't have (and don't get paid for). We decided to use Approach 3, but faced other problems, I updated the question above. Suggestions? – Pettit

Paolo, it's great that you update your post with your findings, but you should put your solution into an answer. Answering your own question is explicitly allowed on stackoverflow. – Heydon

Answer 1

Finally we used approach 3

Unfortunately additionally to the mentioned disadvantage about the target platform jars, we also found out that we need to add each third-party dependency twice. For example the apache-commons-math dependency had to be added once in the productive plugin A (jar in lib folder, and referenced in manifest as bundle-classpath) and once as maven dependency in the test-project POM. We found no other way to get the test project compiling. Basically the test project is not recognizing the jar file included in the Eclipse plugin A, since it is an Eclipse-dependency and not a Maven one. But on the other hand, if I add the library as Maven dependency in the plugin A and remove the jar from the lib folder, the Eclipse IDE is not able to compile the project, since the library is missing (Maven dependencies are not resolved through M2E, if the project has package-type eclipse-plugin.)

Our setup now looks like this (simplified):

Parent POM

• Eclipse plugin A, package type eclipse-plugin, [apache-commons-math in lib folder, added to Manifest Bundle-ClassPath]

• Eclipse plugin B, package type eclipse-plugin

• Test Project, package type jar, Maven Dependency in POM to Plugin A and B and Maven Dependency to apache-commons-math.

Any suggestions?

Pettit