Efficient OSGi development workflow

Question

I work on a product composed of many bundles running as features on top of karaf. Typically our developers work on one bundle at a time. Our normal development goes something like: code, compile, copy bundle to deploy folder, test. We've also found that hotdeploy just refuses to override certain bundles that are installed as features without a server restart or a feature uninstall/reinstall, so sometimes the cycle is longer.

My question is: does anyone in the community have a better way? The way we do things works, but I feel like it's pretty slow and inefficient and I'm betting someone has come up with something better!

EDIT: I realize that I was pretty unclear in my question... We are using Equinox underneath Karaf. We also use Eclipse and Maven, although I don't know that using Maven is relevant.

Majordomo

I'm currently in the process of testing these answers. They're all pretty awesome, and I want to see which works the best for me before I accept one. – Majordomo

Sorry for the long wait! I played around with all of these answers and I like dev:watch the best. It required minimal work (which means people will actually use it) and it does what we need. Ultimately, I don't think that I want eclipse to recompile and redeploy every time I hit save... I'd say that about 95% of the time I hit save, my code isn't in a working state. I have to say that I'm enjoying hitting compile and seeing the newly compiled bundle auto-magically running on my server! :) – Majordomo

I agree. That's the approach I've taken too, but with IDEA. It's also encouraged me to make my bundles contain fewer services each so they don't take as long to reload. It's also a good spot-check on whether you have load order dependencies between your bundles! – Gnostic

Answer 1

Sounds like you want the dev:watch command. From the documentation:

The watch command can be used to help at developement time. It allows you to configure a set of URLs that will be monitored. All bundles location matching the given URL will be automatically updated. This avoids the need for manually updating the bundles or even copying the bundle to the system folder if needed. Note that only maven based urls and maven snapshots will actually be updated automatically, so if you run

dev:watch *

It will actually monitor all bundles that have a location matching mvn:* that have '-SNAPSHOT' in their url.

Doing "dev:watch --help" from the Karaf shell will list its available flags and args.

Something similar is the PAX plugin

Either of these will work quite nicely if you've got the m2 maven plugin for Eclipse.

UPDATED: In my company we strive to be as TDD as possible, therefore a lot a development is done without explicitly starting Karaf. In the normal mix of unit tests we're also using Pax Exam, which is largely fantastic even when run from within Eclipse =)

This helps ensure we're not too tided to any Karaf specifics as it runs with Equinox/Felix/Concierge (so I mock out various Karaf specifics we depend on like JAAS authentication). Along with many other cool tools/functionality, it's capable of provisioning Karaf features and using TinyBundles you can even create bundles on the fly (again useful for mocking/stubbing).

Pax Exam hooks into the JUnit framework by providing a JUnit @Runner, the latest version (2) is much faster and has DSL based API, so the tests are quite concise and readable.

Using Pax Exam gives us good test coverage and short development times. Where tests are less practical or we're hunting bugs that don't surface in tests, the dev:watch command is invaluable.

In summary; IMO you should definitely drive your developments with tests (Pax Exam will slot into your existing build nicely and once you get used to it you'll find development quicker). You can start using the dev:watch command immediately, it will certainly speed up your current situation.

UPDATE 2: In answering another question I've added a maven example Pax-Exam testing a ComponentFactory. Test Driven Development is arguably the most efficient workflow available to developers today. link to question: osgi: Using ServiceFactories? and link to sourcecode: http://dl.dropbox.com/u/2465717/net.earcam.example.servicecomponent_2011-08-16_15-52.tgz

Wiser

Specificially for Karaf-Eclipse integration this might be useful: code.google.com/a/eclipselabs.org/p/eik – Wiser

Answer 2

I've had excellent results using Equinox in Eclipse - even hot code replace works properly. granted, the target platform is small and we have only on the order of approx 50 bundles of our own, but workflow goes like this:

First, we have a target platform that contains all third-party and Eclipse bundles, Eclipse takes care of downloading & managing them. Then, the workspace has all the bundles of the project, grouped in 3-4 working sets. Compilation happens as usual on save, sometimes GWT needs to be recompiled, but even then the changes get picked up immediately because no deployment needs to happen - the running Equinox system uses the unpacked project folders as bundles. Running this from within Eclipse gives us hot code replace, on-the-fly changing template files, only MANIFEST.MF/plugin.xml changes need to refresh the bundle - and even then it's usually faster to just restart the framework than to type in the console.

Stroh

The question is about Karaf+Maven not Equinox/P2/EclipsePlugins – Wiser

Answer 3

if you use Eclipse Eclipse Libra may be useful for you. Libra can start Felix, Equinox and Knopflerfish inside Eclipse as any other server with WST. They have some youtube videos how to use it.

I also wrote some tools that can help:

• An osgi bundle that picks up OSGI services that match the filter (osgitest=junit4). With that you do not write Junit classes but you can provide pre-configured objects (e.g. with OSGI Blueprint). JUnit than runs based on the annotations provided in the interface your service implements.
• A maven plugin that has the following useful goals
  • Start a OSGI containers and deploy the bundle maven project with all of it's dependencies (which are OSGI bundles of course). The OSGI container starting is done with the help of PAX Exam but the JUnit tests are started with the help of the OSGI bundle I wrote (that runs the OSGI services you may provide).
  • Create a folder that contains a shortcut to all dependencies of the project (located at the maven repo or target directory of the folder)

If the projects are deployed onto the server (Eclipse Libra) I have to say only update X where the X is the id of the bundle and everything is refreshed rapidly. You do not have to re-compile the projects that are published to the server if you run Equinox in Libra as it points to the target classes folder which is refreshed as soon as you save your class or pom.xml.

If you do not publish your project onto the server but add it as a bundle in the container pointing to the shortcut folder you can also run the update command on the OSGi console after running mvn install (without the restarting of the server).

A step-by-step guide is available at http://cookbook.everit.org/

With the following method above it is possible to write tests as TDD tests and run them as part of a maven compile on the CI server.

I hope you will find these tools as useful as I do!

Motherwort

Answer 4

It depends on the platform under Karaf: Felix or Equinox.

Equinox

Eclipse has excellent (or almost excellent) support for launching Equinox with bundles of your choice. The two things you need to prepare are:

1. Bundles, being developed, available in the workspace as Plug-in projects
2. Target platform, containing the remaining bundles of the application

Such setup will allow you to easily make changes to your bundles, even runtime and easily restarting the runtime when this is required. I see Karaf as more suitable when you are developing on remote system, where the bundles are deployed via SSH or FTP or when you are using external build tools like Maven, which have ability to automatically copy the bundle in the runtime after it is built.

If you are using Equinox, this will give some extra edge over as the runtime will execute the code directly from the workspace.

Felix

Felix doesn't seem to have such support for launching from Eclipse (although there is a work toward this, tracked in this Jira issue). You can also launch it as normal Java application, but this is far from convenient. In this case, using Maven will be much better alternative. You can still setup Eclipse to take full advantage of the PDE other features, only launching will be done externally.

Summary

In summary, you can always automate everything through Maven and Karaf will greatly help you in this regard. Eclipse will give a little edge, if you are using Equinox. You should be able to have hot-code replace regardless of the method you are using, because the hot-code replace doesn't even consider OSGi at all (except in the only case, when you reload your bundle and fresh class loader is created).

Whitney

This isn't quite correct - 1) launching Karaf isn't the same as launching the Equinox/Felix framework that Karaf wraps up (Karaf adds a multitude of functionality), 2) Using Maven plugin for Eclipse will work regardless of whether Karaf's configured to use Equinox/Felix - there's no advantage to using Equinox in this instance. – Wiser

1) It depends on what you are using Karaf for and whether he needs this extra functionality. If you drop this functionality, which Karaf adds, then you'll have better integration with Eclipse if you are using Equinox. In case of Felix, nothing will change much. 2) I didn't mean that Eclipse can help when you have Maven-based workflow and Equinox, I meant that Eclipse-based will be better than Maven-based workflow if the platform is Equinox as Felix is not so integrated in Eclipse. – Whitney

I wasn't bashing you, just trying to clarify wrt Karaf – Wiser