Best practices for Eclipse Feature Integration Testing

Question

I am developing an Eclipse feature consisting of several Eclipse plugins, using Equinox Declarative Services (DS) to wire the plugins together at runtime. I would like to add integration tests to verify the equinox configuration.

In particular, I want to verify that

• the service components bind together as expected
• the bundles are activated
• the plugins share information as expected (see Edit 2)

Furthermore, I want to make this integration testing a part of my continuous integration process using an Eclipse PDE headless build (as described here and here).

My question is: Can you recommend any frameworks, tools, or practices that will facilitate this type of integration testing within the constraints I've identified?

I've found two leads so far:

• Spring Dynamic Modules includes a framework for OSGi integration testing. However, I haven't been able to get a simple Spring DM test to run within Eclipse. It complains that "the platform is already running".
• Pax Exam (formerly Pax Drone) is another OSGi integration testing framework.

Edit: To clarify, each plugin has an OSGi service component configured with a component definition xml file. A mistake in one of these configuration files will not break any plugin dependencies and could easily go unnoticed until runtime. Integration testing is necessary to detect such failure.

Edit 2: So far every thing I've seen seems to confirm Uri's assertion (see below) that multi-plugin Eclipse features aren't integration-tested at the feature/product level. I'm willing to go without comprehensive integration tests if I can at least automatically verify that the service components bind together correctly.

My approach (not working yet):

In a JUnit test do
   For each bundle/plugin of interest
      Get the osgi Bundle object with org.eclipse.core.runtime.Platform.getBundle()
      Verify that the Bundle is active with Bundle.getState()
      Verify that the Bundle is using the expected services with Bundle.getServicesInUse()
      Verify that the Bundle has registered the expected services with Bundle.getRegisteredServices()

I'm running my code with an Eclipse Plug-in Test launch configuration, launching my Eclipse product as the "Program to Run". When the tests run, I can verify that the bundles are active but the service components do not get activated and the getServicesInUse and getRegisteredServices methods return null. I loaded a class from each bundle in case it was a lazy-activation issue, but that didn't help. I also verified that all the service components are "immediate" components, so they should be activated as soon as their bundle's are activated. Why isn't Equinox DS doing its magic?

Answer 1

That's an excellent question.

I'm not familiar with specific tools, but my impression is that most Eclipse Foundation Projects are primarily regression tests (using detailed JUnit suites) without a separate integration testing effort. They rely on the plugin dependency mechanism to alert them if there are problems.

Answer 2

We wrote our own little test execution framework based on a combination of to approaches: a) rcp bundletestcollector (http://rcpquickstart.com/2008/06/12/running-unit-tests-for-rcp-and-osgi-applications/) This is written by Pascal Rapidcault who is one of the main RCP guys. It collects test classes from bundles in a running OSGi environment.

b) the knopflerfish testing framework (http://knopflerfish.org/releases/2.1.1/knopflerfish_osgi_tests_2.1.1.zip) Registers test cases as services that can be executed by a test runner. There is also an XML output, which unfortunately differs a bit from the ant junit XML format.

This way we can execute integration tests living in separate test bundles as well as tests that are closer to classical unit tests and live in fragments to their bundles under test (see http://rcpquickstart.com/2007/06/20/unit-testing-plug-ins-with-fragments/).