Guillaume Nodet's blog

Unit testing Camel Blueprint routes

2012-01-18T12:32:00.000+01:00

Last week, while in Vegas, I discussed with my colleague Scott Cranton and he told me that something that was really making our users tend to stick with Spring-DM when using Camel Blueprint routes was that unit testing those routes was not possible.

So I hacked a small support library for Camel, leveraging PojoSR which provides a service registry without using a fully compliant OSGi container. This allows defining real unit tests (as opposed to integration tests using Pax Exam.

See the below example.

public class DebugBlueprintTest extends CamelBlueprintTestSupport {

    @Override
    protected Collection<URL> getBlueprintDescriptors() {
        return Collections.singleton(getClass().getResource("camelContext.xml"));
    }

    @Test
    public void testRoute() throws Exception {
        // set mock expectations
        getMockEndpoint("mock:a").expectedMessageCount(1);

        // send a message
        template.sendBody("direct:start", "World");

        // assert mocks
        assertMockEndpointsSatisfied();
    }

}

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">

 <camelcontext autostartup="true" id="camelContext" trace="true" xmlns="http://camel.apache.org/schema/blueprint">

  <route>
   <from uri="direct:start"/>
   <transform>
    <simple>Hello ${body}</simple>
   </transform>
   <to uri="mock:a"/>
  </route>
 </camelcontext>

</blueprint>

Back from Las Vegas

2012-01-17T17:03:00.000+01:00

I'm back from Las Vegas where we had a fantastic FuseSource KickOff meeting.

As you can see below, a real camel had been brought from the west coast and James even had a short ride on it! On the technical side, we expect a bunch of exciting stuff this year, so you'll certainly hear from us soon.

WS-Notification implementation

2011-10-11T18:51:00.000+02:00

Since the early days, Apache ServiceMix has had an implementation of the Oasis WS-Notification specifications. This implementation was built on top of Apache ActiveMQ and available inside the JBI bus as a service. Now that JBI is being put aside in ServiceMix, some questions arose about the future of this component.

So last week, I took a stab at rewriting the component using pure JAX-WS services. For those that are not familiar with WS-Notification, this specification, unlike most other WS-* specifications defines a service described by a WSDL, so there's no low level protocol implementation to write. I took most of the code from ServiceMix implementation and just rewrote the JBI specific bits to leverage JAX-WS instead.

The results have been committed into Apache CXF and should be part of the upcoming 2.5 release. Note that the implementation itself does not require CXF, though it can very well use it as the JAXWS provider.

If you want to give it a try, you need to build CXF trunk locally and then run the following commands in Karaf:

karaf> features:addurl mvn:org.apache.cxf.karaf/apache-cxf/2.5.0-SNAPSHOT/xml/features
karaf> features:install wsn

To check that the WS-Notification broker is correctly started, you can try the following command:

karaf> cat http://0.0.0.0:8182/wsn/NotificationBroker?wsdl
...

The services are also exposed as OSGi services, so that they can be directly accessed if needed. If you use the awesome Fabric DOSGi implementation, it can be very interesting.

Some areas could still be enhanced, as the codebase is mostly the same as the old ServiceMix component. I'm thinking to complex topic expressions or persistence of subscriptions. If you have any real need for something, send an email to the CXF mailing lists ...

Distributed OSGi in Fabric

2011-06-16T09:30:00.000+02:00

The Remote Services OSGi specification describes how services registered in an OSGi framework can be transparently accessed from another OSGi framework. This is in essence a remoting capability for OSGi services. Fabric has a very fast implementation of this specification, leveraging ZooKeeper for the discovery of services.

From the user point of view, registering a service with a property service.exported.interfaces (with a value of ‘*’ or a list of classes to expose) is the only thing to do to make a service available from the outside. The Fabric DOSGi implementation will automatically detect which services have to be imported and will automatically create a proxy in the OSGi registry for those needed. This services to be imported are found through the use of Service Hooks which enable the implementation to be aware of which services are required by existing bundles. For example, if a bundle registers a ServiceListener (directly or indirectly by using Blueprint for example), the Fabric DOSGi implementation will check if there is a local service satisfying the listener and if there’s none, it will look into the ZooKeeper registry and import a matching service.

The Fabric implementation is based on the insanely fast HawtDispatch library which has a very nice support for NIO. The result is a very fast remoting mechanism for OSGi with more than 25.000 requests per second on my laptop. This remoting mechanism is actually not dependant on OSGi and external clients can also connect to a remote OSGi service provided they can find the service identifier from ZooKeeper or some other place.

A complete example can be found in the Fabric source tree at on github.

Provisionning with Fabric

2011-06-14T17:47:00.000+02:00

Already, one month since my last blog entry, but time as been flying with the birth of my son Arthur a few weeks ago. I'd like to talk about the provisioning mechanism of Fabric a bit.

In my last post, I explained how configuration was done through the use of ZooKeeper as the repository for configurations and profiles, propagated to various bundles using the OSGi ConfigurationAdmin service. The provisioning mechanism in Fabric is based on this mechanism: the fabric-agent bundle receives its configuration through the org.fusesource.fabric.agent pid and will automatically update bundles as needed. The nice thing is that the configurations are computed using an overlay mechanism which means that the list of bundles or features to deploy will come from the various profiles a give node has been assigned.

The agent itself supports several kinds of informations:

bundles that need to be installed
karaf features that need to be installed
repositories for karaf features
a url pointing to the osgi framework to use
system properties
osgi framework properties

An example of defining such profiles can be found in the fabric example.

The agent will use those informations and delegate to an internal OSGi resolver to compute the actual list of bundles to deploy. It will then compare this list to the list of bundles already installed and resolve the differences by installing / updating / uninstalling bundles. The agent is able to upgrade all bundles, including core Karaf bundles and even the OSGi framework itself and framework properties.

The profile notion I explained in my previous blog entry allow the use of profiles for provisioning and configuration. For example, a profile can be defined for deploying ActiveM or Camel. And remember one can override profiles, so you can define a profile for your application and change the IP of the ActiveMQ broker in overridden profiles based on the geographic location to connect to the nearest broker.

Due to the configuration bridge between the OSGi Configuration Admin and ZooKeeper, any change made into the zookeeper tree will be reflected to the provisionning agent and the bundles updated immediately. This may or may not be a desired behavior and that's one of the reason why the ZooKeeper tree used by Fabric has a notion of versions. But that'll be explained in a follow up entry.

Configuration with FuseSource Fabric

2011-05-07T10:47:00.005+02:00

From a high level, ZooKeeper can be seen as a replicated tree, and Fabric uses this tree to store the configuration of the various agents (we call Agent, any Karaf instance connected to the same Fabric cluster).

Fabric defines a notion of profile, which is a "kind" of agent. We can define different profiles for various applications we want to deploy in Karaf, for example an ActiveMQ broker or a web server. A given agent can be assigned multiple profiles, allowing a single Karaf instance to serve multiple purposes.

To configure a Karaf instance, Fabric uses the ConfigAdmin OSGi service which is the standard way to push configurations to OSGi applications. An agent will have a bundle deployed which will monitor the ZooKeeper tree and update the configurations according to the profiles it has been assigned. Each bundle will then automatically be notified of the changes through the ConfigAdmin service.

It is often the case that configurations are mostly the same between several nodes, but to one or two parameters that may change. In order to increase sharing between profiles, profiles can inherit other profiles. The actual configurations are computed using an overlay mechanism so that the parent's configurations are overriden by the children ones.

The zookeeper layout for the configurations is setup using the following nodes in the zookeeper tree:

/fabric/configs/agents/{agent}
/fabric/configs/versions/{version}/agents/{agent}
/fabric/configs/versions/{version}/profiles/{profile}

The first node contains the {version} used for a given agent. I'll explain more about versions in a later post. Once we have the version, we can go the the second node which will give the list of profiles assigned to that node. The computation is done by going to the list of parents and retrieving the configurations maps which are organized as following:

 /.../{profile}/{pid}/{key}

A profile can contain any number of such nodes, which gives a map (indexed by the {pid}) of maps (indexed by the {key}) and which values are the content of the node. A special value #deleted# can be used to flag a {pid} or {key} as deleted so that it won't appear in the overlaid configuration. The {pid} will be used as the OSGi configuration pid and the contents will be the various {key}s defined. Note that a {pid}containing a dash will be identified as a factory pid.

After creating creating the ZooKeeper server in Fabric using the fabric:zk-cluster root command, you can experiment with the zookeeper commands and see the changes reflected on agents.

Provisioning is done through this very mechanism. My next post will explain in details how this is done. Stay tuned!

Introducing FuseSource Fabric

2011-05-04T15:11:00.003+02:00

I'm very pleased to introduce FuseSource Fabric, a distributed configuration, management and provisioning system for the products supported by FuseSource: Apache ServiceMix, Apache Camel , Apache ActiveMQ and Apache CXF.

The user's guide contains lots of informations, so I won't rewrite it all again, but I instead invite you to have a quick look at it. The core Fabric is deployed on top of Apache Karaf and heavily reuses the very mature Apache ZooKeeper project.

I'm really excited about Fabric. I've been working on it since Apache Karaf 2.2.0 has been released and I'm glad to share this work. I won't add much more for now, but stay tuned as I plan to write a number of posts explaining the various parts of Fabric.

CamelOne conference

2011-05-03T16:52:00.008+02:00

I just want to point out the great conference FuseSource is sponsoring around open source integration and especially Apache Camel. Unfortunately, I won't be able to attend the event as my wife is expecting our sixth child in the coming weeks :-). That would have been a great opportunity to have my copy of the EIP book signed by Gregor though.

You should definitely go there and listen to all the great speakers. Click on the image below for more informations.

Thoughts about ServiceMix

2010-12-21T18:11:00.002+01:00

I’ve been involved in ServiceMix, the Open Source ESB, nearly since its inception. ServiceMix was started by James Strachan and Rob Davies in May 2005 and the 1.0 version was released in August 2005. I joined LogicBlaze in October 2005 (leaving Mule behind, as I was a committer on Mule) to work on the 2.x releases (2.0 was released in November 2005) and then on the 3.x after the move from Codehaus to the Apache Software Foundation (3.0 was released in May 2007) and the 4.x versions based on OSGi (4.0 was released in March 2009). Even though I’m now focusing more on the OSGi side now (being the VP of Karaf), I’ve done my share of work on ServiceMix (I’m still the first committer in terms of number of commits according to http://svnsearch.org/svnsearch/repos/ASF/search?path=/servicemix) and I’ve been the VP of ServiceMix at the ASF for a few years.

ServiceMix started as a very lightweight implementation of JBI spec. The 2.x version brought much more JBI compliance and the 3.x has seen migration from the lightweight component to fully fledge JBI components and full JBI compliance. In doing so, ServiceMix became a container, as required per the JBI spec and over-time lost a bit of its lightweight-ness. That’s exactly at the same time that Camel was created as a routing library, based on the experience with ServiceMix. After the 3.0 was released it became apparent that the JBI spec was way too limited with respect to the container (classloaders and even the packaging are a real pain in JBI), so we started experimenting with OSGi at that time and that led the path to ServiceMix 4.x and the Karaf project, which started as ServiceMix Kernel.

In 2007, the SCA spec came out, backed by IBM and Oracle, and during a few months, there were a lot of heated debate around JBI versus SCA. It eventually settled a bit when people started understanding that those specs were not really competing, as JBI was targeted around components interoperability while SCA target was application development and could be built on top of JBI. However, JBI was not backed by the big vendors (only really SUN), and when the spec lead for JBI 2.0 left SUN with no replacement, it became clear that the JBI spec was dead.

Another point is that over time, Camel grew to become a top level project and be the very successful project we know, and for some time, we did not really know what to do with the overlap between Camel components and ServiceMix components.

Since JBI 2.0 doesn't appear to be going anywhere we realised we should focus on Camel for the EIP support and connectivity and that OSGi was a better long term standard to represent the registry of endpoints, so we've refactored ServiceMix NMR to be more lightweight, based on the lightweight OSGi container and based around the OSGi registry; with JBI support an optional legacy connector. So we now have a simple lightweight approach to providing a middleware agnostic registry of endpoints with full hot-deploy and supporting powerful class loader versioning schemes via the OSGi registry.

Long live ServiceMix, Camel and OSGi!

Two Karaf related Camel components

2010-09-27T14:40:00.002+02:00

While thinking about a centralized logging system for Karaf and FUSE ESB, I had this idea that instead of using a built-in JMS appender, such as the one provided by Log4j, we could instead easily use Camel for that. Camel is really the best fit for such a thing, as we'd be able to add advanced things such as redelivery, batching, compression and choose easily the transport we want (JMS or plain TCP).

The OSGi EventAdmin service is also an important point for monitoring events in the OSGi runtime, as most of the OSGi services do publish events to it (Blueprint bundles events, bundle events, etc...). So this was another need for a camel component.

Given Camel 2.5 will be released soon, I did not want to destabilize trunk just before the release so I've committed them to a github fork for now.

Those two components are really easy to use:


<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">
    <camelcontext xmlns="http://camel.apache.org/schema/blueprint">
        <route>
            <from uri="paxlogging:camel"/>
            <to uri="stream:out"/>
        </route>
    </camelContext>
</blueprint>

In your Karaf installation, add the osgi:camel as a root appender, and you'll see events printed to the console, though not in a nice way, as the goal is really to send them over the wire.

For the event admin component, things are really easy too:


<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">
    <camelcontext xmlns="http://camel.apache.org/schema/blueprint">
        <route>
            <from uri="eventadmin:*"/>
            <to uri="stream:out"/>
        </route>
    </camelContext>
</blueprint>

The name of the consumer endpoint identifies which topic the endpoint will consume from. This component can also be used as a producer to actually publish events to the Event Admin.

I think those components still miss a bit of configuration, but they seem to be a great way to distribute log events and OSGi events to the outside world in a very powerful configurable way.

As usual, feedback welcome. I think those two components can have some real use case, so I'd like them to be put back into the Camel trunk for 2.6.

Introducing Cade, the Config ADmin Extender

2010-09-22T23:00:00.003+02:00

The OSGi Alliance is working on enhancements to the OSGi ConfigAdmin and some of the experimentation have been unveiled by Peter Kriens in a recent blog. The point is I was in need for enhancing the Apache Karaf blueprint configurations to be more dynamic, for example to restart the SSH server if its configuration had changed.

So I decided to start hacking on this idea and create cade, the Config ADmin Extender. The project is really small and allows you to easily access OSGi configurations in a type-safe manner. It seems to work quite well, so I plan to start using it in Karaf for the various parts that need a bit of dynamism for handling configuration changes.

I haven't done any release yet, but I hope to do that really soon. The source code is Apache Licensed, so feel free to have a look and provide feedback.

RemoteOBR

2010-09-06T19:33:00.007+02:00

Over the last months, I’ve been pondering over the use of OBR, the OSGi Bundle Repository. OBR describes resources (mostly OSGi bundles) in terms of capabilities and requirements. The OBR resolver is able to find the list of bundles needed to fulfill a given bundle requirements. This allows deploying a bundle and its required dependencies without caring too much about the exact dependencies being deployed. In addition, it takes into account the already locally installed bundles and thus reduces unneeded duplication of libraries in different versions.

When I started to use the Felix OBR repository earlier this year, I quickly ran into performance problems during both the parsing phase and the resolution phase. It was nearly unusable for repositories more than a few dozens of resources, which mean in real world scenarii. So I spent some time optimizing things a bit. I also enhanced the API to provide more meaningful informations and to allow more control over the resolution. That work has been released as part of the 1.6.x series of Felix BundleRepository.

One of the enhancement I made was the ability to have a more powerful query mechanism for resources. The goal was to be able to actually use OBR to help people find needed OSGi bundles. This is often one of the main problem people have when building OSGi applications in the integration space. The SpringSource guys have done a good job at providing their Enterprise Bundle Repository and Apache ServiceMix also regularly provide a bunch of OSGi bundles for third party libraries

The main problem with those repositories is that they tend to become huge (more than 7 Mb of xml), which means it takes a long time to download, parse and consumes a lot of memory once it has been parsed. If you use it on multiple JVMs, it becomes quickly a problem and a waste.

That’s when I started to think about the idea of a remote OBR repository. So here we are: I’ve created a small project named RemoteOBR at FuseSource Forge. This project decouples the client API from the real repository manager which is remotely queried using a REST protocol.

It’s not complete yet, but it is already usable, so I invite everyone to go the RemoteOBR website, download, try it and provide feedback.

Apache Karaf

2010-06-17T16:25:00.004+02:00

Yesterday, the Apache Software Foundation established Karaf as a Top Level Project.
It will take some time before the infrastructure resources are correctly set up, but you should see something at http://karaf.apache.org soon.

New version of Apache Felix BundleRepository released!

2010-04-14T01:24:00.002+02:00

Felix BundleRepository 1.6.0 released!

I'm very happy that the Apache Felix Bundlerepository subproject has been released last week. I've been working on it recently and I think it should even have deserved a 2.0.0 release.

This new version provides a new API, very similar to the org.osgi.service.bundlerepository package but which allows more control over the resolution. Performances for both parsing and resolution have been improved by a factory 10, which in my opinion finally move piece of code into a usable state.

At the same time, the Felix Web Console has been released and provides a refactored page for bundle repository browsing (amongst other things). Have a look !

Next steps on this subproject will include the move to the new Felix Framework resolver to be able to support more OSGi constraints such as uses clauses and thus have results that match the OSGi resolver.

I'd also love to have an up to date and correct view of maven central repository using BundleRepository. Combined with the web console which allows querying bundle repository in a powerful way, I think that would help a lot finding OSGi bundles.

Spring-DM, Aries Blueprint and custom namespaces

2010-03-16T18:32:00.001+01:00

Since Spring 2.0, Spring supports custom namespace handlers. This feature is really useful and can be used to reduce the verbosity of your spring based applications. Spring-DM has brought this feature along, so it's also available in an OSGi environment. Unfortunately, the integration of those namespace handlers leads to a lot of problems, mostly due to the legacy spring way of configuring the handlers which does not map well at all in OSGi.

In a standard environment, Spring lives in a simple classloader. The classloader which is used to create the spring application is not supposed to change at any time, and everything that is needed for the application is supposed to be available from that classloader. So let's see what happens with custom namespace handlers. Spring detects custom namespace handlers through the presence of a file in the following location META-INF/spring.handlers. This file contains the name of the class supporting a given namespace. In a non OSGi environment, spring detects the namespace handlers by iterating through the resources and finding all these files. However, in an OSGi environment, things must be done a bit differently. The direction which has been taken by Spring-DM is to watch all started bundles and check for this META-INF/spring.handlers file. The custom namespace is then registered and available for use. This leads to multiple problems.

The first one comes from the fact that there is now way to express a dependency onto a "started" bundle. The OSGi specification defines the Require-Bundle header and Import-Package, but both headers do not care about the fact that the bundle is started or not, only that it is resolved. This means that when you deploy your bundles, you need to control the start order of all your bundles to make sure that all custom namespace handlers are started before any application that use them. Failing to do so will result in applications not being started and the solution is to manually restart the bundle. This really goes against the OSGi best practices.

The second problem comes is caused by the non-staged approach in the parsing of the spring xml configuration files. The Spring-DM extender parses the xml and each time a non standard element is found, try to locate the appropriate namespace handler in its internal registry and delegate to that namespace. Once again, this does not work well in OSGi because namespace handlers can come and go and you can't be sure that the namespace are available at the time your bundle start.

The next problem is related to classloading. The namespace handler does not come with its own classloader than can contribute to the application (remember spring has been designed to work with a single static classloader). This means that in order to use a custom namespace, the classloader of the bundle needs to be able to access all the classes that the custom namespace handler will refer to. If you have ever tried to use CXF in OSGi, you know what I mean: you need to import dozens of packages for your application to start to allow the namespace handler to find the classes it needs.

Another limitation is that custom namespace can't be versioned with Spring-DM and it may lead to having to define different namespaces to solve class-related compatibility issues. Let's say you have a namespace foo. This namespace allows to to create an interface named bar (version 1.0). Later, you have a revision of this package . This new version of the interface may or may not be compatible with the previous 1.0 version, but this has nothing to do with the namespace or xml schema itself. With Spring-DM, you need to version your schema because you can't have two namespace handlers supporting the same namespace. This may be problematic because you end up with two solutions. The first one is to always change your namespace (by including the version in the namespace), which means all the bundles using your namespace need to change their xml configuration even it it was compatible. The other solution is to not allow two versions to live in the same OSGi environment. This is simply not possible unless you fully control everything and this goes against the OSGi very goal.

I really don't blame the Spring-DM guys: I think their goal was to be fully backward compatible and that's what they achieved. Sadly, the end result does not behave well in an OSGi environment.

Now, the nice part is that the Apache Aries Blueprint implementation which I've worked on for quite some time now does a much better job at handling those custom namespaces. All four problems have been solved by leveraging OSGi. The implementation leverages the service registry for custom namespace handlers, each bundle containing a custom handler registering its own service. When starting an application, the blueprint extender will make a first pass on the xml configuration files and detect which namespace are used, then put the application in a waiting state until all namespace handlers are available. Each namespace handler can load its own classes thus avoiding the need for the blueprint bundle to import classes that it's not really aware of (because it uses a custom xml namespace and not plain beans). Last, by ensuring class-space consistency, a much cleaner versioning can be done on the various namespaces.

So give it a try if you hit one of the mentioned issues.

A jira notification system for irc using camel

2009-10-21T16:47:00.005+02:00

Last week, I thought that it would be a nice idea to have JIRA notifications for our Apache projects on IRC (for projects that sometimes use an IRC channel). So I came up with a very simple solution using Camel deployed on top of Karaf.

The application itself consists in a single xml file to be copied in the deploy folder of your Karaf installation. This was one of my requirements for ease of maintenance. This is possible because of the nice hot deployment mechanism for spring xml applications in Karaf.

Now, if you log onto the #camel, #cxf, #servicemix or #activemq IRC channels, you'll see some messages that look like:


jirabot: Claus Ibsen -  Created: (CAMEL-2090) camel-jms - 
   Option autoStartup does not  work 
   (see https://issues.apache.org/activemq/browse/CAMEL-2090 )

Those are all generated by the following applications:


<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans 
       http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://camel.apache.org/schema/spring 
       http://camel.apache.org/schema/spring/camel-spring.xsd
    ">
    
    <manifest>
        DynamicImport-Package=*
    </manifest>

    <camelContext xmlns="http://camel.apache.org/schema/spring">
        <route>
            <from uri="imaps://imap.gmail.com:993?username=xxx&amp;password=xxx&amp;consumer.delay=1000&amp;delete=false" />
            <convertBodyTo type="java.lang.String"/>
            <!-- Remove any cr/lf that would cause regexp to not work -->
            <setHeader headerName="tmpsubject">
                <mvel>
    request.headers.subject.replaceAll("\r\n", " ").replaceAll("\r|\n", "")
                </mvel>
            </setHeader>
            <!-- Extract the user name from the from header -->
            <setHeader headerName="tmpfrom">
                <mvel>
    request.headers.from.replaceAll(".*\"(.*)\".*", "$1").replace("(JIRA)", "")
                </mvel>
            </setHeader>
            <choice>
                <!-- Camel -->
                <when>
                    <mvel>
    request.headers.tmpsubject.matches(".*jira.*(CAMEL-[0-9]+).*")
                    </mvel>
                    <setHeader headerName="irc.target">
                        <constant>#camel</constant>
                    </setHeader>
                    <setHeader headerName="issuelink">
                        <mvel>
    " (see https://issues.apache.org/activemq/browse/" + 
        request.headers.tmpsubject.replaceAll(".*jira.*(CAMEL-[0-9]+).*", "$1") + 
        " )"
                        </mvel>
                    </setHeader>
                </when>
                <!-- other projects filtering using the exact same mechanism -->
                <!-- Other non jira emails -->
                <otherwise>
                    <setBody><mvel>
    request.headers.from + ": " + request.headers.tmpsubject
                    </mvel></setBody>
                    <setHeader headerName="irc.target">
                        <constant>#jirabot.test</constant>
                    </setHeader>
                </otherwise>
            </choice>
            <setBody>
                <mvel>
    request.headers.tmpfrom + "- " + 
        request.headers.tmpsubject.replace("[jira]", "") +
        request.headers.issuelink
                </mvel>
            </setBody>
            <to uri="irc:irc.codehaus.org/?nickname=jirabot&username=jirabot" />
        </route>
    </camelContext>

</beans>

This has been a small and enjoyable exercise. Camel rocks!

Apache SSHD 0.1.0

2009-06-24T15:27:00.003+02:00

Last week, we (the Apache Mina team) released the first version of the SSH library I've began and already blogged about.

The SSHD project provides a server side implementation of the well known SSH protocol. It also provides a client side API which is currently limited and needs improvements (any help welcome!).

Look at the release page for more informations.

OSGi Blueprint Services

2009-06-24T15:04:00.004+02:00

I gave a presentation on Monday at the OSGi DevCon in Zurich about Blueprint services.
Blueprint Services is the OSGi standardized version of Spring-DM which I've been working on (both the spec and the implementation over the last weeks).
Below is the presentation. The spec is not published yet, but the implementation code is currently available at:
http://svn.apache.org/repos/asf/geronimo/sandbox/blueprint
This implementation is now integrated into Apache Felix Karaf, a ready to use distribution of OSGi for enterprise / server side applications.

OSGi Blueprint Services

View more Microsoft Word documents from gnodet.

Apache Karaf

2009-04-02T13:49:00.006+02:00

ApacheCon was really interesting this year! Recently, a lot of people have expressed a real interest in ServiceMix Kernel, our generic OSGi distribution for server side applications. We've been discussing moving this subproject into Apache Felix for several reasons:

raise the visibility and awareness on ServiceMix Kernel
attract a broader community

Several Apache projects are planning to use ServiceMix Kernel as their container: this includes Apache James, Apache Directory and Apache ActiveMQ. The Apache Sling community is also willing to contribute to this effort along with some other groups like the OPS4J project.

During this discussion, a name as been proposed by Jamie Goodyear: Apache Karaf.

A carafe is a small container used for serving wine and other drinks (http://en.wikipedia.org/wiki/Carafe). In similarity to the name the Kernel allows applications to be more easily handled, and improves their characteristics (much like a bottle of wine left to breath in a decanter) :)

I have to thanks Jamie for this name, I really like it. Plus the K might eventually fill a hole in the list of Apache projects and also reminds the Kernel.

Moving into Felix does not mean that Karaf will be tied to the Felix OSGi runtime. We haven't really worked on that yet, but the goal is to be able to use either Felix or Equinox runtimes inside Karaf. Maybe we'll provide two different distributions in the future...

Today, both ServiceMix and Felix community have voted and accepted this move. I think this is a great news for ServiceMix Kernel users, for the Apache communities and for all OSGi users and I'm really excited about that...

New clustering engine in ServiceMix 4

2009-04-02T13:46:00.002+02:00

I have expressed several times on the mailing list that the JMS flow included in ServiceMix 3 has some deficiencies. Learning from that, ServiceMix 4 has a brand new clustering engine that Adrian and I have discussed a bit at ApacheCon. He blogged about that recently, so have a look at his blog post.

Apache ServiceMix 4.0.0 and Apache ServiceMix NMR 1.0.0

2009-04-02T13:37:00.002+02:00

During ApacheCon last week, we're released Apache ServiceMix 4.0.0, the first release of our OSGi-based integration platform.
We've also released Apache ServiceMix NMR 1.0.0 which contains the NMR layer, the JBI layer. The main distribution is a minimal JBI container with no components.
Have a look at those, we welcome your feedback!

Whitepaper: Large scale deployments with ServiceMix 4

2009-04-02T09:58:00.002+02:00

At ApacheCon Europe last week, Andreas Gies, a coworker, has given a nice presentation about large scale deployments with ServiceMix 4. It has been now published as a whitepaper and is available on the FUSE site.

Apache ServiceMix Kernel 1.1.0 released

2009-03-18T09:50:00.002+01:00

The new version of Apache ServiceMix Kernel has just been released.

This new release brings a lot of new features, enhancements and bug fixes:
* remote connection using SSH protocol
* provisioning enhancements: versioning / hot deployment of features
* new commands, including OSGi related commands for the Configuration
Admin and Package Admin services
* improved spring integration: upgrade to spring 2.5.6 and spring-dm
1.2.0-m2, the osgi/list command now displays spring applications status
* container level locking for master / slave deployments
* support for JAXP 1.4 on all platforms
* improved JMX support for managing the OSGi framework and features

You'll find download links and details release notes at the following location.

SSHD becomes an Apache Mina subproject

2008-12-05T09:00:00.003+01:00

Just an update on the SSHD project I started a few weeks ago. It has been accepted yesterday as a MINA subproject :-)
You can find all the needed pointers on the web site.

ServiceMix4: much more than a JBI container

2008-11-25T13:23:00.004+01:00

David Greco has posted a nice blog entry about ServiceMix Kernel.
I do totally agree: ServiceMix 4 is not only a JBI container, and more precisely the ServiceMix Kernel which can be run standalone is a universal container (which does not include JBI by default) where you can deploy virtually any kind of artifact from web applications, to JBI applications or plain JAX-WS services.

SSH Server in Java

2008-11-13T10:19:00.003+01:00

ServiceMix Kernel is a small container based on OSGi. The latest release allows external clients to connect to it and issue commands using a simple protocol implemented on top of TCP or SSL. However, this remoting protocol has some drawbacks as the internals makes it unable to do another remote login from a remote session. In addition to that, completion and history do not really work great.

So I've been thinking about using the SSH protocol, which is widely used, secured, with tons of different clients available. Unfortunately, no SSH server is available in Java. Over the past weeks, I've been working on implementing this SSH server, based on the IEFT specifications, the JSch SSH client library, and the OpenSSH server source code. The server itself is based on Apache Mina which is a great framework for using NIO.

The project is available at http://code.google.com/p/sshd/ and although there are lots of limitations right now, the basics of the SSH protocol work. I plan to integrate it in a bit into ServiceMix Kernel as the default remoting protocol. Maybe over time, we'll be able to implement SFTP or even a full SSH server integrated with Unix OS (I've already made some tests, but I'm missing a few bits on the PAM integration for authenticating against the OS).

Anyway, if anyone want to contribute or have any problems trying to use it, just send me an email. I may set up a google group list for that later if there is any need.

Update: the code is now developed as a subproject of Apache Mina and available at http://mina.apache.org/sshd/

FUSE ESB 4 release!

2008-10-15T23:03:00.002+02:00

FUSE ESB 4 has just been released. I welcome everybody to download and give it a try.

Apache ServiceMix Kernel 1.0.0 released!

2008-09-18T14:15:00.006+02:00

Apache ServiceMix Kernel 1.0.0 has just been released.

Apache ServiceMix Kernel is a small OSGi based runtime which provides
a lightweight container onto which various bundles can be deployed.
Amongst the list of supported features, Apache ServiceMix Kernel supports:

hot deployment of OSGi bundles, exploded bundles or custom artifacts (spring xml configuration files support is provided)
services configuration stored as property files are monitored and provided as standard OSGi configurations
a centralized logging back end supported by Log4J, ServiceMix Kernel supports a number of different APIs (JDK 1.4, JCL, SLF4J, Avalon, Tomcat, OSGi)
provisioning of libraries or applications can be done using simple commands via simple xml descriptors
native OS integration as a service so that the lifecycle will be bound to your Operating System.
an extensible shell console to manage services, applications and libraries
operations on the console can be done remotely via a secured and encrypted channel
a security framework based on JAAS
new instances can be created using a single command line

This release, with the detailed release notes, is available at:
http://servicemix.apache.org/kernel/servicemix-kernel-100.html.

IONA FUSE Open Source Group now offically part of Progress

2008-09-18T14:15:00.005+02:00

I'll just quote Rob's blog entry:

Progress completed its acquisition of IONA Technologies last Friday. The FUSE open source group will be running as its own business unit as part of the Progress Software Corporation - so we are still operating as before - but with considerably more investment.

Progress saw the number and quality of enterprise customers we have using our integration products, saw that we consistently met and exceeded our sales targets and realized our potential to grow significantly.

So although we currently employee a lot of open source developers - we will be recruiting even more - and have some exciting new software projects in development for release in 2009 :)

Bundling third party dependencies in ServiceMix 4

2008-09-09T10:17:00.002+02:00

I'd like to talk a bit about third party dependencies in ServiceMix 4.

ServiceMix 4 is based on OSGi and when deploying applications to it, you have two choices:

use the JBI packaging
use the OSGi packaging

The first solution is a good thing if you want to abide by the JBI specification completely or if you want to use non OSGi aware components. However, you'll come into the known limitations of the JBI packaging with respect to classloaders: the JBI specification is quite limited in this area.
The second solution is more powerful as, in addition to being able to leverage the OSGi classloader mechanisms, you'll also be able to access the OSGi registry, thus registering or retrieving services in the registry.

However, when using the OSGi packaging, you need to deal with third party dependencies. OSGi is becoming more and more popular, so more and more projects are now distributing the jars as native OSGi bundles. But not all have converted to OSGi yet. And of course, you also need to transform your own jars into OSGi bundles.

Costin Leau has covered most of the aspects of creating OSGi bundles in his blog post. But there are a few tweaks that I want to talk about.

When repackaging a library as an OSGi bundle, one as to take care about package imports. I had a question recently about a problem when repackaging c3p0. This library is a JDBC connection pool, so it needs the class for the JDBC driver to be available in its classpath. A simplistic way to handle the problem is to force an optional import of the package containing the driver you use. This would be something like:

   Import-Package: *, com.mysql.jdbc;resolution:=optional

The drawback is that you tie your mysql bundle to the driver you use, which will break things if you want to switch to another database.
OSGi has a solution for such things which is called Dynamic Imports. This is needed when a library uses the Class.forName() method to load classes dynamically, such as the c3p0 library. The big difference between static imports and dynamic imports is that dynamic imports are resolved ... dynamically. This means that when the Class.forName() method is called, the OSGi framework will look for the requested package and wire it to your bundle at runtime. The OSGi headers would look like:

   Import-Package: *
   DynamicImport-Package: *

But this feature must be use with caution, and only when possible to avoid having unwanted wiring between bundles.

A second use case if the legal problems encountered while dealing with proprietary software of non liberal open source licenses. IANAL, so take the following with the appropriate caution and check with your legal department before doing anything. Back to the point: proprietary software often forbid any modification to the jars. In such a case, the only alternative is to embed the jar in an OSGi bundle. Basically, when you repackage a library into an OSGi bundle, you extract the content of the jar, add some OSGi headers to the manifest, and rebuild the jar. There is an alternative though, which is to create a bundle and include the full jar inside it. The bnd tool and felix bundle plugin for maven (which uses bnd) both support that and you end up with using the Bundle-Classpath header:

   Import-Package: *
   Bundle-Classpath: .,bin/mysql-connector-java-5.1.6.jar

The only drawback is that the bundle jar can not be used anymore in a non-OSGi environment.

Before bundling a jar, check the existing bundle repositories (the ServiceMix one or the SpringSource one) for an existing bundle. And if you write one for an open source library, feel free to submit a patch if you want ServiceMix to host it...

Happy bundling!

A fun physic game

2008-09-09T00:46:00.002+02:00

I found Fantastic Contraption a small, but very fun game to play. Here is my solution to the last free level. Enjoy!

What's new in ServiceMix 4.x?

2008-09-08T17:41:00.001+02:00

Rod Biresch has posted a nice blog post about what's new in ServiceMix 4.x. Definitely worth a read!

Is JBI so bad?

2008-07-29T10:40:00.003+02:00

Ross Mason, the CTO of MuleSource, recently started a discussion about JBI.

His first point is that JBI is no TCP/IP, meaning somehow that JBI is targeted at vendors and not at developers. Well, I'm not sure about the TCP/IP thingy, but he's a bit right about JBI being targeted at vendors and not developers. Why? If you have some knowledge about JBI, you know that there are multiple JBI components (bindings that handle a particular protocol such as HTML, JMS, etc... and engines that provides business logic such as a rules engine, a BPEL engine and so on). The developer will mostly see those components, not the JBI APIs: if you work with a BPEL engine and write a process, you won't see the JBI APIs surface in the BPEL at all. If you write a WSDL to define a SOAP/HTTP service, you won't really see JBI there either. So, yes, at some point, he is right that JBI 1.0 hasn't focused on the developer, or I should say, the user. Because if you need a custom component for your JBI container, you, as a developer, will need to dive into JBI and fully understand it. At this point, if you don't use JBI, you'd have to dive into a proprietary set of APIs to write your component. JBI aims to be mostly hidden for the users.

His second point is about some of the restrictions in JBI 1.0, mainly the use of XML, WSDL and no support for streaming. About XML, JBI has ways to convey binary data in non XML formats using attachments (don't you use attachments in your emails?). About WSDL, this point seems a bit weird since MuleSource has a product named Galaxy which is a registry of services described by WSDLs. Everyone knows WSDL is the standard for describing services. Anyway, the JBI specification says each endpoint has to be described by a WSDL, however, since the early days of ServiceMix, WSDL has not been a requirement and you can deploy your endpoints without any WSDL at all. With respect to data streaming, I'm not sure to understand the supposed flaw of JBI here: in the JBI world, a message contains an XML payload (which can be a stream, a DOM document or any kind of XML representation) and a set of attachments (which can also be streams). Just leveraging these APIs, JBI components are able to transfer very large amount of data using streams only.

His third point is about component reuse in JBI. We have users using ServiceMix components inside OpenESB and others using OpenESB components inside ServiceMix. It's true that most of the JBI vendors offer their own set of components. Let me ask the question: doesn't Mule come with its own set of transports? It would not make sense at all to distribute an ESB without any support for the most commonly used transports. It does not mean that these components can not work together.

Last comes some points about the existing standards. Since a long time, I don't think SCA should be viewed as a competitor to JBI. For some reasons exposed above, JBI does not really target the end users, while SCA does. However, SCA does not allow components to interoperate together. Hence, those two standards are complementary, not competitors.

All in all, I think JBI is a good specification. It does not address all the possible use cases in the best way (I'm thinking about handling non XML data) and sometimes goes beyong what should have been required (on the packaging and deployment side), but the core messaging APIs are well defined, and given the success of ServiceMix, I'm far from thinking that JBI should be dismissed.

Java EE specs in OSGi

2008-05-26T11:39:00.005+02:00

For ServiceMix 4, I've been working on making sure the Java EE specifications can be used in OSGi. The first step was to release OSGi versions of the various specifications by just adding the needed manifest entries to make them usable in OSGi. This was done inside the Geronimo project (on which I am a committer). This means that since a few months, most of the Java EE specification jars are available as OSGi bundles (you can grab those from maven 2 public repositories.

However, this is not always sufficient. Some of these specifications (mostly JAXB, SAAJ, JAX-WS and Stax) do not work well in OSGi. The mean reason is that the implementation is discovered using different mechanisms by the API. The most commonly used one is to find a file on the classpath in the META-INF/services directory and find the main entry point class of the implementation. Unfortunately, to make this work in OSGi, the client bundle (the one using one of these APIs) has to have the required file in its classpath, which means the inability to use one provided by the runtime in which the bundle is deployed and that it can not be switched without changing and re-compiling the bundle. Another way would be to add a Require-Bundle OSGi manifest so that the classpath of the implementation becomes part of the client bundle, but this also ties the client bundle to the implementation used.

The solution came to me after a chat with Dan Kulp: an OSGi specific discovery mechanism can be easily plugged into these spec jars. It consists in two small classes shared amongst these spec jars: an OSGi bundle activator and another class not tied to the OSGi api that maintain a list of available implementations. The final step if to rewrite the factory of those jars to look inside this map before performing the usual lookup.

This means that in a non OSGi environment, the jar behaves as usual, but when deployed into an OSGi runtime such as ServiceMix Kernel, the spec bundle will be able to locate dynamically the implementation to use. Therefore, the client bundle using the spec jar is now free of any requirements.

JAAS in OSGi

2008-05-21T21:08:00.004+02:00

I've working on implementing the security framework for ServiceMix 4. ServiceMix 3 used JAAS for the authentication part, and it also makes sense to use it in ServiceMix 4 for several reasons: reuse of existing login modules, integration with the JMX and the console security which are already based on JAAS.
However JAAS is not very OSGi friendly (well, most of the JEE specifications are not, and I'll talk about the others in another post), mostly because is makes some strong assumptions upon the thread context classloader, and this, mainly on the client code. This means the client that uses the JAAS api to authenticate has to have all the login modules available in its thread context classloader. This is usually not the case in OSGi.
So the solution is to use a proxy login module that will be available to all bundles (by using the boot framework delegation package). This proxy login module can use some OSGi properties on the login module configuration to determine the actual class to use and the bundle to load it from.
Using a simple XML schema for Spring, you can deploy a JAAS realm very easily:


<jaas:config id="realm" xmlns:jaas="http://servicemix.apache.org/jaas">
    <jaas:module className="org.apache.servicemix.kernel.jaas.config.SimpleLoginModule" flags="required">
        key=value
    </jaas:module>
</jaas:config>

This will register a service in OSGi that the OSGi specific Configuration for JAAS will discover and make it available for clients.

Find more informations on the ServiceMix Kernel JAAS doc.

ServiceMix 4 NMR on Equinox

2008-05-15T17:02:00.003+02:00

I've done some experiments today to check that ServiceMix 4 NMR can be easily deployed on Equinox instead of Felix. Have a look at this wiki page.

Apache ServiceMix Kernel 1.0-m3

2008-05-13T21:53:00.004+02:00

We've just released the third milestone of ServiceMix Kernel 1.0-m3. This small OSGi based container is really nice, if you haven't had a look at it yet, go and grab it.

It adds a bunch of cool new features. For example you can run:


osgi list | utils grep ServiceMix


log d | utils grep WARN

If you want to have a quick run at it, go and look at the quick start guide.

Spring App Server

2008-04-30T11:31:00.002+02:00

I've just seen the annoucement that SpringSource has launched a new App Server based on OSGi (Spring-DM).
I find it a bit weird that they tend to rewrite existing stuff: the JMS JCA layer in Spring 2.x is somehow derived from the Jencks project, Spring Integration looks a lot like Camel, and now this app server which looks very similar to ServiceMix Kernel with WAR support (which we already had in ServiceMix 4 from the Pax Web project).

Ziphone

2008-04-25T14:47:00.003+02:00

A friend of mine has cracked an iPhone a few months ago (shame on him...) but was quite reluctant to upgrade to a newer version in case the phone would be bricked.
Fortunately, I gave him a link to ZiPhone, which is the best way to unlock your iPhone I've found so far: just upgrade your phone using iTunes, launch the application and click. 5 minutes later, you have a jailbroken and unlocked iPhone with the latest firmware :-)

Dan has also blogged about Fring, a very nice application to to VoIP over the iPhone.

If you like to twitter, have a look at Twinkle too.

ApacheCon EU 2008

2008-04-04T17:27:00.003+02:00

Bruce and I will be both at ApacheCon Europe in Amsterdam next week to talk about ServiceMix and Camel, so if you want to meet with us, send us a mail.
There will be a BOF about ServiceMix on Wednesday, so it will be a good opportunity to meet too.

JBI and SCA (again)

2008-04-04T15:50:00.002+02:00

I've just seen this post saying:

One of the major motivations for doing so is that currently, the Java community is still split between SCA and JBI.

Well, this is somewhat true, I can't really deny it, but it should not be that way. JBI and SCA are not really competitors, even if they are usually seen that way. The JBI specification explains how to create an ESB that allows pluggable components to be wired easily thus allowing vendor independant components to be created. SCA defines a way to write composite applications in a clearly defined way that can be applied to different languages (though you can't really mix those, so you end up with one runtime per language you support).

As a standard, SCA offers value, but I really think both could be supported in a single runtime like FUSE ESB. We've discussed several times about supporting SCA on top of JBI in ServiceMix, but I haven't heard much needs on that. We are currently working hard on ServiceMix 4, based on OSGi and allowing lots of technologies to be plugged onto a single bus, including JBI, Camel, JAX-WS, EJB3, etc... Supporting SCA would only be another profile.

ServiceMix / Fuse ESB press release

2007-12-06T16:52:00.000+01:00

A press release has been published recently on the use of Fuse ESB for the Belgium Government. For french speaking people, you may find lots of informations on the project by browsing Lilian Duchêne's blog.
Together with Anne Noseda, we have presented this projet at the BeJUG SOA conference. You can view the slides or download them.

Apache ServiceMix 3.1.2

2007-09-25T09:21:00.000+02:00

Apache ServiceMix 3.1.2 has been released. Release notes are available here.

This is our first official release since graduation: this means that the artifacts are available from the public repositories now (see http://repo1.maven.org/maven2/org/apache/servicemix/).

ServiceMix has graduated

2007-09-19T20:08:00.000+02:00

I'm really pleased to announce that the ASF Board has approved Apache ServiceMix graduation :-) This means that ServiceMix now becomes a full Apache Top Level Project. So we will soon make our first official release of ServiceMix 3.1.2 and 3.2 coming later. This also means that the resource will be moved to their final location: the website will be moved to http://servicemix.apache.org/ and the svn repo to https://svn.apache.org/repos/asf/servicemix/.

In addition, I'm honored to have been voted as the Apache ServiceMix PMC Chair. The PMC (Project Management Commitee) is the group of people responsible for the project oversight, and its Chair is the interface between the Board and the project.

I just want to remind people the purpose of the Apache Incubator: its main responsibility is to watch new projects at the ASF to make sure that no IP problems remains and that there is a vibrant, diverse and sustainable community around the project that goes the "Apache Way". This is in no case about the quality or maturity of the software itself, as incubating projects greatly vary in these areas (from brand new projects to very mature projects joining the ASF).

FUSE ESB

2007-08-30T14:39:00.000+02:00

My company IONA provides support on Apache ServiceMix via a distribution called FUSE ESB available from the Open Source IONA site.

What's in FUSE ESB right now? Well, it's roughly a distribution of ServiceMix trunk. This implies that FUSE ESB is released ahead of the Apache ServiceMix distribution (the latest release is 3.1.1, whereas FUSE ESB is based on 3.2).

Why are you using the latest trunk instead of the most stable branch? Well, mostly because our customers needs some of the latest features available. We take great care of what is in our Fuse branch: we do not necessarily backport all new stuff from trunk. We may also add some specific customer needs inside our own distribution, features that are not present in the trunk version.

So is that a fork? Certainly not :-) We have always supported Apache ServiceMix and we will continue to do so. But our customers have specific needs, so we may need to do custom development or have custom branches for them to fullfill these needs. Over time, we put the features that we consider generic enough back to the community. But this level of support can not be provided in a community driven environment such as the Apache Software Foundation. Let alone the fact that our customers often require privacy regarding their issues or their specific needs, environments and projects, so that it becomes difficult to use the open mailing lists of the community.

So is that a closed source version? No. The Apache License would allow that (quite the opposite of the GPL), but this is not what we aim for. Our process is quite open: as an example, I've recently developed an AsyncBridge EIP pattern for FUSE ESB (see the issue) that will certainly be contributed back to the community...

Hopefully, this blog entry will clarify a bit the IONA policy with respect to the Apache projects it supports and the relationship between FUSE ESB and ServiceMix. Feel free to download it and give it a try...

Switching to Mac

2007-08-13T23:04:00.000+02:00

My ThinkPad has some serious cooling problems (it keeps shutting down when I really use it, which is kinda annoying, because this is when you really use it that you want it up and running of course) so I decided to go for a MacBook Pro. Why? Well, I'm a long time users of Windows, but for developing, I usually use cygwin for command line and all the unix good stuff. Additionally, all my team mates are Mac users, so I was quite sure I would not loose much. Now, I must admit I don't regret. Thanks to James for having pointed a few useful softwares...

ServiceMix 3.1.1 released

2007-07-04T14:06:00.000+02:00

Apache ServiceMix 3.1.1 release is finally out! This release is a bug fix version with a few minor improvements.

Accessing databases in servicemix-drools

2007-06-28T14:00:00.001+02:00

ServiceMix provides a Service Engine for Drools, the famous Rules Engine.

People often want to retrieve data from the rules and such data is usually stored in a database. Previously there was no easy way to configure a DataSource and inject it in the rules definitions. This is a small enhancement that I've just written and that will be included in next major release.

So let's say you write your database access code in a simple helper object:

import javax.sql.DataSource;
import org.springframework.jdbc.core.JdbcTemplate;

public class DbHelper {
    private DataSource dataSource;
    private JdbcTemplate jdbcTemplate;

    public void setDataSource(DataSource dataSource) {
      this.dataSource = dataSource;
      this.jdbcTemplate = new JdbcTemplate(dataSource);
      this.jdbcTemplate.afterPropertiesSet();
    }

    public String getSurname(String name) {
        String surname = (String) this.jdbcTemplate
          .queryForObject(
              "select surname from t_actor where name = ?", 
              new Object[]{name}, String.class);
    }
}

Now, let's use it in our rule definition:

import org.apache.servicemix.drools.model.Exchange;
global org.apache.servicemix.drools.model.JbiHelper jbi;
global DbHelper helper;

rule Init
    when
        $me : Exchange()
    then
        String t = $me.getIn().getContent().getTextContent();
        jbi.answer("<surname>"
            + helper.getSurname(t)
            + "</surname>");
end

And the xbean.xml would look like:

<drools:endpoint  service="test:service"
                  endpoint="endpoint"
                  ruleBaseResource="classpath:router.drl"
                  globals="#globals" />
<util:map id="globals">
  <entry key="helper" value-ref="helper" />
</util:map>
<bean id="helper" class="DbHelper">
  <property name="dataSource">
    <bean class="org.springframework.jndi.JndiObjectFactoryBean">
      <property name="jndiName" value="java:myDataBase" /> 
    </bean>
  </property>
</bean>

This will make available the helper bean to the rules while initializing it with a DataSource looked up from JNDI.

Happy hacking !

Loose coupling in JBI

2007-06-22T08:54:00.001+02:00

Loose coupling is a feature that can be easily achieved in JBI but which is sometimes not well understood by newbies in the JBI world.

In JBI, services are exposed inside the bus by JBI components when you deploy a given service onto it. These components can be Service Engines (contain business logic) or Binding Components (handling a specific protocol). The distinction is the key for loose coupling: the service itself it decoupled from the protocols used to access it. This is true for all services accessed from inside the JBI bus, be it internal to the bus or external to it.

Let's take an example. You need to write a BPEL process and expose it over SOAP/HTTP, and this process will consume several other services. These services can be inside the JBI bus, accessible via SOAP/HTTP or plain JMS. The important point is that the BPEL process has no knowledge of the protocol and location of these services. How does it work ?

The key is the WSDL. A WSDL can be split into two parts: the abstract and the concrete model. The abstract model contains the interface (Port) definition along with the messages and xml structures definition. The concrete part contains the binding (tying the service to a particular protocol) and service definition (location and binding for a given service). When JBI components talk together, they only use the abstract WSDL definition which does not contain any protocol or location information. The NMR will select a JBI endpoint based on some informations (name of the interface, name of the service) and will send the JBI exchange to the selected endpoint. If the endpoint has been activated by a Service Engine, it means that the service is hosted inside the JBI bus, and not protocol transformation will be performed. However, if the endpoint is activated from a Binding Component, the endpoint will act as a proxy for an external service provider: the binding component will send a request to the service in the protocol it handles to the known location, but the consumer (the BPEL process) has no knowledge of the exact protocol and location: the only things it needs is the contract (the abstract WSDL).

Take a look at ServiceMix and it's User's Guide.

Father for the fifth time

2007-06-02T14:47:00.000+02:00

Welcome to this world, Alexandre! My new son was born on May the 30th, weights 3.8 kg and his height is 52 cm.

Performances of ServiceMix WSDL-First example

2007-03-08T01:46:00.000+01:00

I have spent some time profiling ServiceMix a bit, and particularly the WSDL-First example from the distribution. Here are the results while running a SOAP UI load tests on the example from the 3.1 distribution and the upcoming 3.1.1 distribution:


Version,   min, max, avg,   last, cnt,   tps,    bytes,    bps,    err
SMX 3.1,   9,   234, 20.64, 14,   57724, 484.43, 19106644, 159450, 0
SMX 3.1.1, 5,   204, 16.75, 14,   71065, 596.99, 23522515, 196095, 0

The most interesting number is the tps (transaction per seconds) absolute number: roughly 600 transactions per seconds sounds like a reasonable number (this small tests were running on my laptop without any particular tuning on the JVM or ServiceMix configuration) !

The other one is the tpc increase between 3.1 and 3.1.1:


   595.99 / 484.43 = 1.2324

which means this sample performances for this sample has been boosted by 23 percents !

Using maven to switch runtime properties

2007-03-07T09:33:00.000+01:00

For those who don't know all of Maven's features, I will show how to leverage maven at built-time to easily share run-time properties between service units in ServiceMix.

Let's say that you expose a few services over HTTP/SOAP using servicemix-http. If you are building several service assemblies, you will end up having several files containing:


<http:endpoint service="test:MyConsumerService"
               endpoint="myConsumer"
               role="consumer" 
               locationURI="http://0.0.0.0:8192/Service/"
               soap="true"
               soapVersion="1.1" />

You will certainly want all your services to be exposed on the same port, so it can become a bit tedious if you need to change them all, or if you need to change between different work environments (test, production, etc...).

When working with several service assemblies at the same time, I would recommend to use a hierachical organization for your maven projects:


  root
    |- module1
    |   |- module1-http-su
    |   |- module1-jsr181-su
    |   \- module1-sa
    |- module2
    |   |- module2-http-su
    |   |- module2-jsr181-su
    |   \- module2-sa
    ...

This has several advantages: aside from organizing your code logically, you can use mvn jbi:projectDeploy at the root level, or at a sub-level.

So, now, how can we use maven for our properties ? Well, we just have to create them in the root pom:


<properties>
  <deployment-port>8192</deployment-port>
  <deployment-soap-version>1.2</deployment-soap-version>
</properties>

Then, we need to enable maven resource filtering in our root pom:


<build>
  <resources>
    <resource>
      <directory>src/main/resources
      <filtering>true
    </resource>
  </resources>
</build>

And of course, we need to use them. So instead of the previous xbean.xml, we can now use:


<http:endpoint service="test:MyConsumerService"
               endpoint="myConsumer"
               role="consumer" 
               locationURI="http://0.0.0.0:${deployment-port}/Service/"
               soap="true"
               soapVersion="${deployment-soap-version}" />

So now, if you want to switch your port, just change the root pom property and rebuild :-)

The next step is to use maven profiles to define environment and easily switch between those. This is maybe not useful for changing the HTTP port use, but if you configure some database access, you can easily switch between your production database and an embedded one for testing ...

ServiceMix dependencies

2007-03-05T14:58:00.000+01:00

I have just fixed a small bug in ServiceMix (SM-865) and this makes ServiceMix a *very* lightweight ESB. The bare minimum dependencies to run the following example are:

servicemix-core
servicemix-jbi
servicemix-services
backport-util-concurrent
spring
xbean-spring

The example I ran is very simple, and of course dependencies need to be added depending on the JBI components you use. Anyway, here is the servicemix.xml file:


<beans xmlns:sm="http://servicemix.apache.org/config/1.0"
     xmlns:test="urn:test">
  <sm:container id="jbi" embedded="true">                 
    <sm:activationSpecs>
      <sm:activationSpec service="test:echo">
        <sm:component>
          <bean class="org.apache.servicemix.components.util.EchoComponent" />
        </sm:component>
      </sm:activationSpec>
    </sm:activationSpecs>
  </sm:container>
  <sm:client id="client" container="#jbi" />
</beans>

And here is the java code used:


import javax.jbi.messaging.InOut;
import javax.xml.namespace.QName;
import org.apache.servicemix.client.ServiceMixClient;
import org.apache.servicemix.jbi.jaxp.StringSource;
import org.apache.xbean.spring.context.ClassPathXmlApplicationContext;
import org.springframework.context.ApplicationContext;

public class Main {
   public static void main(String[] args) throws Exception {
       ApplicationContext context = new ClassPathXmlApplicationContext("servicemix.xml");
       ServiceMixClient client = (ServiceMixClient) context.getBean("client");
       InOut me = client.createInOutExchange();
       me.setService(new QName("urn:test", "echo"));
       me.getInMessage().setContent(new StringSource("world"));
       client.sendSync(me);
       System.err.println(me);
   }
}

This is the results of the work that has been done for the 3.1 release. The next step is to use the standard JBI components from ServiceMix ...

ServiceMix 3.1 released

2007-02-12T12:48:00.000+01:00

Woohooo ! ServiceMix 3.1 has just been released officially.
This is a very important version with tons of new features, enhancements and bug fixes.

JMS Soap Binding

2007-01-14T01:03:00.000+01:00

Dan Diephouse just blogged about the announcement of a public review of a JMS SOAP binding and IRI scheme from BEA, IBM, Sonic and Tibco. This is a great news for JMS/SOAP interoperability !

He mentioned that the content-type header has a limited value because of the lack of support for very large messages in JMS. While this may be true for some providers, ActiveMQ provides such a feature.

Also, I'd like to mention I would have been very happy to have a JMS Binding for non-SOAP WSDLs, like the WSDL2 HTTP Binding. I think it would be even more useful !
Anyway, hopefully ServiceMix will have support for the spec soon ...

Jencks 2.0

2007-01-09T10:22:00.000+01:00

Jencks 2.0 has been released.
This is a major release because there are lots of incompatibilities, but the main point has been to upgrade to Geronimo 1.2 Transaction Manager, which leads to a much simpler configuration.
The release also includes the ActiveMQ Pool from my previous blog entry.

ActiveMQ Pooling

2006-12-20T02:35:00.000+01:00

Over the past weeks, I spend some times load testing ServiceMix and ActiveMQ.
I discovered two things:
* ActiveMQ broker is currently single threaded for a given JMS connection. The main effect is that if you send messages from several threads on the same connection, all threads will be processing sequentially (well, not exactly, but they will all be processed by a single thread on the broker side).
* ActiveMQ Resource Adapter does not pool sessions and producers: this means that sending a single message with Jencks using the JMS best practices in a J2EE environment (create a connection, create a session, create a producer, send, close all) will take three consecutive roundtrips to the JMS broker and is very CPU intensive for the broker (at least, when using JMX).

The first problem is mainly apparent when using persistent messages and there are a few things than can be used to speed up things (see the LogicBlaze tuning guide for ActiveMQ). However these tuning tweaks are not always desirable, and there was an easy solution: as the broker use a single thread for given JMS connection, we just have to use multiple connections. Hence, I enhanced the org.apache.activemq.pool package so that the PooledConnectionFactory can reuse more than one connection.

The second point was more difficult to solve, as I did not had time to rewrite the ActiveMQ Resource Adapter -- and I'm still not sure of the downside of pooling sessions and consumers inside a full J2EE environment (security for example). But my needs were only to be able to have XA support and session pooling at the same time. Once again, the solution was to enhance the existing PooledConnectionFactory to support XA resource enlistment. This was quite easy, as the jencks project already contained some code written by a contributor. Unfortunately, I could not reuse it, as it uses the XA JMS api, and in such a case, ActiveMQ force the use of an XA transaction, and I wanted the same connection to be reused with and without transactions. So I rewrite it.

Last, but not least, I needed this ConnectionPool to be correctly used with the Jencks inbound JCA support. It sounds easy (and I thought it was), but at first, the two JMS session were enlisted in the transaction, but they were not recognized as using the same Resource Manager. It was inefficient, but worse, the transaction did not complete successfully. Back to the drawing board ...

Well, the good news is that it now works !
So jencks now features a new module called jencks-amqpool which defines three different factory beans:

PooledConnectionFactory: the standard JMS pooled connection factory
XaPooledConnectionFactory: this one supports XA transaction auto enlistement
JcaPooledConnectionFactory: to be used with jencks JCA inbound support

Note that the three connection factories inherit from each other, so that it is completely safe to use the JCA connection factory in all cases.

Code is available at http://fisheye.codehaus.org/browse/jencks/trunk/jencks-amqpool/src/main/java/org/jencks/amqpool/ and will be include in the upcoming 2.0 release of Jencks.

Here are some simple performance tests. The test spawns 10 threads, each sending 100 messages through the same JMS connection factory.
Using a single connection for the factory:

without transactions: 38 msgs/sec
with a transaction for each message: 41 msgs/sec
with a transaction for 10 messages: 283 msgs/sec

Using 10 connections in the factory:

without transactions: 150 msgs/sec
with a transaction for each message: 173 msgs/sec
with a transaction for 10 messages: 353 msgs/sec

As you can see, the performance boost over the previous pooled connection factory is quite important: 4 times faster ! (unless you use transaction batching).

For the JCA outbound part, the tests are launched with 10 threads, 100 messages per thread.

JCA managed connection factory: 11 msgs/sec
Pooled connection factory: 31 msgs/sec
Pooled connection factory with 10 connections: 143 msgs/sec

The pooled connection factory is three times faster and 12 times faster when using 10 connections ! In addition, the tests were using 10% of my CPU with the pooled connection factory, but 100% with JCA managed connection factory.

GWT

2006-12-12T22:50:00.000+01:00

I have just viewed Bruce Johnson tech talk on GWT (Google Web Toolkit).
It sounds really powerful, especially the fact that your code is statically checked and the hosted mode.
I really need to try it. Maybe a new console for ServiceMix which would allow the management of a ServiceMix cluster ...