Spring-DM, Aries Blueprint and custom namespaces
Since Spring 2.0, Spring supports custom namespace handlers. This feature is really useful and can be used to reduce the verbosity of your spring based applications. Spring-DM has brought this feature along, so it's also available in an OSGi environment. Unfortunately, the integration of those namespace handlers leads to a lot of problems, mostly due to the legacy spring way of configuring the handlers which does not map well at all in OSGi.
In a standard environment, Spring lives in a simple classloader. The classloader which is used to create the spring application is not supposed to change at any time, and everything that is needed for the application is supposed to be available from that classloader. So let's see what happens with custom namespace handlers. Spring detects custom namespace handlers through the presence of a file in the following location META-INF/spring.handlers. This file contains the name of the class supporting a given namespace. In a non OSGi environment, spring detects the namespace handlers by iterating through the resources and finding all these files. However, in an OSGi environment, things must be done a bit differently. The direction which has been taken by Spring-DM is to watch all started bundles and check for this META-INF/spring.handlers file. The custom namespace is then registered and available for use. This leads to multiple problems.
The first one comes from the fact that there is now way to express a dependency onto a "started" bundle. The OSGi specification defines the Require-Bundle header and Import-Package, but both headers do not care about the fact that the bundle is started or not, only that it is resolved. This means that when you deploy your bundles, you need to control the start order of all your bundles to make sure that all custom namespace handlers are started before any application that use them. Failing to do so will result in applications not being started and the solution is to manually restart the bundle. This really goes against the OSGi best practices.
The second problem comes is caused by the non-staged approach in the parsing of the spring xml configuration files. The Spring-DM extender parses the xml and each time a non standard element is found, try to locate the appropriate namespace handler in its internal registry and delegate to that namespace. Once again, this does not work well in OSGi because namespace handlers can come and go and you can't be sure that the namespace are available at the time your bundle start.
The next problem is related to classloading. The namespace handler does not come with its own classloader than can contribute to the application (remember spring has been designed to work with a single static classloader). This means that in order to use a custom namespace, the classloader of the bundle needs to be able to access all the classes that the custom namespace handler will refer to. If you have ever tried to use CXF in OSGi, you know what I mean: you need to import dozens of packages for your application to start to allow the namespace handler to find the classes it needs.
Another limitation is that custom namespace can't be versioned with Spring-DM and it may lead to having to define different namespaces to solve class-related compatibility issues. Let's say you have a namespace foo. This namespace allows to to create an interface named bar (version 1.0). Later, you have a revision of this package . This new version of the interface may or may not be compatible with the previous 1.0 version, but this has nothing to do with the namespace or xml schema itself. With Spring-DM, you need to version your schema because you can't have two namespace handlers supporting the same namespace. This may be problematic because you end up with two solutions. The first one is to always change your namespace (by including the version in the namespace), which means all the bundles using your namespace need to change their xml configuration even it it was compatible. The other solution is to not allow two versions to live in the same OSGi environment. This is simply not possible unless you fully control everything and this goes against the OSGi very goal.
I really don't blame the Spring-DM guys: I think their goal was to be fully backward compatible and that's what they achieved. Sadly, the end result does not behave well in an OSGi environment.
Now, the nice part is that the Apache Aries Blueprint implementation which I've worked on for quite some time now does a much better job at handling those custom namespaces. All four problems have been solved by leveraging OSGi. The implementation leverages the service registry for custom namespace handlers, each bundle containing a custom handler registering its own service. When starting an application, the blueprint extender will make a first pass on the xml configuration files and detect which namespace are used, then put the application in a waiting state until all namespace handlers are available. Each namespace handler can load its own classes thus avoiding the need for the blueprint bundle to import classes that it's not really aware of (because it uses a custom xml namespace and not plain beans). Last, by ensuring class-space consistency, a much cleaner versioning can be done on the various namespaces.
So give it a try if you hit one of the mentioned issues.
In a standard environment, Spring lives in a simple classloader. The classloader which is used to create the spring application is not supposed to change at any time, and everything that is needed for the application is supposed to be available from that classloader. So let's see what happens with custom namespace handlers. Spring detects custom namespace handlers through the presence of a file in the following location META-INF/spring.handlers. This file contains the name of the class supporting a given namespace. In a non OSGi environment, spring detects the namespace handlers by iterating through the resources and finding all these files. However, in an OSGi environment, things must be done a bit differently. The direction which has been taken by Spring-DM is to watch all started bundles and check for this META-INF/spring.handlers file. The custom namespace is then registered and available for use. This leads to multiple problems.
The first one comes from the fact that there is now way to express a dependency onto a "started" bundle. The OSGi specification defines the Require-Bundle header and Import-Package, but both headers do not care about the fact that the bundle is started or not, only that it is resolved. This means that when you deploy your bundles, you need to control the start order of all your bundles to make sure that all custom namespace handlers are started before any application that use them. Failing to do so will result in applications not being started and the solution is to manually restart the bundle. This really goes against the OSGi best practices.
The second problem comes is caused by the non-staged approach in the parsing of the spring xml configuration files. The Spring-DM extender parses the xml and each time a non standard element is found, try to locate the appropriate namespace handler in its internal registry and delegate to that namespace. Once again, this does not work well in OSGi because namespace handlers can come and go and you can't be sure that the namespace are available at the time your bundle start.
The next problem is related to classloading. The namespace handler does not come with its own classloader than can contribute to the application (remember spring has been designed to work with a single static classloader). This means that in order to use a custom namespace, the classloader of the bundle needs to be able to access all the classes that the custom namespace handler will refer to. If you have ever tried to use CXF in OSGi, you know what I mean: you need to import dozens of packages for your application to start to allow the namespace handler to find the classes it needs.
Another limitation is that custom namespace can't be versioned with Spring-DM and it may lead to having to define different namespaces to solve class-related compatibility issues. Let's say you have a namespace foo. This namespace allows to to create an interface named bar (version 1.0). Later, you have a revision of this package . This new version of the interface may or may not be compatible with the previous 1.0 version, but this has nothing to do with the namespace or xml schema itself. With Spring-DM, you need to version your schema because you can't have two namespace handlers supporting the same namespace. This may be problematic because you end up with two solutions. The first one is to always change your namespace (by including the version in the namespace), which means all the bundles using your namespace need to change their xml configuration even it it was compatible. The other solution is to not allow two versions to live in the same OSGi environment. This is simply not possible unless you fully control everything and this goes against the OSGi very goal.
I really don't blame the Spring-DM guys: I think their goal was to be fully backward compatible and that's what they achieved. Sadly, the end result does not behave well in an OSGi environment.
Now, the nice part is that the Apache Aries Blueprint implementation which I've worked on for quite some time now does a much better job at handling those custom namespaces. All four problems have been solved by leveraging OSGi. The implementation leverages the service registry for custom namespace handlers, each bundle containing a custom handler registering its own service. When starting an application, the blueprint extender will make a first pass on the xml configuration files and detect which namespace are used, then put the application in a waiting state until all namespace handlers are available. Each namespace handler can load its own classes thus avoiding the need for the blueprint bundle to import classes that it's not really aware of (because it uses a custom xml namespace and not plain beans). Last, by ensuring class-space consistency, a much cleaner versioning can be done on the various namespaces.
So give it a try if you hit one of the mentioned issues.
Comments
The issues you listed with namespaces are side-effects of a generic problem in OSGi regarding resource discovery and sharing - otherwise known as the extender pattern.
While types (as in java.lang.Class) are versioned and treated as first-class citizens in OSGi, resources (as in files) are not. You can share the packages in which files are defined (as long as they follow the java conventions which means META-INF/ cannot be used) but you still cannot differentiate them apart as there is no equivalent of a ClassLoader or a resource-consistent space. A common example is the META-INF/services location - without dedicated support from the OSGi framework, it's pretty much impossible to leverage this mechanism in OSGi (without modifying existing jars or duplicating the OSGi resolver).
However, as you probably know, there are proposals in the OSGi Alliance to address this gap (which we plan to incorporate).
As for actual solutions to the problems mentioned:
1. Depending on the number of namespaces used, ordering the bundles might be needed. Spring DM doesn't address this yet but dm Server (upcoming Eclipse Virgo) does.
Moreover, as you probably know, namespaces are currently discussed for the next Blueprint spec release.
2. Falls under 1) category however we have plan to improve the current behaviour (regardless of 1) in the next release).
3. The namespace handler implementation details (let alone class loading) are not the concern of the client - that's the whole point of decoupling after all. As for technicalities, the namespaces are services (as you've mentioned) that are bound to their declaring bundle, which means they do have a consistent class space (guaranteed by
the OSGi platform). Spring DM checks the compatibility between the namespace handler and the client and further more, for the rare cases where the handler has an intrusive approach, one can simply enforce consistency through the existing OSGi manifest directives.
4. If you want to use different versions of a custom namespace/schema, you can either version it (which is a best practice and it makes sense) or not (in which case Spring DM treats it as having a "default" version). At runtime, the best matching "class consistent" version is picked up but then again, for the cases where this doesn't apply, the OSGi directive can solve this elegantly, out of the box.
These being said, I'm glad to see Spring namespaces supported in Aries and interested to know more about the problems you faced and how you addressed them.
Cheers,
Costin Leau
Spring DM/Blueprint RI Lead
Regards
Stefan
What happens at the end is that as soon as you use custom namespaces you need to fully control the start order of all bundles so that bundles containing namespace handlers are started before the application bundles. And that's independent of the number of namespace handlers used. The problem is that Spring-DM decided to keep full backward compatibility so that dropping a spring namespace handler (using the META-INF folder) in osgi would work with spring-dm. Using a whiteboard pattern for the handlers would have been more appropriate I think.
A staged parsing would then have been able to preparse the xml files, extract the namespaces in use and wait for those to be available (as it's done in the grace period).
On point 3, I don't understand what you're saying. When you implement a custom namespace, the clients of this namespace don't need to know what implementation classes are used behind the scene. As you say, that's exactly the point of decoupling. The problem is that the current design of Spring namespaces does not allow that, thus this very problem of packages surfaces and the clients actually have to care about importing the needed packages in order for the namespace to work.
On the last point, I agree that OSGi support that. The problem is that Spring-DM does not support multiple handlers for the same namespace as far as I know. It only supports multiple namespaces wired onto different spring extenders (you still have at most one handler for a given namespace / spring-dm extender), which is not the case I'm talking about. The use case I'm referring to is a bit more subtle than that: it's about running multiple extenders for the same namespace and choose the one according to the class-space consistency of the client. Let's say you have Apache Camel 2.0 and 2.1 running at the same time in the framework, you want to be able to deploy camel routes and have them wired to the right camel version depending on which version of camel the client uses, but that does not mean that you want to version the namespace, because this put additional burden on the clients and the migration to a higher version very painful.
And I will for sure bring all that feedback when we'll work on the standardized blueprint namespaces in the OSGi EEG.
It has worked great so far and except the ordering issue (which again we decided to not address ourselves with dm Server/Eclipse Virgo and the proposals in the OSGi Alliance).
You mention that Spring DM is "fully backwards compatible" - which is what we (Spring DM team) wanted, but does this mean your approach is not? And if so, can you point out in what ways?
Do you provide a different namespace handler abstraction or reuse the existing one?
Regarding 3), we'll probably have to take this offline since don't understand why the client needs to load the classes provided by the namespace handler. The handler can just inject the classes into the container (which is what happens now) - it's simple, clean and efficient.
For 4), I never encountered that case before. From you are telling me, the namespace/schema model was extended from a one to one mapping, to one to many. I'll see what I can do to support this in the next version but it would help to get some more information on your exact case - are there some links out there that you can share?
We are using FUSEESB which runs on Karaf.
It is very common that the blueprint file is not able to resolve any of the spring namespaces. For most of the implementation blueprint has created its own implementation which resolves most of the senarios. but in case we want to use spring namespaces in blueprint it becomes impossible. Adding a service with blueprint Namespace handler did not help or probably I am not sure if it was done correctly.
If there is way to resolve this then we will be able to use lot of spring features with blueprint.