I just uploaded my slide deck from my presentation about Apache CXF‘s new features that I presented at CamelOne in Boston to Slideshare.
I will hopefully be getting a full video of it sometime soon. When I do, I’ll post it here.
I haven’t looked at the Apache Jenkins instance leader board in a few months. It’s kind of a silly and meaningless stat, but I broke the CXF build a couple times this week and decided to click on it to see if I had dropped by much. I was VERY surprise, but very impressed with what I saw:
ALL 10 of the top 10 on the leader board are CXF committers. Every one of them. There is definitely something to be said for having a community that encourages frequent commits as well as a build system that is easy to use and promotes testing, but also a general mantra instilled in the community of “Don’t break the build”. (and fix it quickly if you do)
Another interesting note is that 7 of the top 14 are Talend committers. Unfortunately, the other 2 Talend committers are near the bottom of the list (negative even). I guess I need to start pushing them harder to move up the list… 
Anyone on a PMC at Apache likely saw a message sent out about a month ago from infrastructure@ that mentioned the mandate about project websites (and dist areas) change over to using svnpubsub by the end of the year. If you missed the email, it’s also nicely mentioned on the project-site page. For many of the newer projects, using the Apache CMS is definitely the easiest way to achieve that. However, for some of the older projects, this was a bit of an issue. Various projects at Apache have adopted various technologies from forest to anika to maven-site-plugin to Confluence to, well, lots of things. Migrating all that content to the CMS could be a lot of work that the projects just don’t have the time or resources to pursue.
Luckily, Apache does have some excellent people on the infrastructure team that have worked pretty hard to make migrating to just svnpubsub a bit easier. Joe Schaefer, in particular, has worked very hard to update the various buildbot scripts and cms scripts and such to allow it to support external builders (see their blog) to build the actual site content. As long as the technology to build the site can accept an “output directory” flag to output the site, it’s not a hard migration. Thus, projects can remain on their current technology choice as long as they have a build script to build it.
However, that still left out the sites that are using Confluence, such as Apache CXF and Apache Camel. Those projects don’t have a build script. They relied on a proprietary (and pretty buggy and annoying) plugin to Confluence to to render the pages to a local directory on the confluence host, then a series of multiple rsyncs from various personal crontabs to get the pages to the live site. It worked, but it was very slow causing several hours of delays between changes and it appearing on the live site. In any case, the Confuence based sites needed a solution to migrate to svnpubsub.
A while ago, I noticed Confluence has a soap interface. It’s a crappy, ancient, rpc/encoded interface, but it’s at least a usable interface. The interface provides methods to get the page information, render content to HTML, etc… Basically, everything we need to render the site externally. Thus, I used Apache CXF to create a simple program that would act as an external builder to render a site, grabbing all the attachments, applying a template, etc…. With that, plugging it into the svnpubsub infrastructure at Apache is easy. The program also uses Velocity as the template just like the autoexport plugin so migrating an existing template is relatively trivial.
However, by using an external program, I was able to make it MUCH MUCH better than the crappy autoexport plugin that Confluence currently uses. This includes:
In any case, both Apache CXF and Apache Camel now have their main websites migrated over to using this new process to generate their sites from Confluence. If other projects would like to try it out, I’d suggest doing an svn checkout of the Camel website area from http://svn.apache.org/repos/asf/camel/website/ and taking a look.
A long time ago (ok, only 5.5 years ago), when Apache CXF was fist being spun up, we made a decision to try and separate the “API” and the “implementation” of the various CXF components into separate module. This was a very good idea for the most part. However, we did this prior to really investigating OSGi requirements (OSGi was just beginning to become well known). Thus, we didn’t take into account the OSGi split-package issue. The common pattern we used was to put the interfaces in cxf-api, and the “impl” in the exact same package in cxf-rt-core. In OSGi, that’s a big problem (without using fragments).
A few years ago, when users started really pushing the idea of using CXF in OSGi, we had to come up with a solution. I’ll admit we kind of took the “easy road” at the time. As part of the CXF build, we already were creating an “uber bundle” which contained all of the individual CXF modules squashed together into a single jar. We primarily did that to simplify classpath issues, dependencies, etc… for users (single jar to deal with, single pom dependency, etc…). When looking at options for OSGi, we saw that bundle as an “easy” path to OSGi. By adding OSGi manifest entries, CXF can be brought up in OSGi. Since ALL the packages are in that jar, no split-package issues.
For the most part, that setup has worked “OK”, but it’s really not ideal. CXF has a wide range of functionality, much of it may not be required by every user. The bundle ended up importing a LOT of packages. The current bundle imports about 240 packages. Thus, we had to go through and mark many of the imports “optional”. Over 160 of those imports are marked optional. (and likely some of the others could as well) This type of setup, while it “works”, is really less than ideal. It doesn’t work well with things like the OBR resolvers. It requires extra refreshes to add functionality that may be required by new bundles. It requires a lot of extra dependencies to be loaded upfront to reduce refreshes. Etc…
Over the past few weeks, Christian Schneider and I have been re-evaluating the split-package issue in CXF to try and come up with a better solution. An initial evaluation back in December found 25 packages that were split across jars. I have spent quite a bit of time the last couple weeks looking at each of those 25 split packages to figure out what can be done. We did hit some challenges and I ended up talking quite a bit with Christian about possible solutions, but I’m really happy to say that as of my commits this morning, all 25 are now resolved. All of the CXF individual modules are loadable as individual OSGi bundles. A couple of very simple test cases are working.
What kind of problems did we hit:
There were other issues caused by promoting stuff into API, but for the most part, they ended up being resolvable by loading an implementation via a bus extension. That seems to have cleaned things up quite a bit.
Anyway, it’s been a very challenging few weeks as we worked through all the split package issues. To be honest, the end result isn’t “ideal”. There is definitely more stuff in API than I’d really like to have there, but we were able to maintain a high degree of compatibility with existing applications. That’s important. Maybe when we do a “3.0″ we can re-look at that, but for now, compatibility is very important. With the split package issue resolved, Christian and I are embarking on a new mission to examine the OSGi manifests of all the individual jars to adjust import version ranges, check for optionals, update the karaf features file, etc… One advantage of the big bundle was we only needed to do that in one place. Now we have many more touch points. However, the end result should definitely be a better experience for end users. I’m very excited about that.
After writing my post about setting up Apache CXF in OSGi, I’ve had a couple people ask about how to do the same with Apache Camel. Camel is a bit more complex due to all the libraries that it may pull in depending on which components you use, but it’s not really that much more complex.
As with CXF, the easiest way to get it up and running would be to grab a distribution that has it ready to go. Again, I’d recommend either the Talend Integration Factory or Talend ESB. Both have Apache Camel, Apache CXF and Apache ActiveMQ pre setup to run and everything well configured. The more popular Camel components are pre-installed with their dependencies making it fairly easy to get started.
That said, setting up your own instance is really not that hard.
features:addurl mvn:org.apache.activemq/activemq-karaf/5.5.0/xml/features features:addurl mvn:org.apache.cxf.karaf/apache-cxf/2.5.0/xml/features features:addurl mvn:org.apache.camel.karaf/apache-camel/2.8.2/xml/features
That will add the ActiveMQ, CXF, and Camel features into Karafs feature resolver.
features:install activemq features:install activemq-spring (if you use spring)
features:install cxf
features:install camel-core features:install camel-spring features:install camel-jms features:install camel-cxf etc...
You can get a list of the Camel features by doing a “features:list | grep camel” from the Karaf command line.
That’s pretty much all there is to it. It’s not hard. 
It’s now all setup! To try it out, I’d once again grabbing the samples package from Talend Integration Factory. There are several examples in there on how to do various Camel things, all running in OSGi.
This morning, I sent out an announcement that Apache CXF 2.5.0 is now released! Over 6 months of work went into this release to expand the feature set, stabilize it, and get it ready for release.
There are two major categories of new “stuff” in this release:
I’m very excited about the second category of enhancements. Traditionally, CXF has always been “just a framework” that can be used to build your services. CXF never really provided any services “out of the box”. I’m hoping that by providing some enterprise level services out of the box, it will become even easier for people to develop and deploy CXF based applications in their enterprise. Both of the new services, while “new” for CXF, are really enterprise ready. The WS-Notification Service is a port (mostly done by Guillaume Nodet) of the ServiceMix WS-Notification service which has been deployed and used in production in many places. By removing the JBI dependencies and re-writing the communications to use pure JAX-WS API’s, the WS-Notification service is now much more usable by a wider audience. The STS was developed by Talend for some of our customers and has been deployed into production. Colm has done an amazing job with the STS to get it to meet a wide range of requirements. Definitely check out his blog for more information about the capabilities of the STS.
Anyway, this is another big release for Apache CXF. Definitely give it a spin and let us know how we’re doing!
I’ve had a bunch of people asking me lately about getting Apache CXF up and running in OSGi. A lot of people have run into issues trying to find the right third party bundles, configuring things like JAX-WS and JAXB, normal “class loader” issues, etc…. Thus, I decided I need to write a blog entry.
Obviously, the EASIEST way to get CXF up and running in OSGi is to grab a preconfigured OSGi container that has CXF already setup. There are a couple of good ones out there. For “pure CXF”, the best option is the Talend Service Factory. It’s pretty much Apache Karaf and Apache CXF completely setup to run in OSGi out of the box. Very simple. If you want/need a little more functionality, you could grab the Talend Integration Factory or the Talend ESB. Both of them contain Apache Karaf, Apache CXF, and Apache Camel pre-setup to run in OSGi. If you don’t mind an older version of CXF, you can also use the latest Apache ServiceMix 4.x distributions as they would have CXF ready to run as well. However, the current downloads are based on a fairly old version of CXF. They are working on a newer version that would contain a newer version of CXF, but that may not be available for a couple weeks. Also, ServiceMix contains a lot of other things that may or may not be useful for you. For the “lightest weight” option, the Talend Service Factory is the best option.
However, not everyone wants a pre-configured thing. They may want to customize things a bit. Or maybe add some of their own things. For those types of people here are some basic instructions that I would recommend:
1) Start with Apache Karaf. Karaf is an excellent general purpose OSGi container and provides a bunch of features to make life with OSGi much easier. Grab the latest (currently 2.2.4) version, download it, and unpack it.
2) In the unpacked Karaf area, edit the etc/jre.properties file. For CXF to work we need to use OSGi enabled bundles of various API’s. The versions of the API’s built into the JDK are not OSGi enabled and thus will not work properly with CXF. In the jdk-1.6 (and 1.7 if using Java7), you need to commend out the lines:
# javax.activation, \ # javax.annotation;version="1.1", \ # javax.jws, \ # javax.jws.soap, \ # javax.xml.bind, \ # javax.xml.bind.annotation, \ # javax.xml.bind.annotation.adapters, \ # javax.xml.bind.attachment, \ # javax.xml.bind.helpers, \ # javax.xml.bind.util, \ # javax.xml.soap, \ # javax.xml.stream, \ # javax.xml.stream.events, \ # javax.xml.stream.util, \ # javax.xml.ws, \ # javax.xml.ws.handler, \ # javax.xml.ws.handler.soap, \ # javax.xml.ws.http, \ # javax.xml.ws.soap, \ # javax.xml.ws.spi, \
All of those API’s use the SPI mechanism in the JDK to find implementations. That mechanism relies on Thread contextClassLoader mechanisms that don’t work reliably in OSGi. For CXF, we have specific bundled versions of those API’s that use a proper BundleActivator/Listener method to find the implementations.
I’d also recommend editing the etc/custom.properties file to add karaf.framework=equinox to set the underlying OSGi runtime to Equinox instead of Felix. Currently, CXF is a bit better tested with Equinox. Future versions of CXF will work better with Felix, but right now, the testing is more complete with Equinox.
3) Start the container by running the bin/karaf file.
4) From the Karaf command line, type:
features:addurl mvn:org.apache.cxf.karaf/apache-cxf/2.5.0/xml/features
That will install the CXF Karaf features file. After that, you can run “features:list” and see the features that CXF provides.
5) Run
features:install cxf
To install the CXF feature. Karaf will then download and install all the required dependencies, bundles, etc… and install them into the OSGi container. It will also start the bundles. If you run “list -t 0″ you should see “Apache CXF Bundle Jar” active at the bottom.
That’s really all there is to it. At that point, CXF is installed and ready to be used. Now, USING it is a subject for another blog post. For now, your best option is to grab the Talend Service Factory examples and take a look at them. All of them are ready to run in OSGi.
This past week, there was a discussion on the Apache CXF developer list about adding an enterprise grade Security Token Service (STS) into CXF. Talend has been developing an STS internally for some of our customers for a while now and we’re pretty excited to be able to contribute the results of that work back to the community. CXF 2.4.0 had a very basic “framework” for developing an STS built into it along with a simple example that showed how to use the framework to setup a very simple STS. However, most of the details of creating and verifying tokens and such was completely left to the user/developer. The new STS, while built on that same framework, has a lot more capabilities “built in” and requires significantly less work to setup.
Colm’s blog definitely covers most of the details about the functionality of the new STS. Over the upcoming weeks, he should be adding more details about how to configure and leverage many of the features he mentions. Keep an eye out on his blog for details.
I’m quite excited about this. One of my personal goals in developing Open Source software is making that software more approachable and usable in enterprise environments. Over the years, CXF has done a great job become the defacto open source services framework by trying to meet the needs of enterprise deployments. This takes it even a step further by providing the first ready made service based on CXF. Hopefully, there will be more to come. Stay tuned!
In the last week or so, there’s been a bit of work going on in CXF to improve the OSGi experience with CXF as well as make it easier to manage.
First off, Glen Mazza has done quite a bit of testing with JMX and wrote a very good blog entry about how to JMX enable your CXF services (and Metro). As part of that, he’s been testing it with various app servers and updating the documentation and such. All great work.
On the OSGi side, CXF will now register the Bus objects into the OSGi service registry. Thus, other OSGi services can query them and pull information out of them. The primary driver for this is the new Karaf WebConsole. It’s very very early (just added to the WebConsole yesterday), but you can see some of the stuff that can be queried here. Also, to make this more useful, the Bus ID now defaults to the bundle SymbolicName+”cxf” when running in OSGi. This makes it much easier in the WebConsole and in JMX to find the endpoints of interest when there are many deployed.
Combining both of the above, when running in OSGi, CXF will now lookup an MBeanServer from the container and register its MBeans automatically. With previous versions of CXF, the user would need to add a bunch of configuration into their blueprint or spring-dm configs to enable the JMX management. That will be done automatically now.
I’m quite happy with the above changes. It will really make the administrators job easier when using CXF, particularly in OSGi based ESB’s like Talend’s ESB.
One of the important differentiators between Apache CXF and some of it’s competitors has been it’s support for standards, both “on the wire” standards like SOAP and API standards like JAX-WS and JAX-RS. When Apache resigned from the JCP last year, there was a lot of concern about whether CXF would be able to continue to be involved with the defining and implementing the standards. As an Apache member, I fully support the actions that Apache took as there really wasn’t a good alternative. However, it did leave a gap in some of the projects abilities to participate in the specifications.
I’m happy to announce that Talend has joined the JCP to allow some of our contributors like myself and Sergey to continue being involved and help shape the future of the standards that CXF implements. The immediate benefit is that Sergey can jump into the JAX-RS 2.0 discussions and help make sure that JAX-RS 2.0 can meet the needs of CXF users. The combination of the JAX-WS and JAX-RS programming models in CXF is a great combination of technologies and making sure the CXF users needs are well represented is important.
For a more complete discussion of Talends thoughts, see Fabrice’s blog entry on Talends site. From my standpoint, the most important piece to understand is that this shows Talend is not just committed to supporting CXF and the other Apache projects we are involved in, but also in investing in making sure the projects continue to innovate and move forward to solve users needs and problems. That is one of the unique points that makes Talend special.
