Blog from January, 2012

CXF 2.6.0 will bring a lot of improvements for deployment in OSGi. Till now cxf was bundled in one OSGi bundle. Either with all features or with a minimal feature set. Thanks to Dan Kulp cxf is now delivered as individual bundles. So it can be installed with only the needed features. Besides the smaller size in many use cases this also means that we have less optional dependencies which make installation difficult. Each bundle defines the imports it really needs. This makes it much easier to get the dependencies right. Of course the Karaf feature file will still be provided to make it easy to install CXF in Apache Karaf.

Based on the work of Dan I recently started to optimize the imports of the typical bundles most people will use from cxf. At the start we had many dependencies like spring, velocity, neethi, .., that I felt should not be needed and make cxf quite big. By refactoring some of the modules I was able to slim these down to the bare minimum. The current code on trunk already reflects these changes.

If you want to try this yourself you can easily install the snapshot of cxf in karaf 2.2.5. As the feature file is not yet changed I uploaded a gist of the commands you need to execute. Remember to also use the jre.properties.cxf for karaf to disable some default java apis so CXF can replace them with newer versions.

So after this install the karaf list -u command shows the following bundles:

This installation of CXF is ready for SOAP/HTTP and REST with JAX-WS and JAXB on the java side which reflects what most people will need.

To test the features I recommend to install the example from my Karaf Tutorial about CXF.

Shows how to access databases from OSGi applications running in Karaf and how to abstract from the DB product by installing DataSources as OSGi services. Some new Karaf shell commands can be used to work with the database from the command line. Finally JDBC and JPA examples show how to use such a DataSource from user code.

Installing the driver

The first step is to install the driver jar(s) for your database system into Karaf. Most drivers are already valid bundles and available in the maven repo. So this is tpyically only
one Karaf command. If the driver is available in maven but no bundle we can most times use the wrap: protocol of Karaf to make it a bundle on the fly. If the driver is not even
in the repo we have to install the file into the maven repo first.

For derby the following command will work

See the db/datasource folder on github for installation instructions for (db2, derby, h2, mysql and oracle).

Installing the datasource

In Java EE servers and servlet containters you typically use JNDI to install a DataSource on the server level so the application can just refer to it and so does not have to know the specific driver or database url. In OSGi JNDI is replaced by OSGi services. So the best way to decouple your application from the database is to offer a DataSource as an OSGi service.

As we can deploy simple blueprint xml files in Karaf this is really easy. We define a bean with the class of the specific datasource impl and configure it. Then we publish that bean as an OSGi service with the interface a javax.sql.DataSource. This works because Karaf uses dynamic imports when it deploys blueprint context files so all classes are available.

For each database flalour you can find a suitable blueprint.xml in db/datasource.

Browsing the database using the Karaf db:* commands

As part of this tutorial I created some simple Karaf commands to work with databases from the Karaf shell. These commands proved to be quite handy so I will try to move them to the Karaf project.

db:select <name>

When called without parameters the command shows all available DataSources.
Example:

When called with the name of a DataSource it will select the DataSource:

Example:

db:exec "<sql>"

Executes a SQL command.
Example:

This creates a table person and adds a row to it.

db:tables

Shows the current tables in the database.

Example:

db:query

Executes a query and shows the results.

Example:

Accessing the database using JDBC

The project db/examplejdbc shows how to use the datasource we installed and execute jdbc commands on it. The example uses a blueprint.xml to refer to the OSGi service for the DataSource and injects it into the class
DbExample.The test method is then called as init method and shows some jdbc statements on the DataSource.The DbExample class is completely independent of OSGi and can be easily tested standalone using the DbExampleTest. This test shows how to manually set up the DataSource outside of OSGi.

Build and install

Build works like always using maven

In Karaf we just need our own bundle as we have no special dependencies

After installation the bundle should directly print the db info and the persisted person.

Accessing the database using JPA

For larger projects it is quite typical that JPA is used instead of hand crafted SQL. Using JPA has two big advantages over JDBC. You need to maintain less SQL code and JPA provides dialects for the subtle differences in databases that else you would have to code yourself. For this example we use OpenJPA as the JPA Implementation. On top of it we add Apache Aries JPA which supplies an implementation of the OSGi JPA Service Specification and blueprint integration for JPA.

The project examplejpa shows a simple project that implements a PersonService that manages Person objects.
Person is just a java bean annotated with JPA @Entitiy. As OpenJPA needs to enhance the bytecode of the classes we need to add the openjpa-maven-plugin to the pom.xml which prepares the classes for JPA.

Additionally the project implements two Karaf shell commands person:add and person:list that allow to easily test the project.

persistence.xml

Like in a typical JPA project the peristence.xml defines the DataSource lookup, database settings and lists the persistent classes. The datasource is refered using "osgi:service/javax.sql.DataSource/(osgi.jndi.service.name=jdbc/derbyds)". This makes a lookup for an OSGi service with the given interface and properties.

The OSGi JPA Service Specification defines that the Manifest should contain an attribute "Meta-Persistence" that points to the persistence.xml. So this needs to be defined in the config of the maven bundle plugin in the prom. The Aries JPA container will scan for these attributes
and register an initialized EntityMangerFactory as an OSGi service on behalf of the use bundle.

blueprint.xml

We use a blueprint.xml context to inject an EntityManager into our service implementation and to provide automatic transaction support.
The following snippet is the interesting part:

This makes a lookup for the EntityManagerFactory OSGi service that is suitable for the persistence unit person and injects an EnityManager into the
PersonServiceImpl. Additionally it wraps each call to a method of PersonServiceImpl with code that opens a transaction
before the method and commits on success or rollbacks on any exception thrown.

Build and Install

The project builds with mvn clean install like usual. A prerequisite is that the derby datasource is installed like described above. Then we have to install the bundles for openjpa, aries (jpa, transaction, proxy and jndi) and of course our db-examplejpa bundle.
See ReadMe.txt for the exact commands to use.

Test

This should create a person object with the above data and persist it to the database. Unfortunately this currently does not work. I guess I still have an error somewhere. So instead we use the db commands to populate the DB by manually:

Then we list the persisted persons

Using pooling for datasources

I any real world scenario you will need pooling for the DataSource. To achieve this you have two good options:

1. Use a pooling datasource from the vendor:

DB

Class

Derby

org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource

MySQL

com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource

Oracle

oracle.jdbc.pool.OracleConnectionPoolDataSource

2. Use the PoolingDataSource from dbcp like described in this gist by Andreas Pieber: https://gist.github.com/2761628

Summary

In this tutorial we learned how to work with databases in Apache Karaf. We installed drivers for our database and a DataSource. We were able to check and manipulate the DataSource using the db:* commands. In the examplejdbc we learned how to acquire a datasource
and work with it using plain jdbc. This is really easy but a bit verbose. You might want to try the spring JdbcTemplate to remove all the cleanup code. Last but not least we also used jpa to access our database.

In theory JPA and OSGi work together really well. Keep in mind though that JPA support for OSGi is still quite fresh. It took me quite a while to get it all running. The documentation is quite sparse and I still have not been able to fix the persist issue. I will update the code and blog entry as soon as I have the jpa persist working.

Back to Karaf Tutorials

Shows how to run your camel routes in the OSGi server Apache Karaf. Like for CXF blueprint is used to boot up camel. The tutorial shows three examples - a simple blueprint route, a jms2rest adapter and an order processing example.

Installing Karaf and making Camel features available

  • Download Karaf 2.3.1 and unpack to the file system
  • Start bin\karaf.bat or bin/karaf for unix

In Karaf type:

You should see the camel features

Getting and building the examples

You can find the examples for this tutorial on github Karaf Tutorial - camel.

So either clone the git repo or just download and unpack the zip of it.To build the code do:

Starting simple with a pure blueprint deployment

Our first example does not even require a java project. In Karaf it is possible to deploy pure blueprint xml files. As camel is well integrated with blueprint you can define a complete camel context with routes in a simple blueprint file.

simple-camel-blueprint.xml

The blueprint xml for a camel context is very similar to the same in spring. Mainly the namespaces are different. Blueprint discovers the dependency on camel so it will automatically require the at least the camel-blueprint feature is installed. The camel components in routes are discovered as OSGi services. So as soon as a camel component is installed using the respective feature it is automatically available for usage in routes.

So to install the above blueprint based camel integration you only have to do the following steps:

Copy simple-camel-blueprint.xml to the deploy folder of karaf. You should now see "Hello Camel" written to the console every 5 seconds.

The blueprint file will be automatically monitored for changes so any changes we do are directly refelcted in Karaf. To try this open the simple-camel-blueprint.xml file from the deploy folder in an editor, change "stream:out" to "log:test" and save. Now the messages on the console should stop and instead you should be able to see "Hello Camel" in the Karaf log file formatted as a normal log line.

JMS to REST Adapter (jms2rest)

jms2rest

Icon

This example is not completely standalone. As a prerequisite install the person service example like described in Karaf Tutorial 4.

The example shows how to create a bridge from the messaging world to a REST service. It is simple enough that it could be done in a pure blueprint file like the example above. As any bigger integration needs some java code I opted to use a java project for that case.

Like most times we mainly use the maven bundle plugin with defaults and the packaging type bundle to make the project OSGi ready. The camel context is booted up using a blueprint file blueprint.xml and the routes are defined in the java class Jms2RestRoute.

Routes

The first route watches the directory "in" and writes the content of any file placed there to the jms queue "person". It is not strictly necessary but makes it much simpler to test the example by hand.

The seconds route is the real jms2rest adapter. It listens on the jms queue person and expects to get xml content with persons like also used in the PersonService. In the route the id of the person is extracted from the xml and stored in a camel message header. This header is then used to build the rest uri. As a last step the content from the message is sent to the rest uri with a PUT request. So this tells the service to store the person with the given id and data.

Use of Properties

Besides the pure route the example shows some more tpyical things you need in camel. So it is a good practice to externalize the url of services we access. Camel uses the Properties component for this task.

 This enables us to write {{personServiceUri}} in endpoints or ${properties:personServiceUri} in the simple language.

In a blueprint context the Properties component is automatically aware of injected properties from the config admin service. We use a cm:property-placeholder definition to inject the attributes of the config admin pid "net.lr.tutorial.karaf.cxf.personservice". As there might be no such pid we also define a default value for the personServiceUri so the integration can be deployed without further configuation.

JMS Component

We are using the camel jms component in our routes. This is one of the few components that need further configuration to work. We also do this in the blueprint context by defining a JmsComponent and injecting a connection factory into it. In OSGi it is good practice to not define connection factories or data sources directly in the bundle instead we are simply refering to it using a OSGi service reference.

Deploying and testing the jms2rest Adapter

Just type the following in Karaf:

This installs the activemq and camel feature files and features in karaf. The activemq:create command creates a broker defintions in the deploy folder. This broker is then automatically started. The broker defintion also publishes an OSGi service for a suitable connection factory. This is then referenced later by our bundle.

As a last step we install our own bundle with the camel route.

Now the route should be visible when typing:

Now copy the file person1.xml to the folder "in" below the karaf directory. The file should be sent to the queue person by the first route and then sent to the rest service by the second route.

In case the personservice is instaleld you should now see a message like "Update request received for ...". In case it is not installed you should see a 404 in the karaf error when accessing the rest service.

Order processing example

order

The business case in this example is a shop that partly works with external vendors.

We receive an order as an xml file (See: order1.xml). The order contains a customer element and several item elements. Each item specifies a vendor. This can be either "direct" when we deliver the item ourself or a external vendor name. If the item vendor is "direct" then the item should be exported to a file in a directory with the customer name. All other items are sent out by mail. The mail content should be customizeable. The mail address has to be fetched from a service that maps vendor name to mail address.

How it works

This example again uses maven to build, a blueprint.xml context to boot up camel and a java class OrderRouteBuilder for the camel routes. So from an OSGi perspective it works almost the same as the jms2rest example.

The routes are defined in net.lr.tutorial.karaf.camel.order.OrderRouteBuilder. The "order" route listens on the directory "orderin" and expects xml order files to be placed there. The route uses xpath to extract several attributes of the order into message headers. A splitter is used to handle each (/order/item) spearately. Then a content based router is used to handle "direct" items different from others.

In the case of a direct item the recipientlist pattern is used to build the destination folder dynamically using a simple language expression.

If the vendor is not "direct" then the route "mailtovendor" is called to create and send a mail to the vendor. Some subject and to address are set using special header names that the mail component understands. The content of the mail is expected in the message body. As the body also should be comfigureable the velocity component is used to fill the mailtemplate.txt with values from the headers that were extracted before.

Deploy into karaf

The deployment is also very similar to the previous example but a little simpler as we do not need jms. Type the following in karaf

To be able to receive the mail you have to edit the configuration pid. You can either do this by placing a properties file
into etc/net.lr.tutorial.karaf.cxf.personservice.cfg or editing the config pid using the karaf webconsole. (See part 2 and part 3 of the Karaf Tutorial series).

Basically you have to set these two properties according to your own mail environment.

Test the order example

Copy the file order1.xml into the folder "ordersin" below the karaf dir.

The Karaf console will show:

The same should be in a mail in your inbox. At the same time a file should be created in ordersout/Christian Schneider/order1.xml that contains the book item.

Wrapping it up and outlook

The examples show that fairly sophisticated integrations can be done using camel and be nicely deployed in an Apache Karaf container. The examples also show some best practices around configuration management, jms connection factories and templates for customization. The examples should also provide a good starting point for you own integration projects. Many people are a bit hesitant using OSGi in production. I hope these simple examples can show how easy this is in practice. Still problems can arise of course. For that case it is advisable to think about getting a support contract from a vendor like Talend. The whole Talend Integration portfolio is based on Apache Karaf so we are quite experienced in this area.

I have left out one big use case for Apache Camel in this tutorial - Database integrations. This is a big area and warrants a separate tutorial that will soon follow. There I will also explain how to handle DataSources and Connection Factories with drivers that are not already OSGi compliant.

Back to Karaf Tutorials