Blog
Skip to end of metadata
Go to start of metadata

Some time ago I did some CXF performance measurements. See How fast is CXF ? - Measuring CXF performance on http, https and jms.

For cxf 3.0.0 I did some massive changes on the JMS transport. So I thought it is a good time to compare cxf 2.x and 3 in JMS performance. My goal was to reach at least the original performance. As my test system is different now I am also measuring the cxf 2.x performance again to have a good comparison.

Test System

Dell Precision with Intel Core i7, 16 GB Ram, 256 GB SSD running ubuntu Linux 13.10.

Test Setup

I am using a new version of my performance-tests project on github.

The test runs on one machine using one activemq Server, one test server and one test client.

The test calls the example cxf CustomerService.

The following call types are supported:

Call type

Code

Description

oneway

customerService.updateCustomer(customer);

Asynchronous one way call. Sends one soap message to server

requestReply

List<Customer> customers = customerService.getCustomersByName("test2");

Synchronous request reply. Sends one soap message to server and waits for the reply

requestReplyAsync

Future<GetCustomersByNameResponse> resp = customerService.getCustomersByNameAsync("test2");
GetCustomersByNameResponse res1 = resp.get();

Sends an asynchronous request reply. Sends one soap message to server and returns without waiting.
In this test we wait directly after the call for simplicity.

The requests above are sent using an executor with a fixed number of threads.

For the test you can specify the total number of messages, the number of threads and the call type.
First the number of requests are sent for warmup and then for the real measured test.
To run the test with cxf 3.0.0-SNAPSHOT you have to compile cxf from source.

Test execution

1. Run a standalone activemq 5.9.0 server with the activemq.xml from the github sources above.

2. Start the jms server in a new console from the project source using:

3. Start the jms client using:

Test results

The test is executed with several combinations of the parameters. Using the pom property cxf.version we also switch between cxf 2.7.10 and cxf 3.0.0-SNAPSHOT.

CXF 2.7.10

threads
call type

1

20

40

oneway

10541

12143

11737

requestReply

610

661

691

requestReplyAsync

1561

3448

3859

CXF 3.0.0-SNAPSHOT

threads
call type

1

20

40

oneway

11170

11632

12010

requestReply

1524

3248

3671

requestReplyAsync

1590

3569

3909

Observations

The first interesting fact here is that one way messaging does not profit from the number of threads. One thread already seems to achieve the same performance like 40 threads. This is quit intuitive as activemq needs to synchronize the calls on the one thread holding the jms connection. On the other hand using more processes also does not seem to improve the performance so we seem to be quite at the limit of activemq here which is good.

For request reply the performance seems to scale with the number threads. This can be explained as we have to wait for the response and can use this time to send some more requests.

One really astonishing thing here is that CXF 2.7.10 seems to be really bad when using synchronous request reply. This is because it uses consumer.receive in this case while it uses a jms message listener for async calls. So the jms message listener seems to perform much better than the consumer.receive case. For CXF 2.7.10 this means we can speed up our calls if we use the asynchronous interface even if it is more inconvenient.

The most important observation here is that CXF 3 performs a lot better for the synchronous request reply case. It is as fast as for the asynchronous case. The reason is that we now also use a message listener for synchronous calls as long as our correlation id is based on the conduit id prefix. This is the default so this case is vastly improved. CXF 3 is up to 5 times faster than CXF 2.7.10.

There is one down side still. If you use message id as correlation id or a user correlation id set on the cxf message then cxf 3 will switch back to consumer.receive and will be as slow as CXF 2 again.