The Service Activity Monitoring (SAM) component logs service calls made with the Apache CXF Framework to a database. Typical use cases are tracking of errors, creating statistics and providing a basis for business monitoring.
Let´s first have a look at a typical message exchange for a service call:
|
Each service call generates four events two on the client side and two on the server side. The monitoring solution logs each of these events to a central database and also correlates the events so they can be viewed in the context of the original service call.
Architecture
The Talend Service Activity Monitoring soltion consists of three parts
- Agent (sam-agent): Hooks into cxf on consumer and provider side to trace the messages sent and received. For each messages a monitorijng event is created and sent to the monitoring server
- Monitoring Server (sam-server): Offers a Monitoring Service where it listens for incoming monitoring events. Each event is stored into the database. Offers a rest based interface to browse and search events
- Monitoring GUI: Offers a web based user interface to browse and search monitoring events
While the Agent and Monitoring Server are part of the community edition and apache Licensed the Monitoring GUI will only be able for the Commercial product.
The following diagram shows how the components are deployed:
|
Event Structure in database
| Field | Type | Description |
|---|---|---|
| ID | bigint(20) | Stores the persistence id after commit |
| MESSAGE_CONTENT | longtext | SOAP message content |
| EI_TIMESTAMP | datetime | Timestamp of the event creation |
| EI_EVENT_TYPE | varchar | EventType is an enumeration. Values: REQ_IN; REQ_OUT; RESP_IN; RESP_OUT; FAULT_IN; FAULT_OUT |
| ORIG_CUSTOM_ID | varchar | Custom id is an empty field. It is not used by Agents currently |
| ORIG_PROCESS_ID | varchar | Process id is the OS process id |
| ORIG_HOSTNAME | varchar | Hostname of the event creator |
| ORIG_IP | varchar | IP address of the event creator |
| MI_PORT_TYPE | varchar | Service port type of event creator |
| MI_OPERATION_NAME | varchar | Service operation name of event creator |
| MI_MESSAGE_ID | varchar | Unique id for one message. (UUID) |
| MI_FLOW_ID | varchar | Unique id for the message flow. (UUID) All events with the same id belong together |
| MI_TRANSPORT_TYPE | varchar | Transport type of event creator |
| ORIG_PRINCIPAL | varchar | Principal info in the message header |
| CONTENT_CUT | tinyint | Flag if event content was cut |
Additionally to the fixed event structure the EVENTS_CUSTOMINFO table holds name/value pairs the customer can freely define.
| Field | Type | Description |
|---|---|---|
| ID | bigint | id |
| EVENT_ID | bigint | ID of the event the custom info belongs to |
| CUST_KEY | varchar | custom property's key, e.g. Application name |
| CUST_VALUE | varchar | custom property's value, e.g. Dummy App |
Filters and Handlers
On the Agent as well as on the Monitoring Server events can be filtered. An event filter is defined by an interface and decides if an event should be processed or discarded. There are already several filters available and new ones can be created by the customer by implementing the interface. Among the available filters the most powerful is a JXPath filter that can be used to filter events by any meta data. Handlers are processed after the filters and can change the events. Typical applications of handlers are hiding Passwords or Cutting off Attachments.
Try it yourself
You can download the Talend ESB Community Edition and try the Service Activity Monitroing yourself. The gettings started guide in the documentation will help with the setup.
http://de.talend.com/products-application-integration/application-integration-esb-se.php
Talend recently released the new Talend ESB Standard Edition product that is based on many Apache projects (Apache CXF, Apache Camel, Apache Karaf). It allows users to build integrations and services using the reliable and wide spread Apache projects while extending these projects with several enterprise features:
- Service Locator - provides automatic and transparent failover and load balancing between service Consumers and Providers through a newly developed extension that allows for dynamic endpoint registration and lookup via Apache Zookeeper
- Service Activity Monitoring - facilitates the capture of analysis of service activity, including service response times, traffic patterns, auditing and more, by capturing events and storing information
- Security Token Service (STS) Framework - supports Security Assertion Markup Language 2.0 (SAML 2.0) to federate security credentials. This allows clients and services to securely and transparently authenticate during connections without the need for custom coding
The product is Apache Licensed so users get great flexibility to use and even enhance the code.
While this already gives a lot of value to our customers there is much more to come in the near future. The big news is the unified platform that will come with the enterprise edition of the ESB. In the future all Talend products will use a common Development, Monitoring and Runtime Platform.
See also:
http://de.talend.com/products-application-integration/application-integration-esb-se.php