Talend ESB Service Activity Monitoring

Skip to end of metadata
Go to start of metadata

The Service Activity Monitoring (SAM) component logs service calls made with the Apache CXF Framework to a database. Typical use cases are tracking of errors, creating statistics and providing a basis for business monitoring.

Let´s first have a look at a typical message exchange for a service call:

Full Size
A Gliffy Diagram named: GeneratedEvents

Each service call generates four events two on the client side and two on the server side. The monitoring solution logs each of these events to a central database and also correlates the events so they can be viewed in the context of the original service call.

Architecture

The Talend Service Activity Monitoring soltion consists of three parts

  • Agent (sam-agent): Hooks into cxf on consumer and provider side to trace the messages sent and received. For each messages a monitorijng event is created and sent to the monitoring server
  • Monitoring Server (sam-server): Offers a Monitoring Service where it listens for incoming monitoring events. Each event is stored into the database. Offers a rest based interface to browse and search events
  • Monitoring GUI: Offers a web based user interface to browse and search monitoring events

While the Agent and Monitoring Server are part of the community edition and apache Licensed the Monitoring GUI will only be able for the Commercial product.

The following diagram shows how the components are deployed:

Full Size
A Gliffy Diagram named: Deployment

Event Structure in database

Field Type Description
ID bigint(20) Stores the persistence id after commit
MESSAGE_CONTENT longtext SOAP message content
EI_TIMESTAMP datetime Timestamp of the event creation
EI_EVENT_TYPE varchar EventType is an enumeration. Values: REQ_IN; REQ_OUT; RESP_IN; RESP_OUT; FAULT_IN; FAULT_OUT
ORIG_CUSTOM_ID varchar Custom id is an empty field. It is not used by Agents currently
ORIG_PROCESS_ID varchar Process id is the OS process id
ORIG_HOSTNAME varchar Hostname of the event creator
ORIG_IP varchar IP address of the event creator
MI_PORT_TYPE varchar Service port type of event creator
MI_OPERATION_NAME varchar Service operation name of event creator
MI_MESSAGE_ID varchar Unique id for one message. (UUID)
MI_FLOW_ID varchar Unique id for the message flow. (UUID) All events with the same id belong together
MI_TRANSPORT_TYPE varchar Transport type of event creator
ORIG_PRINCIPAL varchar Principal info in the message header
CONTENT_CUT tinyint Flag if event content was cut

Additionally to the fixed event structure the EVENTS_CUSTOMINFO table holds name/value pairs the customer can freely define.

Field Type Description
ID bigint id
EVENT_ID bigint ID of the event the custom info belongs to
CUST_KEY varchar custom property's key, e.g. Application name
CUST_VALUE varchar custom property's value, e.g. Dummy App

Filters and Handlers

On the Agent as well as on the Monitoring Server events can be filtered. An event filter is defined by an interface and decides if an event should be processed or discarded. There are already several filters available and new ones can be created by the customer by implementing the interface. Among the available filters the most powerful is a JXPath filter that can be used to filter events by any meta data. Handlers are processed after the filters and can change the events. Typical applications of handlers are hiding Passwords or Cutting off Attachments.

Try it yourself

You can download the Talend ESB Community Edition and try the Service Activity Monitroing yourself. The gettings started guide in the documentation will help with the setup.

Contact

mail: chris_at_die-schneider.net

Subscribe

RSS Feed

Follow me on twitter: @schneider_chris

Popular Labels

My colleagues at Talend

Talend Community Coders

Blog Archive

  1. 2014
    1. March
    2. January
  2. 2013
    1. December
    2. November
    3. February
  3. 2012
    1. November
    2. July
    3. April
    4. March
    5. January
  4. 2011
    1. December
    2. November
    3. October
    4. September
    5. August
    6. June
    7. May
    8. April
    9. February
    10. January
  5. 2010
    1. September
    2. June
    3. February
  6. 2009
    1. September
  7. 2008
    1. December
    2. October
    3. September
    4. August




Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.