Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
This article expands on the Talend Knowledge Base (KB) article, Improve Camel Flow Monitoring, by demonstrating how to use the monitoring solution for routes created in Talend Studio, and how to minimize changes in the routes. It also provides a sample of how to extend the information written to the sinks by Karaf Decanter.
The article includes:
Read the Knowledge Base (KB) article Improve Camel Flow Monitoring
Basic Java knowledge
Experience building bundles for Talend Runtime using Maven
Opposite to the configuration file in the Improve Camel Flow Monitoring KB article, which is a Spring file, the one provided in the Java project is an OSGi blueprint file, as Spring is not supported in version 7.0. In addition, it registers the DecanterEventNotifier bean and Tracer bean as a service in the OSGi registry. When running in an OSGi environment, Camel includes the OSGi registry to lookup beans. So, registering the beans in the OSGi registry ensures they are found by the Camel components dealing with tracing and event notification. The Tracer bean is class provided by Camel allowing you to configure how the traces are logged. The Apache Camel documentation page, Camel Tracer, provides information about the available configuration properties.
Without having to build the project, you can copy the configuration file below, or use the decanter-camel.xml file attached to this article, into the deploy directory. The logLevel property of the Tracer bean is set to OFF, that disables tracing. If tracing is enabled, the property must have one of the following values: FATAL, ERROR, WARN, INFO, DEBUG, TRACE.
<?xml version="1.0" encoding="UTF-8"?> <blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"> <cm:property-placeholder persistent-id="org.talend.ps.decanter-camel"> <cm:default-properties> <cm:property name="camel.event.includeBody" value="false" /> <cm:property name="camel.event.includeProperties" value="true" /> <cm:property name="camel.event.includeHeaders" value="true" /> <cm:property name="camel.event.silent" value="true" /> <cm:property name="camel.trace.enabled" value="false"/> <cm:property name="camel.trace.includeBody" value="false"/> <cm:property name="camel.trace.includeProperties" value="true" /> <cm:property name="camel.trace.includeHeaders" value="true" /> </cm:default-properties> </cm:property-placeholder> <reference id="eventAdmin" interface="org.osgi.service.event.EventAdmin"/> <bean id="decanterEventNotifier" class="org.apache.karaf.decanter.collector.camel.DecanterEventNotifier"> <property name="eventAdmin" ref="eventAdmin"/> <!-- Optional to add custom properties to the event --> <!--<property name="extender" ref="customExtender" />--> <property name="includeBody" value="${camel.event.includeBody}"/> <property name="includeProperties" value="${camel.event.includeProperties}"/> <property name="includeHeaders" value="${camel.event.includeHeaders}"/> <property name="ignoreCamelContextEvents" value="true" /> <property name="ignoreRouteEvents" value="true" /> <property name="ignoreServiceEvents" value="true" /> <property name="ignoreExchangeEvents" value="false" /> <property name="ignoreExchangeCreatedEvent" value="false" /> <property name="ignoreExchangeCompletedEvent" value="false" /> <property name="ignoreExchangeFailedEvents" value="false" /> <property name="ignoreExchangeRedeliveryEvents" value="false" /> <property name="ignoreExchangeSendingEvents" value="${camel.event.silent}" /> <property name="ignoreExchangeSentEvents" value="${camel.event.silent}" /> </bean> <service id="eventNotifierService" ref="decanterEventNotifier" interface="org.apache.camel.spi.EventNotifier" /> <bean id="traceHandler" class="org.apache.karaf.decanter.collector.camel.DecanterTraceEventHandler"> <property name="eventAdmin" ref="eventAdmin"/> <!-- Optional to add custom properties to the event --> <!--<property name="extender" ref="customExtender" />--> <property name="includeBody" value="${camel.trace.includeBody}"/> <property name="includeProperties" value="${camel.trace.includeProperties}"/> <property name="includeHeaders" value="${camel.trace.includeHeaders}"/> </bean> <bean id="tracer" class="org.apache.camel.processor.interceptor.Tracer"> <property name="traceHandler" ref="traceHandler"/> <property name="enabled" value="${camel.trace.enabled}"/> <property name="traceOutExchanges" value="true"/> <property name="logLevel" value="OFF"/> </bean> <service id="tracerService" ref="tracer" interface="org.apache.camel.processor.interceptor.Tracer" /> </blueprint>
Decanter allows you to enrich the events generated by the Camel Collector through additional properties. It provides the DecanterCamelEventExtender interface, that must be implemented by a custom extender. An example of an implementation of an extender, is shown below:
public class ActivityMonitoringExtender implements DecanterCamelEventExtender{ @Override public void extend(Map<String, Object> decanterData, Exchange exchange) { String eventType = (String) decanterData.get("eventType"); if (ExchangeCreatedEvent.class.getName().equals(eventType)) { Endpoint ep = exchange.getFromEndpoint(); if (ep instanceof JmsEndpoint) { JmsEndpoint jmsEp = (JmsEndpoint) ep; extendForJmsEndpoint(decanterData, exchange, jmsEp); } else if (ep instanceof FtpEndpoint) { FtpEndpoint<?> ftpEp = (FtpEndpoint<?>) ep; extendForFtpEndpoint(decanterData, exchange, ftpEp); } } } ... }
To activate the extender, inject an instance of this extender into the DecanterEventNotifier Bean or the DecanterTraceEventHandler. An example of the relevant snippet of the blueprint configuration of the Java project, is shown below:
... <bean id="customExtender" class="org.talend.ps.monitoring.ActivityMonitoringExtender"/> <bean id="decanterEventNotifier" class="org.apache.karaf.decanter.collector.camel.DecanterEventNotifier"> <property name="eventAdmin" ref="eventAdmin"/> <!-- Optional to add custom properties to the event --> <property name="extender" ref="customExtender" /> <property name="includeBody" value="${camel.event.includeBody}"/> ...
Install a recent version of Maven, 3.3 or higher. To build the project, invoke the command mvn install from within the root directory of the project, where the pom.xml file is located. To install the project, the Decanter Camel Collector and at least one Decanter appender must be installed. For test purposes, the Decanter File appender is best suited. If you already have an Elasticsearch instance, you may use the Elasticsearch specific appenders. The Decanter documentation lists the available appenders and their configuration options. The installation must be executed in the console of the Talend Runtime. The required commands are shown below:
/_ __/__ _/ /__ ___ ___/ / / / / _ `/ / -_) _ \/ _ / /_/ \_,_/_/\__/_//_/\_,_/ (version 7.0.1) Hit '<tab>' for a list of available commands and '[cmd] --help' for help on a specific command. Hit '<ctrl-d>' or 'system:shutdown' to shutdown the TRUN. karaf@trun()> feature:repo-add decanter 2.0.0 Adding feature url mvn:org.apache.karaf.decanter/apache-karaf-decanter/2.0.0/xml/features karaf@trun()> feature:install decanter-collector-camel karaf@trun()> feature:install decanter-appender-file karaf@trun()>
Install the bundle by executing the following command in the runtime console, or copy the corresponding JAR file, camel-decanter-EventNotifier.zip attached to this article, to the deploy folder of the runtime installation.
install -s mvn:org.talend.ps/camel-decanter-eventnotifier/1.0.0-SNAPSHOT
Enable Studio routes for tracing and event notification. Figure 1 shows the sample routes from the monitored-routes.zip file, that is attached to this article. It includes a cConfig component where the CamelContext containing all the routes is retrieved, and the tracing property is set. Unfortunately, even if you're only interested in event notifications, tracing must be activated, and the corresponding tracer and traceHandling bean, in the blueprint file, must be configured. Otherwise, the Decanter collector throws exceptions instead of creating events.
Before using the sample routes, complete the properties for the FTP, ActiveMQ server, and the data directory.
A sample Decanter event created when running the above routes, is shown below:
{ "@timestamp": "2018-09-13T19:33:50,101Z", "hostName": "sopmac23", "camelContextName": "activitymonitoring.monitored_0_1.Monitored", "exchangePattern": "InOnly", "eventType": "org.apache.camel.management.event.ExchangeCreatedEvent", "type": "camelEvent", "shortExchangeId": "ID-sopmac23-1536867218901-0-2", "exchangeId": "ID-sopmac23-1536867218901-0-2", "jms": { "destinationName": "monitored", "destinationType": "queue", "messageID": "ID:sopmac23-56922-1536867229058-1:1:2:1:1" }, "routeId": "Monitored_cJMS_1", "inHeaders": { "breadcrumbId": "ID-sopmac23-1536867218901-0-1", "CamelFileAbsolute": false, "CamelFileAbsolutePath": "message/message_ID:sopmac23-51225-1536237210202-4:2:1:1:3.txt", "CamelFileLastModified": 1536493380000, "CamelFileLength": 44, "CamelFileName": "message_ID:sopmac23-51225-1536237210202-4:2:1:1:3.txt", "CamelFileNameConsumed": "message_ID:sopmac23-51225-1536237210202-4:2:1:1:3.txt", "CamelFileNameOnly": "message_ID:sopmac23-51225-1536237210202-4:2:1:1:3.txt", "CamelFileParent": "message", "CamelFilePath": "message/message_ID:sopmac23-51225-1536237210202-4:2:1:1:3.txt", "CamelFileRelativePath": "message_ID:sopmac23-51225-1536237210202-4:2:1:1:3.txt", "CamelFtpReplyCode": 226, "CamelFtpReplyString": "226 Transfer complete.\r\n", "JMSDeliveryMode": 2, "JMSExpiration": 0, "JMSMessageID": "ID:sopmac23-56922-1536867229058-1:1:2:1:1", "JMSPriority": 4, "JMSRedelivered": false, "JMSTimestamp": 1536867230089 }, "fromEndpointUri": "cMQConnectionFactory1://queue:monitored", "karafName": "trun", "hostAddress": "127.0.0.1", "properties": { "CamelExternalRedelivered": false }, "timestamp": 1536867230101, "inBodyType": "byte[]", "event_topics": "decanter/collect/camel/event" }
Note: Apache Decanter is officially supported starting v7.1.1. It has been introduced as a new feature with Event Monitoring.
Hello Team,
Attachments are missing.
@test1233 Thanks for pointing it out, we've uploaded the attachments to the article.
Regards
Shicong