My requirement is to receive updates from devices scattered across the country and store the updates in a Mysql database. The devices can only send to a socket (3 times a day). And the devices are more than 10,000 so the TCP server should be resilient to accommodate the traffic. As a solution, I created a job in Talend with, among others, the following components: a) tWaitForSocket - Opens and listens to a TCP port b) tFixedFlowInput - for reading incoming message from the port c) tExtractDelimitedFields - to split the incoming TCP payload d) tMysqlOutput - to store the data Problem: a) If I do a simulation of many TCP requests the job crashes Error from the client:
java.net.SocketException: Software caused connection abort: recv failed at java.net.SocketInputStream.socketRead0(Native Method)
b) I finally deployed the solution on Talend Runtime. The feature is installed and running but after some days it stops responding. I can still telnet to the port (meaning the port is open) but the messages I sent are not received. I installed the log wrapper with "ALL" level of logging. The message string does not even appear on the log anymore. It's like it's not even being received. I have no idea how to explain this, considering the port is still open and there are no errors. If I am using the wrong approach please let me know because I'm certain this is a use case Talend can handle.
Hi, from an ESB point of view it would be likely better to use the Route Builder (Mediation Perspective) and setup a route which use the Netty4 for the TCP Socket stuff cMessageEndpoint (in teh Advanced Tab of the component add Netty4 to the table) and then specific the consumer URL as defined here: http://camel.apache.org/netty4.html and then either use JDBC or JPA stuff within the Route or even a cTalendJob with the Job Design you used already behind your tWaitforSocket. Running on a 24/7 with a listener is something which we recommend to use the Routes in ESB and to deploy them to the Talend Runtime for a permanent execution / listening use case. Would this be an option for you ? Dietmar
That sounds like an awesome idea. Let me work on it over the weekend and update
Update: cMessagingEndpoint is the way to go.
As advised by dlenzen I created a route in the mediation perspective using Netty4 then linked the route to the job I already had created.
Thanks dlenzen
