Best architecture to process MQTT messages (statefull)

[Anonymous]

Hi,

this is my use case and I really don't know which is the best option between Talend ESB o just Data Integration.

I need to subscribe to a "wildcard" topic since I need to get all messages sent on specific channels. The topic name contains even the device which is sending the message, so I need to even process the topic string to get the "device id".

The message has a JSON payload with a three level status (1, 2, 3). I need to add a row with device id and status in a table for each message (ok this is the easy task). I would like even to keep the "last status" for a device and when I receive a new one message store for the previous status the triplet device id, status and time elapsed until the new status.

Of course I can use a Java job, there is no problem to write some code.

Ideas?

Thank you, Stefano.

[Anonymous]

Hello,

Please have a look at talend data mapper structure:TalendHelpCenterata mapping.

Best regards

Sabrina

[Anonymous]

The ESB Module is the best choice if you need to consume the MQTT messages in a permanent way (24/7) and we offer this with the cMQTT component in the Route Builder. For the JSON part we recommend to use our TDM module as said by my colleague before it allows you to handle also complex Json data, the related component is cMap (or via cTalendJob also tHMap in case you like to use more of our tXXX components with the route. )

HTH,

Dietmar

[Anonymous]

Thank you for all your answers. Actually with an cMQTT component I can connect and get the json body of the messages then processed with a job.

But a problem remains: can I transfer to the job even the topic associated with the received message? The topic contains the client id and I need a "generic" subscription for a variable set of clients all publishing to a topic like:

brand/[client_id]/status

Being able to get the topic I can enrich the json data received and store the relevant information with the client id.

Thank you!