topic Re: routing MQ message to another Q in Talend Studio

routing MQ message to another Q

Anonymous — Sat, 16 Nov 2024 07:19:01 GMT

Hi, I have one scenario. I am reading messages from MQ passing to Standard job. Thru Talend Standard job, writing that message in file and then uploading file to S3. If processed/uploaded to S3 then it's happy path.

But if suppose, an error occurred while S3 connection or uploading to S3 in Standard job, I have to send this message to another error Queue which failed during S3 connection/upload process.

As I already consumed the message from RabbitMQ in route job, this message will go out from the MQ,

my question is, how I will send this same message to another error Q from same exchange/server.

Re: routing MQ message to another Q

Anonymous — Thu, 15 Nov 2018 11:29:32 GMT

Hi,

If you want to retain some data in memory during your job processing, you can use tHashInput/tHashOutput components.

Once you read the data from queue, you can send the payload to tHashOutput component and read it from there for all further processing.

If some component/ subjob is failing, you can always use On SubJob Error/ On Component Error to handle the error conditions and during this time also you can read from earlier Hash components.

Please make sure that you are clearing the Hash after each run of the job.

If the answer has helped you, could you please mark the topic as resolved? Kudos are also welcome 🙂

Warm Regards,

Nikhil Thampi

Re: routing MQ message to another Q

DoDo69 — Fri, 16 Nov 2018 10:14:36 GMT

Hello,

Maybe you can use dead letter queue (https://www.rabbitmq.com/dlx.html).

To do this, I think you should check the field "Use Transacted" on your tMomConnection and to consume your message you should commit it. And when an error occurs, I think to send it to the dead letter queue you should Rollback (not sure I don't have test it but it's the solution I have made with other ESB like Tibco BW).

Re: routing MQ message to another Q

Anonymous — Mon, 19 Nov 2018 10:25:02 GMT

@Donatien wrote:

Hello,

Maybe you can use dead letter queue (https://www.rabbitmq.com/dlx custom writing).

To do this, I think you should check the field "Use Transacted" on your tMomConnection and to consume your message you should commit it. And when an error occurs, I think to send it to the dead letter queue you should Rollback (not sure I don't have test it but it's the solution I have made with other ESB like Tibco BW).

Thank you for providing a detailed solution. To be honest I would never think of that and your experience here is very appreciated.