topic Re: Talend and RabbitMQ in Talend Studio

Talend and RabbitMQ

Anonymous — Sat, 16 Nov 2024 11:04:59 GMT

In our current setup (where we are now introducing Talend) we have RabbitMQ already in place as our central event broker. We now want Talend to start subscribing to queues in RabbitMQ, but this wasn't as easy as I'd initially hoped.
It seems that the ConnectionFactory in Talend is using AMQP 1.0, while RabbitMQ is using AMQP 0.9. There is an experimental plugin for RabbitMQ that introduces AMQP 1.0, but I would rather have Talend use AMQP 0.9 than having experimental plugins in production. For obvious reasons
What is the best way to having Talend interacting with RabbitMQ?

Re: Talend and RabbitMQ

Anonymous — Wed, 26 Aug 2015 10:07:58 GMT

Hi darkhorse,
We have redirected your issue to ESB experts then come back to you as soon as we can.
Best regards
Sabrina

Re: Talend and RabbitMQ

Anonymous — Wed, 26 Aug 2015 12:56:00 GMT

Hi,
the best way in this case it to use the cMessageEndpoint component (Mediation Perspective in the Talend Studio) and to use the 'RabbitMQ'-component' (select 'rabbitmq' in the advanced tab of the cMessageEndpoint component - (add a new line to the table then you see a combobox which contains 'rabbitmq'. ) In this case you can access RabbitMQ via AMQP within a Talend-Route (the camel-rabbitmq component uses the RabbitMQ AMQP client).
The cAMQP component can only be used with an AMQP 1.0 compatible broker as you also mentioned / found out.
More information on options of the camel-rabbitmq component can be found also here: http://camel.apache.org/rabbitmq.html
Would be nice to get a short feedback if this has solved your questions and allowed you to progress on your tasks.
Thanks,
Dietmar

Re: Talend and RabbitMQ

_AnonymousUser — Sun, 18 Oct 2015 10:57:06 GMT

Hello dietmar,
Am using Talend ESB to publish messages to Rabbit mq.
earlier i was using components like cConnectionFactory,Ctimer,CAMQP,Clog.
In your prev comment i read about cMessagingEndpoint. i enabled it and the rabbitmqcomponnet.
Can you tell me now what all componnets to use now, and how to use.
I order to connect to rabbit mq, ill need to pass url, exchange name, routing key, data

Re: Talend and RabbitMQ

mourina — Mon, 07 Nov 2016 00:19:22 GMT

Hi,
My ESB open studio does not have mdiation perspective option. Also i cannot find CmessageInput component. Any pointer?
Thanks

Re: Talend and RabbitMQ

vapukov — Mon, 07 Nov 2016 01:40:16 GMT

what version number?
in 6.2. it named - Routes, component will be available when You create (open) new route

Re: Talend and RabbitMQ

mourina — Mon, 07 Nov 2016 04:14:59 GMT

Hi vapukov,
Thanks a lot.
So I have the job in place now. My URI looks like "rabbitmq://XXXXXXX:5672/Myqueue?username=myuser&password=mypass".
But when I run I get
Starting job rmqnew at 15:12 07/11/2016.
connecting to socket on port 3356
connected
And it pretty much stays like that until I kill it. Any idea?
Thanks

Re: Talend and RabbitMQ

vapukov — Mon, 07 Nov 2016 08:17:16 GMT

I not great expert in RabbitMQ, but I use different settings, You can see from 2nd picture - You use not full format

Re: Talend and RabbitMQ

mourina — Tue, 08 Nov 2016 02:12:07 GMT

Hi Vapukov,
You have been a lot of help so far. My module is working now. Problem was with my URI. I had to pass more parameters to match exactly with the exchange set up.
Now can anyone suggest the best way to dump these messages to a database?
Thanks,

Re: Talend and RabbitMQ

vapukov — Tue, 08 Nov 2016 03:02:44 GMT

Add runJob, and parse header and body to Job, inside Job as usual - parse Json, do anything what You need

Re: Talend and RabbitMQ

mourina — Tue, 08 Nov 2016 06:21:16 GMT

I have these 2 components now. I assume there is no need for a JSON parser as I want the whole payload to be loaded in Oracle. Again I am a bit stuck. Nothing gets passed on to the oracle table.

Re: Talend and RabbitMQ

vapukov — Tue, 08 Nov 2016 18:35:53 GMT

It work, how You pass body?

Re: Talend and RabbitMQ

mourina — Wed, 09 Nov 2016 09:30:00 GMT

Hey Vapukov,
Its working now . I am getting something like this in my oracle table

So now I have a tJsonextract component as I want all the tags in JSON payload to come as coumns in oracle table. Or atleast the ones which I define while creating the Oracle table. Can you advice what should be the idle configuration for this tjsonextract component.
I am a bit confused as I saw in your previous screen shot that you used the tjsonextract component directly. However in talend documentation for this component, there is a twritejsonfield before tjsonextract

Thanks

Re: Talend and RabbitMQ

Anonymous — Tue, 16 Oct 2018 13:28:49 GMT

Hello,

I am working on something quite similar but I don't manage to make it work ... I succeed in getting messages from rabbitmq, that means my cMessagingEndpoint component is well configured. But then i need to extract all informations contained in my message (Headers and Payload). I have built the same route and job as you but i still don't know how to configure these 3 components : cSetHeader, cProcessor, cExtractJsonFields.

You can find attached my current work.

Thanks.

Capture d’écran_2018-10-15_17-56-11.png
Capture d’écran_2018-10-15_17-56-40.png
Capture d’écran_2018-10-15_17-57-08.png
Capture d’écran_2018-10-15_17-57-22.png
Capture d’écran_2018-10-15_17-57-48.png

Re: Talend and RabbitMQ

Anonymous — Tue, 30 Oct 2018 07:44:25 GMT

Hello guys,

I'm still stuck on that purpose... noone knows how to manage that ?

Re: Talend and RabbitMQ

Anonymous — Mon, 12 Nov 2018 20:18:29 GMT

Hi, please check the below link,

https://community.talend.com/t5/Design-and-Development/Talend-ESB-amp-Rabbitmq-Explaining-way-to-consume-and-parse/m-p/140313#M86744