Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
We need to read data published to Azure Event Hub in real-time. Which Talend component is best suited to fulfill this requirement?
I've not done this, however from what I understand Azure Event Hub is compliant with AMQP (https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-amqp-protocol-guide). As such you can use Talend ESB and the cAMQP component.
I am trying to use cAMQP component with cConnectionFactory. However, I am running into connection issue due to firewall blocking the port (Connection refused). Per Azure documentation, Event Hubs support AMQP over websockets. Is there a way to connect to Event Hub using AMQP over websockets, so that we don't have to open up firewall for AMQP specific ports for TCP?
I'm afraid I am not an expert in this and have never tried. However Apache Camel do provide a Websocket component. Maybe this will be of use to you
https://camel.apache.org/websocket.html
Thank you for your feedback. I am working on opening firewall, as we do not have Talend experts who can help create custom components that can communicate via AMQP protocol over Camel websockets. I was hoping that with so many standard components available in Talend, we would have a standard component to connect to Event hub, which is similar to Service Bus. I tried using MQconnectionFactory in combination with cAMQP component on a machine without firewall restriction and I keep getting amqp authorization failure. The configuration on Event Hub seems sufficient, as the same SAS credentials work on a .Net stub provided by Microsoft. Its surprising that there are several examples of using different components, but I could not find a single example explaining connecting Talend to Azure Event Hub. Has anyone ever successfully connected to Azure Event Hub using Talend? If so, can you please advise if there are any other components or pre-requisites needed to get it to work?
I followed example on Talend documentation site (https://help.talend.com/reader/Cu21OB3bCNfqp6r4JCCNzQ/kkt0YENIDNzdc21~v50_1w) and used below values for connection factory
MQ Server : AMQP 1.0
Host: "aaaa-eventhub.servicebus.windows.net/;SharedAccessKeyName=<name>;SharedAccessKey=<SASprimaryKey>"
Port: 5671 (tried both 5671 and 5672 as number and string)
You do not need to create a new component to use the Apache Camel components. All you need to do is to use the cMessagingEndpoint component. The Apache Camel components are simply configured using URIs. This document explains it...
https://help.talend.com/reader/M0piPAcrxN~73Yo0IsrZtQ/_F6Lk7R_Q5b5k3zruFp~6A
With regard to shortage of examples, I can request that an example is written. But I do not believe it will be produced immediately. If you manage to get this working, then I welcome an example from you which I will happily publish with you as the author 🙂
Sorry I can't be of immediate help, but I will look at getting a demonstration produced