https://community.qlik.com/t5/Suggest-an-Idea/No-transactional-integrity-on-Event-Hub-targets/idi-p/1858288 <P>We recently discovered that messages being sent to Microsoft Azure Event Hub targets are not maintaining transactional integrity.&nbsp; We opened a case with Qlik support and they confirmed this is the case and the documentation is incorrect.</P> <P>The internal support team gave us this option: "<SPAN>As discussed, your options are either to use 'cdcOnAirBatchesPerClient' &amp; 'cdcMaxClientsPerHub' internal parameters(with value=1) in order to keep only 1 batch on air during CDC, that way it should ensure that the order will be kept, however, it is a tradeoff with bad performance.</SPAN>"&nbsp; We tested those 2 internal parameters and they did address the transactional integrity issue but performance was terrible as we saw latency steadily grow and were soon hours behind.</P> <P>We would like Qlik to fix the Event Hub client so that transactional integrity is maintained.</P> <P>Microsoft suggested using the Kafka client as a work-around and we are currently using that method.&nbsp; Here is URL to the documentation.</P> <P><SPAN><A href="https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs" target="_blank">https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs</A></SPAN></P> <P><SPAN>This is not a Qlik supported method to send data to Event Hub but does work.&nbsp; Can Qlik certify and support this method of sending data to Event Hub.</SPAN></P> Thu, 11 Nov 2021 15:54:42 GMT tfarber 2021-11-11T15:54:42Z https://community.qlik.com/t5/Suggest-an-Idea/No-transactional-integrity-on-Event-Hub-targets/idi-p/1858288 <P>We recently discovered that messages being sent to Microsoft Azure Event Hub targets are not maintaining transactional integrity.&nbsp; We opened a case with Qlik support and they confirmed this is the case and the documentation is incorrect.</P> <P>The internal support team gave us this option: "<SPAN>As discussed, your options are either to use 'cdcOnAirBatchesPerClient' &amp; 'cdcMaxClientsPerHub' internal parameters(with value=1) in order to keep only 1 batch on air during CDC, that way it should ensure that the order will be kept, however, it is a tradeoff with bad performance.</SPAN>"&nbsp; We tested those 2 internal parameters and they did address the transactional integrity issue but performance was terrible as we saw latency steadily grow and were soon hours behind.</P> <P>We would like Qlik to fix the Event Hub client so that transactional integrity is maintained.</P> <P>Microsoft suggested using the Kafka client as a work-around and we are currently using that method.&nbsp; Here is URL to the documentation.</P> <P><SPAN><A href="https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs" target="_blank">https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-kafka-enabled-event-hubs</A></SPAN></P> <P><SPAN>This is not a Qlik supported method to send data to Event Hub but does work.&nbsp; Can Qlik certify and support this method of sending data to Event Hub.</SPAN></P> Thu, 11 Nov 2021 15:54:42 GMT https://community.qlik.com/t5/Suggest-an-Idea/No-transactional-integrity-on-Event-Hub-targets/idi-p/1858288 tfarber 2021-11-11T15:54:42Z https://community.qlik.com/t5/Suggest-an-Idea/No-transactional-integrity-on-Event-Hub-targets/idc-p/2057091#M12503 Tue, 04 Apr 2023 18:49:03 GMT https://community.qlik.com/t5/Suggest-an-Idea/No-transactional-integrity-on-Event-Hub-targets/idc-p/2057091#M12503 Nulee_Massaro 2023-04-04T18:49:03Z