topic Re: Replicate Kafka target with publicly-trusted TLS/SSL certificate: ca path? in Qlik Replicate

Replicate Kafka target with publicly-trusted TLS/SSL certificate: ca path?

brett — Tue, 09 Jun 2020 09:08:01 GMT

I have a question about using a Replicate Kafka target, which is protected with TLS/SSL with a publicly-trusted X.509 certificate - that is, one that your web-browser would trust if it were talking HTTPS, since the certificate is ultimately signed by a trusted root certificate. Example of such a service - Confluent Cloud Kafka cluster endpoints.

If I connect to this endpoint using e.g. openssl, on Linux this will load a standard set of trusted root cacerts, and the handshake will be trusted.

In Replicate Kafka target, if using TLS/SSL, the "CA file" field becomes mandatory - is there any option to avoid the need to locate a PEM-encoded certificate-chain, instead trusting via a local root-certificate store?

This question extends to using the excellent Replicate Test Drive - if I want to connect from that hosted environment to a Confluent Cloud TLS Kafka bootstrap/broker - is there any path on the server that will work for "CA path" which includes the trusted roots, or if not, can I upload one?

Re: Replicate Kafka target with publicly-trusted TLS/SSL certificate: ca path?

EyalSilner — Sun, 02 Aug 2020 15:49:58 GMT

In Replicate Kafka endpoint we use librdkafka as our client library. We truly and mandatory verify the existence of the CA file. The new librdkafka allows you to set the ssl.sa.location as probe and it allows you to use known CA cert paths.

Quoting librdkafka v1.5:

If OpenSSL is linked statically, or ssl.ca.location=probe is configured,
librdkafka will probe known CA certificate paths and automatically use the
first one found. This should alleviate the need to configure
ssl.ca.location when the statically linked OpenSSL's OPENSSLDIR differs
from the system's CA certificate path.

Currently, our compiled librdkafka is v1.3, which does not support this option.

I think this is a legitimate request and I suggest you add it as a "new idea" (in Community > Qlik Product Insight & Ideas), where the PM can gather votes and then consider pushing it into the work-plan. Eventually, we will upgrade our librdkafka to the most innovative and open this option for the users, but this "new idea" may promote it and may focus us on the list of recently supported features that our customers really need.

Re: Replicate Kafka target with publicly-trusted TLS/SSL certificate: ca path?

chrislarsen3 — Thu, 03 Sep 2020 20:27:48 GMT

To connect Qlik Replicate with Confluent Cloud, what you need to do is use the default root CA pem file for your openssl version next to the checked SSL box where it says CA Path.

You can get this by doing:

openssl  version -a

On linux boxes the path will be /etc/pki/tls/cert.pem

Make sure you have an up to date version of Attunity (Qlik Replicate) where you can specify SASL/PLAIN as the Authentication type.

The same path is used for Schema Registry.

Just transferred 1.8M rows in about 1 minute, works like a champ.

Chris