How to enable debug level testing endpoint connections?

[danielrf]

Hi,

I am moving Qlik Replicate form our current server to another linux which is a clean server. So now i need to reinstall and test all the drivers for databases and endpoints.

When i test my old working endpoint connections for Kafka  as target, i get an error but all I see is this message:

Failed to get Kafka metadata: Local: Broker transport failure. _TRANSPORT

Which is very broad and unespecific,

So, How can i enable a more verbose error message testing enpoint connections?

thanks.

[SachinB · Accepted solution]

Hello @danielrf ,

Thanks for contacting the Qlik community forum.

To enable debug level logs for testing Kafka endpoints, set the logging parameter to verbose. Then, click on "Test Connection" to initiate the test. After the test, click on "View Logs" in the same window to check the repsrv.log files for debug level messages

Hope this helps you.

Regards,

Sachin B

[danielrf]

Thanks!

Now i am able to see in more detail why the endpoinnt conection is not working....

Is weird, i have broker servers in place , but the log says I am not:

00420926: 2024-07-29T11:23:01:792443 [INFRASTRUCTURE  ]T:  Syntax object for 'KafkaSyntax' provider was not found in the repository  (dbprops.c:424)

00420926: 2024-07-29T11:23:01:792466 [INFRASTRUCTURE  ]T:  The 'KafkaSyntax' provider syntax was not found in the repository.  (dbprops.c:1160)

00420926: 2024-07-29T11:23:01:792473 [INFRASTRUCTURE  ]T:  The default 'KafkaSyntax' provider syntax will be used instead.  (dbprops.c:1166)

00420926: 2024-07-29T11:23:01:807199 [SERVER          ]I:  Target endpoint 'Kafka' is using provider syntax 'KafkaSyntax'  (provider_syntax_manager.c:955)

00420926: 2024-07-29T11:23:01:841995 [SERVER          ]T:  RDKAFKA - 5 - CONFWARN: [thrd:app]: No `bootstrap.servers` configured: client will not be able to connect to Kafka cluster  (kafka_client.c:244)

00420926: 2024-07-29T11:23:11:843709 [SERVER          ]T:  Failed to get Kafka metadata: Local: Broker transport failure. _TRANSPORT [1020401]  (kafka_client.c:437)

[john_wang]

Hello @danielrf ,

I'm not sure if this is a random error or if it has been persistent for a while. It seems to be more of a Kafka server or connectivity issue.

To troubleshoot, you might want to run the "ping" command from the Qlik Replicate server machine to check for any random connection issues. Alternatively, you could try replacing the hostname with the IP address in the Endpoint settings to see if that makes a difference.

In the past, we've experienced similar errors caused by a DNS server that wasn't working consistently.

Hope this helps.

Best regards,
John.

[danielrf]

Hi,

This behaviour is persistent 100% of the times. The thing is I am doing a migration from an old server where all the connectors and tasks are running for years. On the new server i installed the version 2023.11 to fix this issue: product issue (RECOB-8021)

Case # 00156465

[john_wang]

Hello @danielrf ,

Thank you so much for the sharing. It's very helpful for all of us.

I'm pasting the entire fix information:

Jira issue       : RECOB-8021
Salesforce case  : 108548
Type             : Issue
Component/Process: Kafka
Description      : Tasks using the Kafka target endpoint would consume excessive memory when capturing the REAL data type and using JSON message format.

Regards,

John.

[danielrf]

Hi , now i did a test using a kafka cli client and using the same configuration as Qlik Replicate enpoint, from the console i am able to list topics but not working from Replicate.

]$ ./kafka-topics.sh --list --bootstrap-server broker-0-kafka-east-sit:9093 --command-config ssl.properties | more
cdc.fin.jdetest.attrep_apply_exceptions
cdc.fin.jdetest.crpctl.f0004
cdc.fin.jdetest.crpctl.f0005

[john_wang]

Hello @danielrf ,

 Did you run the Kafka client program from Replicate Server machine?

thanks,

John.

[danielrf]

Hi ,

Yes the test was done from the same new server. And i was able to connect and list the topics with the console kafka client with the same certificates as from qlik replicate.

[john_wang]

Hello @danielrf ,

Thank you for the clarification. Would you please open a support ticket with below information:

1- The Replicate task Diag Package: we need to check your Kafka endpoint configuration; And also the "repsrv.log" which will record the endpoint test failure information there.

2- The information you used in the Kafka client: we need to compare to Replicate endpoint.

Our support team will be more than happy to assist you.

Regards,

John.