topic Re: Qlik Replicate - Databricks Query in Qlik Replicate

Qlik Replicate - Databricks Query

RPANICHER — Thu, 04 Jul 2024 22:36:26 GMT

Hi Team,

we are using Qlik Replicate & Qlik Compose to replicate the data from Oracle to Databricks Delta lake.

We have a unique problem with one of our tables.

A record that was last updated in the source table more than a year back keeps getting updated by Qlik Replicate multiple times daily in the databricks tables.

Has any of you encountered this issue before?

Any suggestions will be highly appreciated.

Kind Regards

Re: Qlik Replicate - Databricks Query

john_wang — Fri, 05 Jul 2024 00:33:01 GMT

Hello RP, @RPANICHER

Welcome to Qlik Community forum and thanks for reaching out here!

It appears that a specific record was updated or an event triggered a DML operation in the Oracle source database. To resolve this issue, please follow these steps:

1- Set SOURCE_CAPTURE and TARGET_APPLY to Verbose in the task logging configuration

2- Add the internal parameter keepCSVFiles to the Databricks Delta target endpoint and set its value to TRUE.

3- Decrypt the newly generated task log files

After the issue is reproduced, set the logging level back to the default. Then, examine the CSV files (under Qlik Replicate Server task subfolder) and task log files (e.g., by searching for the specific record's primary key values or distinctive strings) to identify the root cause.

Hope this helps.

John.

Re: Qlik Replicate - Databricks Query

RPANICHER — Wed, 10 Jul 2024 09:59:54 GMT

Hi John,

Thx for your response.

We had raise a case with Qlik Support and their recommendation was to install the latest patch for Qlik Replicate.

This was the below statement from Qlik Support

First from reviewing the diagnostic pakcage I see that you are running Replicate 2023.5 PR02. In Pr03 there was a fix to a product issue (RECOB-7458) that potentially was causing missing events in the Databrick endpoint target.

To eliminate the effect of this product issue please upgrade to latest patch release of 2023.5 (PR06).

Kind Regards

Re: Qlik Replicate - Databricks Query

john_wang — Wed, 10 Jul 2024 10:06:17 GMT

Thank you for the update @RPANICHER , please monitor the status and let us know if the problem is solved completely.