idea Re: QLIK Replicate Notification for Data Discrepancy - Status changed to: Open - Not Planned in Suggest an Idea

QLIK Replicate Notification for Data Discrepancy

DanielSol — Tue, 08 Jun 2021 12:25:27 GMT

If there is a way to compare the source data with target, that would be a good feature. Then notification can be sent to alert and investigate validation issue. This kind of data inconsistency notifications will help users to rapidly pinpoint the issue on QLIK Replication. It might be a bit difficult to make row by row comparison but at least comparing the total row count of individual tables enable users to check the overall consistency.

Re: QLIK Replicate Notification for Data Discrepancy

Carlo_P — Tue, 08 Jun 2021 21:28:55 GMT

We have built a separate process which we call counts and sums to compare record counts and perform basic aggregation on the tables from both systems based on the primary key and configurable criteria to ensure overall consistency and accuracy. Having this functionality built into replicate or compose would save people an enormous amount of QA time and give piece of mind that the tool is working as intended. I would love to have an option to run this on a scheduler in replicate or after a data load in Compose. I think this would also save a lot of support calls on your end as this would validate the data is actually making it all the way though properly.

Re: QLIK Replicate Notification for Data Discrepancy

DanielSol — Wed, 09 Jun 2021 15:08:40 GMT

Thanks Carlo_P. Are you using ETL tool for the separate process.

Re: QLIK Replicate Notification for Data Discrepancy

Carlo_P — Wed, 09 Jun 2021 16:39:50 GMT

Yes, our process is currently designed using SSIS but we are looking into using Microsoft power automate when we port it to snowflake.

Re: QLIK Replicate Notification for Data Discrepancy - Status changed to: Open - Not Planned

Shelley_Brennan — Mon, 14 Jun 2021 12:16:09 GMT

Data validation is not currently on the Replicate roadmap. However, there is some validation functionality available with Qlik Catalog - you may reach out to your account rep to learn if this would meet your needs. Several of our customers also prefer to do validation and reconciliation with their own scripting or "independent" 3rd party tools.

Re: QLIK Replicate Notification for Data Discrepancy - Status changed to: Closed - Archived

Meghann_MacDonald — Tue, 28 Mar 2023 17:34:01 GMT

Archiving due to age of idea and relevancy. If it is still needed please resubmit.

Meghann