topic Re: Data Purging with Qlik compose or Qlik Replica in Qlik Replicate

Data Purging with Qlik compose or Qlik Replica

SUJITH123 — Wed, 15 Mar 2023 07:52:35 GMT

Hi How can we do Data Purging with Qlik Replicate or Qlik compose, we only required data for the last 3 year , rest of the data needs to get deleted, Can we automate the process specified to some date value

Re: Data Purging with Qlik compose or Qlik Replica

john_wang — Wed, 15 Mar 2023 08:29:16 GMT

Hello @SUJITH123 ,

Thanks for you opening the article.

Not very sure in which stage you want to apply the filter, you may define the filter on table level or global level (acts for all tables) during the data replication , in both Full Load stage and/or CDC stage.

However if the data loaded into target database already and you just want to do housekeeping jobs then you need not Replicate as it's a pure DBA or SQL operations.

Hope this helps.

Regards,

John.

Re: Data Purging with Qlik compose or Qlik Replica

SUJITH123 — Wed, 15 Mar 2023 11:59:18 GMT

Thank you John !

we were trying to purge data from the Data Mart level, as you mentioned ,not sure if it is possible with Qlik compose .our Target is loaded with data and we want it to be like ,the data for the last 3 year and rest needs to get delete.

For this scenario is it like pure DBA or SQL operations ? or do we have any other options in the Qlik compose level

Re: Data Purging with Qlik compose or Qlik Replica

SUJITH123 — Wed, 15 Mar 2023 12:00:36 GMT

Like After every day the data get refreshed automatically and only maintain the last 3 year data

Re: Data Purging with Qlik compose or Qlik Replica

shashi_holla — Wed, 15 Mar 2023 14:41:40 GMT

@SUJITH123

We can create Post Loading ETL scripts in Compose DataMart and provide the delete statements to delete records which are older than 1095 days.

Thank you,

Re: Data Purging with Qlik compose or Qlik Replica

SUJITH123 — Thu, 16 Mar 2023 05:19:39 GMT

Thank you John!!