topic Replicate for datalake in Qlik Replicate

Replicate for datalake

suvbin — Fri, 10 Apr 2020 12:21:13 GMT

Hi,

Is there any option, to maintain the historical data from source to datalake using attunity replicate. I know that Qlik also has attunity compose for datalake. But instead of taking another tool, is there any option to maintain historical data in replicate tool itself.

Instead of creating the change tables, the main tables itself should maintain the history of the record for past some days , say example : past 7 days data.

If so please let me know. Appreciate your help.

Thanks,

Deepthi.

Re: Replicate for datalake

Ola_Mayer — Fri, 10 Apr 2020 13:29:01 GMT

Hello,

Replicate is designed for capturing changed records from the source and delivering it to the target.

As you said your self, we can deliver the data in change tables or in an audit stream.

But to organize it in into historical consumable records, you need ETL post processing.

This can be automated with Compose or even home grown scripting.

Re: Replicate for datalake

suvbin — Fri, 10 Apr 2020 14:25:12 GMT

Thank you for the response.

The scripting can we write in replicate for maintain historical data. If so where it can be written in replicate.

Can you please be more detail on this.

Thank you.

Re: Replicate for datalake

Ola_Mayer — Fri, 10 Apr 2020 18:41:58 GMT

Replicate is designed to deliver data from point to point with maximum speed and minimum impact on the sources.

Unless transformation can happen in-line, the scripting should be written outside of Replicate.

Re: Replicate for datalake

stevenguyen — Tue, 14 Apr 2020 14:27:49 GMT

I do not see any feature to store history of the record for ADLS.

This would be consider as Store change tables.