Replicate for datalake

[suvbin]

Hi,

Is there any option, to maintain the historical data from source to datalake using attunity replicate. I know that Qlik also has attunity compose for datalake. But instead of taking another tool, is there any option to maintain historical data in replicate tool itself. 

Instead of creating the change tables, the main tables itself should maintain the history of the record for past some days , say example : past 7 days data.

If so please let me know. Appreciate your help.

Thanks,

Deepthi.

[Ola_Mayer]

Hello,

Replicate is designed for capturing changed records from the source and delivering it to the target.

As you said your self, we can deliver the data in change tables or in an audit stream.

But to organize it in into historical consumable records, you need ETL post processing.

This can be automated with Compose or even home grown scripting.

[suvbin]

Thank you for the response. 

The scripting can we write in replicate for maintain historical data. If so where it can be written in replicate.

Can you please be more detail on this.

Thank you.

[Ola_Mayer]

Replicate is designed to deliver data from point to point with maximum speed and minimum impact on the sources.

Unless transformation can happen in-line, the scripting should be written outside of Replicate.

[stevenguyen · Accepted solution]

I do not see any feature to store  history of the record for ADLS.

This would be consider as Store change tables.