topic Re: Can Qlik Replicate update/delete on Datalake? in Qlik Replicate

Can Qlik Replicate update/delete on Datalake?

ppeeradej — Mon, 19 Apr 2021 13:34:56 GMT

I have a requirement to do the CDC from AS400 to Azure Data Lake Storage (Azure ADLS)

There are 2 things I want to keep on Azure data lake storage

1. Change transaction

2. Data and structure same as Source endpoint (with realtime update) <-- realtime update to file on ADLS

I checked the Qlik document and found that if I choose the target end point as Azure Data lake Storage

There is a limitation for

UPDATE and DELETE statements are not supported in Apply Changes replication mode

If this solution cannot, what is alternative solution?

PS. I dont have environment to test but need to propose the solution very soon.

If anyone has suggestion, feel free to share.

Thanks

Re: Can Qlik Replicate update/delete on Datalake?

Barb_Fill21 — Fri, 17 Jun 2022 19:33:49 GMT

Hello @ppeeradej ,

It's true about UPDATE and DELETE statements are not supported in Apply Changes replication mode. You could use the Store Changes and use transactional apply and it should work for you. I have an existing cusomer is is doing this now.

https://help.qlik.com/en-US/replicate/May2022/Content/Replicate/Main/Microsoft%20Azure%20ADLS/azureadls_limitations_file_targets3.htm#ar_files_source_877478117_1474381

https://help.qlik.com/en-US/replicate/May2022/Content/Global_Common/Content/SharedEMReplicate/Customize%20Tasks/store_changes_settings.htm

Please let me know if this helps you !

Sincerely,

Barb

Re: Can Qlik Replicate update/delete on Datalake?

JonathanHoarau — Wed, 27 Sep 2023 14:24:13 GMT

Hello @ppeeradej ,

I am working with the ADLS endpoint.

Does the stored changes permit you the update and delete ?

Thank you