Qlik Replicate features

[AKJ]

Hi, I'm new to Qlik Replicate. I need to work with Qlik Replicate on below aspects. Can anyone let me know if they are possible and how?

1. Track schema changes using Qlik Replicate
2. How to implement CDC using logs?
4. As an end user how can I check, which all records are updated/Inserted/Soft deleted from the source table. Do we have any flag metadata for each of the DML process?
5. What are the different kind of transformations that I can apply in Qlik Replicate? 
Does it support complex window partitions?
6. How can I implement SCD2 using Qlik Replicate?
6. How can I execute Stored Procedure, Python scripts in Qlik Replicate? Is it possible?
7. If I need to execute QR pipelines using external scheduler, how can I do that ?

Any help here would be appreciated!

[john_wang · Accepted solution]

Hello @AKJ ,

Welcome to community again!

1. Track schema changes using Qlik Replicate

      Replicate supports DDL changes capture and apply on many databases, however it depends on source database types and target database types.

2. How to implement CDC using logs?

      Qlik Replicate is log-based CDC product (a few exceptions), it captures changes from database's transaction logs (TLOG in SQL Server, Redo LOG in Oracle, Journal in DB2i, BinLog in MySQL, OPLOG in MongoDB etc...) and apply the changes to target database/data warehouse/ message streaming systems. Only simple configurations steps (no coding needed at all!) in GUI then the job done easily!

3. As an end user how can I check, which all records are updated/Inserted/Soft deleted from the source table. Do we have any flag metadata for each of the DML process?

      Counting the rows number after full load done is a commonly used method. Certainly, there are other options as well.

4. What are the different kind of transformations that I can apply in Qlik Replicate? 

      In Replicate there are different granularity in transformations, in column level, table level, and global level. Global level acts on all tables within the same task. The transformation can be used in data, or metadata etc.

5. Does it support complex window partitions?

      Are you meaning Windows cluster installation ? Yes, it's supported.

6. How can I implement SCD2 using Qlik Replicate?

      Good question! I think Qlik Compose can be used to implement SCD2 in data warehouse.

7. How can I execute Stored Procedure, Python scripts in Qlik Replicate? Is it possible?

      Yes , or No. Depends what, when- and where you want to execute them, it's hard to tell.

8. If I need to execute QR pipelines using external scheduler, how can I do that ?

      Are you meaning start/stop Qlik Replicate tasks by using external scheduler? Yes, it can be done by call executable programs by command line , or do it by REST API calls.

BTW, I strongly suggest you contact Qlik Sales Team to get further assistance if you need have doubts/concerns still, feel free to let us know. The sales team may conduct PoC, or live demo to answer your questions. 

Regards,

John.

[AKJ]

Thanks John Wang!

Can you please give more details on this ? 

" 7. How can I execute Stored Procedure, Python scripts in Qlik Replicate? Is it possible?

      Yes , or No. Depends what, when- and where you want to execute them, it's hard to tell. "

Will I be able to use a stored procedure or a python transformation code inside QR and ask Qlik Replicate to trigger it before loading the data in target warehouse/file?

[john_wang · Accepted solution]

Hi @AKJ ,

Thanks for the update.

Although Qlik Replicate provides rich pre-defined transformations, sometimes we still need extra functionality not provided by Replicate's built-in transformations, user can develop their own transformations, Replicate supports User-defined transformations , and also support similar function Using external credentials.

BTW, 

(1) The user-defined transformations should be written using C language, I'm afraid Python code is not supported at present. Unless you call the Python code by C code, and the C code being called by Replicate (in theory).

(2) For file target, besides the transformation, there is post-process mechanism, Replicate may call batch program (on Windows) or shell script (on Linux) to process the generated files automatically after their generation.

Good luck,

Regards,

John.

[AKJ]

Thanks John!