topic Re: Qlik integration with Commvault in Qlik Replicate

Qlik integration with Commvault

adg_r2d2 — Thu, 14 Mar 2024 16:02:29 GMT

I'm transferring data from SAP (DB2) to S3.
My DBA is asking:

Would Qlik Replicate support integration with Commvault (backup tool) so Qlik retrieves archive log directly from backup instead of log files locally from server?

If so could you send me the steps to set it up?

Best regards,

Re: Qlik integration with Commvault

aarun_arasu — Thu, 14 Mar 2024 18:30:42 GMT

Hello @adg_r2d2 ,

Good day!

Thank you for creating a community post based on my request.

Upon checking with the internal team, it was mentioned that this cannot be achieved with the DB2 endpoint. They do not read the logs directly; instead, they use an API or UDTF to read the logs.

Hope this helps.

Regards,
Arun

Re: Qlik integration with Commvault

PGN — Thu, 14 Mar 2024 18:45:34 GMT

I did a ditto, but in my case it's Microsoft SQL Server. Can this be done with that endpoint source?

Re: Qlik integration with Commvault

aarun_arasu — Thu, 14 Mar 2024 19:06:42 GMT

Hi @PGN ,

Yes you are right , this can be done in MS sql.

https://help.qlik.com/en-US/replicate/November2023/Content/Replicate/Main/SQL%20Server/SQLServerDBSource_AdvProps.htm#ar_sqlserverds_1727505754_1669928

Unfortunately , In DB2 endpoint this cannot be achieved.

Regards

Arun

Re: Qlik integration with Commvault

john_wang — Fri, 15 Mar 2024 01:41:30 GMT

Hello @adg_r2d2 , @PGN ,

While Qlik Replicate capturing changes from database transaction log files (TLOG in SQL Server, Redo LOG in Oracle, Journal in DB2i, BinLog in MySQL, OPLOG in MongoDB ...), in general there are 2 typical technologies:

1- Via calling the database itself APIs to get changes from transaction logs. For example in Oracle source endpoint, the Oracle LogMiner method; The default method in DB2 LUW , and SQL Server.

IN this method, the transaction log files location are determined by the database itself, unless the folder can be changed within the database management.

For DB2 LUW source endpoint, db2ReadLog API function is used to retrieve the transaction logs. I'm afraid there is no way to change the transaction log locations by manual, or 3rd party tools eg Commvault (backup tool), at least there is no input parameters in the API to specify the alternative transaction log files location.

2- Qlik Replicate unique technology rather than the database built-in methods

In this method, Qlik Replicate capture changes by read the transaction log files directly, the transaction log files location can be different than the database transaction log files preconfigured path. Only a few of source endpoint support this method, eg Oracle source endpoint Replicate Log Reader, Alternate backup folder in SQL Server.

Hope this helps.

John.