topic Re: Distributed Source Table to multiple target databases in Qlik Replicate

Distributed Source Table to multiple target databases

Al_gar — Tue, 03 Jun 2025 22:01:31 GMT

For a Replicate task using HANA source with trigger-based CDC, what is the best way to distribute the same source table across different target databases? Should we just create multiple tasks reading to the same HANA table source?

Qlik Replicate November 2024
Source: HANA S/4
Targets: SQL Server 2019

Re: Distributed Source Table to multiple target databases

john_wang — Wed, 04 Jun 2025 02:44:18 GMT

Hello @Al_gar ,

I'm not entirely sure how the distribution logic is defined, whether it's based on schema, primary key values, or some other criteria.

However, one practical approach is to use Log Stream to capture the changes in a single task, and write them to a staging area files. Then, you can create multiple downstream tasks that read from this staging task and distribute the data to the appropriate target databases.

This method offers better scalability and central control over data distribution logic, especially when multiple targets are involved.

Hope this helps.

John.

Re: Distributed Source Table to multiple target databases

Al_gar — Thu, 05 Jun 2025 17:33:57 GMT

Hi @john_wang ,

Thanks. Following Replicate documentation I got confused for a moment on step 3 "Duplicate the source endpoint and add it to the log stream staging task" because after duplicating the source endpoint connection the "Read changes from log stream staging folder" check box wouldn't show up:

However, when I tried to create a new endpoint connection the check box will show up:

After adding the source endpoint's details I was able to get the task created and running. Thanks.

Re: Distributed Source Table to multiple target databases

Al_gar — Thu, 05 Jun 2025 18:08:56 GMT

@john_wang, I forgot to ask if there's any limitation on the amount of tables that you can load on a single Log Stream task.

Re: Distributed Source Table to multiple target databases

john_wang — Mon, 09 Jun 2025 14:15:22 GMT

Hello @Al_gar ,

Personally I do not see such a limitation, in our customers there are than more several thousands tables were included in a single task.

Regards,

John.

Re: Distributed Source Table to multiple target databases

Dana_Baldwin — Mon, 09 Jun 2025 16:36:14 GMT

Hi @Al_gar

All I would add to what @john_wang has shared is that you will need more disk space for the audit files the more tables you have in the staging task.

Thanks,

Dana

Re: Distributed Source Table to multiple target databases

Al_gar — Tue, 10 Jun 2025 11:47:08 GMT

Thanks @john_wang and @Dana_Baldwin.

Re: Distributed Source Table to multiple target databases

john_wang — Tue, 10 Jun 2025 12:56:41 GMT

Thank you for your support! @Al_gar