topic Re: Adding a child taks to a logstream setup in Qlik Replicate

Adding a child taks to a logstream setup

GerardQ — Wed, 14 Feb 2024 11:09:07 GMT

My scenario: one logstream task and already some child tasks consuming from the that logstream.

Suppose I want to add another child task, how do I do a full load only for this new task and in CDC mode have it synchonized with the ongoing logstream at the right point in time. Is it all handled automatically by Qlik Replicate?

The same would happen in case for any incident I need to do a full reload of one of the existing tasks.

Re: Adding a child taks to a logstream setup

Yadunandan — Wed, 14 Feb 2024 11:18:24 GMT

Hello @GerardQ

When you add another child task, configure it as Full load + CDC. Once you start the task it will perform a full load first and then start the CDC. Full load is performed from querying directly to the source database, once it is complete your data will be up-to-date with the source database till the time when Replicate has completed the Full load, going forward you will have real time data Replicated to the target reading from the staging task, so it will be in synch with the ongoing logstream task.

Hope this helps.

Regards,
Yadunandan

Re: Adding a child taks to a logstream setup

aarun_arasu — Wed, 14 Feb 2024 11:28:50 GMT

Hello @GerardQ ,

Thanks for reaching Qlik community

Please refer to the below article for more insight

https://community.qlik.com/t5/Official-Support-Articles/How-to-add-a-table-to-logstream-task-on-Qlik-Replicate/ta-p/1973596

Regards

Arun

Re: Adding a child taks to a logstream setup

Yadunandan — Wed, 14 Feb 2024 11:31:06 GMT

Hello @GerardQ ,

If our response has been helpful, please consider clicking "Accept as Solution". This will assist other users in easily finding the answer.

Regards,
Yadunandan

Re: Adding a child taks to a logstream setup

aarun_arasu — Wed, 14 Feb 2024 11:37:46 GMT

Hello @GerardQ ,

Well, to answer your question, yes, Replicate will automatically handle that. So, let's say you are creating a new child task with full load and CDC. When you start the task, it will perform a reload on the child task. Once the reload is completed, it will start reading the CDC events from your logstream task, starting from the time when the reload started. Qlik Replicate is designed in such a way that it is capable of doing all this on its own, and there is no manual intervention required.

Regards,

Arun

Re: Adding a child taks to a logstream setup

GerardQ — Wed, 14 Feb 2024 12:49:55 GMT

Thanks, that answers my question.

Regards, Gerard

Re: Adding a child taks to a logstream setup

deepaksahirwar — Wed, 14 Feb 2024 16:18:01 GMT

Dear @GerardQ ,

Welcome and Thank you for reaching out to us on Qlik Community.

Yes, Qlik Replicate can handle the scenario you described. Here’s how you can add a new child task and synchronize it with an ongoing logstream:

Adding a New Child Task: When you add another child task, configure it as Full Load + CDC. Once you start the task, it will perform a full load first and then start the CDC. The full load is performed by querying directly from the source database.Once it is complete, your data will be up-to-date with the source database till the time when Replicate has completed the Full Load. Going forward, you will have real-time data replicated to the target reading from the staging task, so it will be in sync with the ongoing logstream task.

Full Load for a New Task in CDC Mode: During a Full Load and CDC task combined, the CDC actually starts before the full load and caches changes read from the source transaction log. As the full load completes, the task is able to apply these cached changes in a way that does not produce errors such as duplicate keys for insert and records not found for the update.

Handling Incidents Requiring Full Reload of an Existing Task: If you need to perform a full reload of one of the existing tasks, you can stop the replicate task the table/tables are in. Then run your CDC to make sure you get any current __ct records processed (as a full load of a table clears that) From the Replicate Task Monitor tab, pick the tables you want to reload and click the Reload button. This will trigger a reload on the added table but not to other tables.

I hope this helps! Let me know if you have any other queries.

If our response has been helpful, please consider clicking "Accept as Solution". 
This will assist other users in easily finding the answer. ‌

Best Regards,
Deepak Ahirwar

Re: Adding a child taks to a logstream setup

SushilKumar — Thu, 22 Feb 2024 09:37:09 GMT

Hello team,

If our response has been helpful, please consider clicking "Accept as Solution". This will assist other users in easily finding the answer.

Regards,

Sushil Kumar