Sync between Apply Changes & Store Changes

[NakulanR]

Hi Support,

Just a general question regarding the two methods of writing changes using Qlik Replicate.

If a task containing a table 'Customers' has both Applied Changes and Store Changes switched on, and a change is written to 'Customers' in the source, can it be assumed that the change gets applied to the target 'Customers' table at the same time the Insert (containing the info about the change) gets sent to the 'Customers__CT' table? Or will they be sent at different times since the change being sent to the 'Customers__CT' table is an INSERT whereas the change being sent to the target 'Customers' table could be any of INSERT, UPDATE or DELETE?

Any advice on the above concept would be greatly appreciated?

Thanks,

Nak

[john_wang]

Hello Nak, @NakulanR 

In Batch Apply Mode, each batch (containing all source changes within a given interval) is applied to the target base table. At the same time, the change records are written to the __ct table (there may be a very slight delay between the two operations, as they are executed separately).

The data reflected in both the target base table and the __ct table represents the same source-side changes, but in different formats/layouts.

BTW, For applying changes to the base table (as opposed to the __ct table), the process can use either:

• Individual batched INSERT / UPDATE / DELETE statements, or
• A single MERGE statement (if supported by the target, e.g., Snowflake or other cloud data warehouses)

Using MERGE is recommended, as it generally provides better performance.

Hope this helps.
John.

[NakulanR]

Hi @john_wang,

Thank you for the prompt reply. 

Is there any way in which the time the change is applied to the base table and the time the change is written into Snowflake can be synced such that there is no discrepancy between the two? Would changing the apply mode to Transactional Apply have any effect? Or is it infeasible to achieve such a result?

Thanks,

Nak

[john_wang]

Hi Nak, @NakulanR 

I’m not sure how large the lag between the two tables is. If you are experiencing a big difference, I recommend opening a support ticket with verbose logging enabled on both SOURCE_CAPTURE /TARGET_APPLY for further investigation.

Please note that Transactional Apply is not supported in many cloud-based warehouses, including Snowflake.

Hope this helps.

John.