Prioritization is at two level. Full Load and CDC.
Full Load - Applicable at task level only. Say you have 10 tables with 3 tables with more records, rest 7 with less. then you can set the "Load Order" as HIGHEST to the 7 tables and HIGHER to the 3 tables so that small tables load first and then the large tables. You can do vice versa as well.
CDC - By nature of product, it reads the database logs and "replays" the transaction and hence 'replicate' from source to target. Hence you will not find an option to 'prioritize' incoming CDC changes as Replicate do not control which table sends changes first.
If you still think there is a specific use case where this would be helpful, please submit it as an Idea in the ideation portal. Ensure to provide elaborate details of the use case and how it would benefit you and specify the endpoints as well.
In a production environment, you would let the task run continuously once started. Do you intent to STOP the tasks during a certain time of a day and then resume later? The design needs to be thoroughly vetted in this case.
However, you can use QEM (Qlik Enterprise Manager) RunTask API to run a task. You can script the API and use an enterprise scheduler to schedule the tasks. But replicate cannot identify 'table activity' at source end as it is external to Replicate. If there is a way you can script that at source end, then you can use that as a 'trigger' using the enterprise scheduler and control tasks execution.
Thanks for your response. Tasks will be running all the time - not planning to STOP and Start these. We are also using Log Stream.
We would like to prioritize replication of a set of tables over others and were advised by Qlik SME to separate these into multiple tasks. We would prefer to assign priority to tasks so that when CDC occurs - higher priority task replicates first to minimize latency of these tables.
Please advise if this is feasible to do. If not then how can we accomplish this.
Prioritization is at two level. Full Load and CDC.
Full Load - Applicable at task level only. Say you have 10 tables with 3 tables with more records, rest 7 with less. then you can set the "Load Order" as HIGHEST to the 7 tables and HIGHER to the 3 tables so that small tables load first and then the large tables. You can do vice versa as well.
CDC - By nature of product, it reads the database logs and "replays" the transaction and hence 'replicate' from source to target. Hence you will not find an option to 'prioritize' incoming CDC changes as Replicate do not control which table sends changes first.
If you still think there is a specific use case where this would be helpful, please submit it as an Idea in the ideation portal. Ensure to provide elaborate details of the use case and how it would benefit you and specify the endpoints as well.
