Adding or deleting tables to/from LSS tasks without using web console

[iti-attunity-sup]

Hello Experts

I have some questions about the way to add or delete tables to/from LSS tasks without using web console.

Currently, the customer concerns about operational error by editing tasks on the web console.
Therefore, they use a JSON files to add or delete tables to/from the LSS tasks.
(export the task, edit the JSON file and then import the file)

However, Full Load is required after importing tasks and it takes much time.
So we are looking for any other good methods which does not require full loading to modify the task without using web console.

Are there any good ideas to achieve this?
Any advice would be appreciated.

Regards,
Kyoko Tajima

[john_wang]

Hello Tajima-san, @iti-attunity-sup 

How about if you disable the Full Load (by editing the JSON) and then import it to repository? So far when you resume the task, the Full Load will be skipped.

Regards,
John.

[SachinB]

Hello @iti-attunity-sup ,

When you add a new table to an existing task with load settings (Full Load + CDC), it defaults to a full load for the added table upon resuming the task. To skip the initial load for the new table, you can disable the full load option as suggested by @john_wang 

Regards,

Sachin B

[iti-attunity-sup]

Hello @john_wang @SachinB 

Thank you for your kind reply.

When we changed a task definition and imported from JSON file, isn't full load necessarily required?

Can I confirm my understanding is correct? (as far as I tried in house, it works well ... )

Qlik Replicate Version : v2023.05

* Adding tables to the task:

1. Stop Replication task
2. Stop LSS task
3. Export both the tasks
4. Modify JSON file as below:
LSS task JSON file : "full_load_enabled": false + add table information 
Replication Task JSON files : add table information
5. import the task 
6. Resume LSS Task
7. Resume Replication task
--> only newly added tables will be full-loaded

* Deleting tables from the task :

1. Stop Replication task
2. Stop LSS task
3. Export both the tasks
4. Modify JSON file as below:
LSS task JSON file : "full_load_enabled": false + delete table information 
Replication Task JSON files : delete table information
5. Import the task
6. Start LSS Task
7. Start Replication task

 Could you let us know if there is anything else we should care?

Regards,

Kyoko Tajima

[SushilKumar]

Hello @iti-attunity-sup 

It Should work this way. Seems all the steps are ok.

Check the Full load is Disabled or not in Task Settings.  If not, then Disable it from UI and check the behaviour

Regards,

Sushil Kumar

[iti-attunity-sup]

@john_wang @SachinB @SushilKumar 

Thank you for looking into this issue and kindly reply.  And I'm sorry for confusing ...

In trying out several scenarios, I noticed the following points:

If a transaction for the Table#A to be added has already started before stopping the tasks, it seems that the transaction will NOT be reflected to the target.

That is :

0. Transaction#1 for Table#A started and NOT committed.
1. Stop Replication task
2. Stop LSS task
3. Export both the tasks
4. Modify JSON files (add Table#A)
5. Import the tasks
6. Resume LSS task
7. Resume Replication task
8. Transaction#1 is committed.
--> This transaction is not reflected to the target.
Because it started before stopping the tasks.

So am I correct in assuming below?
- the only way to avoid the above situation is reloading after importing the JSON file
- we can just restart the task only if it is certain that no updates have been made to the table to be added

Regards,
Kyoko Tajima

[SachinB]

Hello @iti-attunity-sup ,

The newly added to the table will set different flag which will go for reload one you just resume the task not all the table from task. In this way the data will be sync for CDC after the full load.

If you still have doubts you can open the support case and we work together on this.

Regards,

Sachin B