Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hello Qlik community,
I have some flows that replicate data from DB2 to Snowflake.
To do this, the process goes through jounals and libraries.
To sum up, I have two type of tasks : A reading task that reads data (changes) from DB2 (endpoint IBM DB2 for iSeries) and a writing task to replicate data in Snowflake.
The issue :
We have noticed a significant increase in CPU usage related to a database we are querying.
To give you the context:
One of our reading tasks failed and it was impossible to restart it. This task involves three schemas with over 170 tables (per schema). We therefore split this reading task into two, one task that has one schema and a second task that has two schemas. This action allowed us to restart the reading tasks. As a result, we created two writing tasks. It should be noted that we created two DB2 endpoints for each reading task
We did not notice any major change in CPU usage.
A few days later, the two reading tasks failed again. This time we split them into three, one reading task per schema. We then launched these three reading tasks and associated three writing tasks and it was from this moment that there was an abnormal CPU consumption. For the creation of these three reading tasks, we used 3 different endpoints to connect to DB2 (the same logic as with the two tasks above)
We are wondered if it wasn't the creation of a dedicated endpoint for each reading task that generated this CPU percentage. What is inconsistent is that there was no increase when initially creating 2 endpoints. We know it's possible to create a single endpoint for the three reading tasks, but before making this change, we'd like to have your opinion.
Do you have please any idea about the origin of this problem? It is urgent because the peak of the activity is coming.
Thank you all
