Hi All,
I have a load script that deals with large amounts of data. It runs weekly for an incremental load process.
Recently, it has been taking an extremely long time to load. I have analysed the logs and found the points in the process that are taking the longest and I'm trying to replicate them in isolation by manually reloading certain parts in a stand-alone QVW.
One step in particular takes 1day 1hour 28mins. This simply reads a large QVD with minimal changes then does an inner join with another QVD. I have loaded the first QVD which took only 38mins using the same portion of code. Then I thought perhaps the time related to the inner join, so I ran this in isolation also and that took a total of 42 mins using the same code as the main script. So obviously there is something else at play here.
The regular reload is kicked off using command-line (qlikview /r xx.qvw).
Has anyone experienced this type of behaviour before? My understanding is that synthetic keys aren't built until the end of a load process, so that is not the problem because all tables are dropped once dealt with and stored out. I've also made sure I hadn't set up any regular processes that might be having an impact on resources.
Any ideas would be greatly appreciated!
Cheers,
GPC
