Hi,
I'm having a problem where Qlik performance in runnings load scripts has degraded suddenly & significantly. Wondering if there are any thought out there as to where I may investigate?
I'm using enterprise edition Sept/2020 and about a week ago my overnight processes have slowed down (most nights). The overnight processing starts about 1am.
We have two servers (DEV & PROD) that both run materially the same process, and over all it takes about 6 hrs (normally) for the process to run, however in the last week our PROD environment is now more like 8-9hrs (except on the weekend when it came back to 6 hrs). The two servers are equivalent in CPU & memory (16CPUs & 128G RAM), and plenty of disk space. Both have similar bandwidth to our source systems.
The only material difference that I can really see between the two is that our business team is making use of the PROD server to work with Apps, but they are pretty consistent that they work roughly business hours (7am-6pm) so I do not see any activity from them overnight.
The two theories that I have now is that:
a) there is an unknown issue with the server and we need to restart (doing that later this week when I can get it approved). This seems unlikely as we had a restart 3 weeks ago, and I've seen the machines running for many months without a restart/issue.
b) something about the caching that Qlik does when users are working with Apps is causing the overnight processes to slow down (I can clearly see the memory consumption profile is different between the 2 servers). I'm thinking maybe put something in place each day (shortly before the overnight process starts) to force Qlik to dump most of the cache and free up that memory before the processing starts - this seems like a long shot as I've generally found Qlik Sense to be pretty good on freeing up memory when it needs it, and doesn't seem to me like something that should cause a 30-50% degradation in performance.
Any ideas you have would certainly be appreciated.
