My guess is that it is memory - it "fills up" with cache data. Useful to speed up calculations, but takes to much of RAM after some time.
If you can set up Publisher on a separate box, not with Server, the Publisher task problem will be resolved. But the problem on QVS remains. See QlikView Server Performance tab in QMC, maybe you want to reduce Working set.
Another approach is just to restart QVS daily (weekly). In this case you loose the advantage of calculation caching.
Many thanks for your suggestions.
I have checked RAM usage, we have 192 GBytes and it never exceeds 30 GBytes being used. I will monitor RAM usage on going though. We are only just starting to roll out dashboards, hence why the current over provision of RAM.
We have plans in place for a separate dedicated physical Publisher Server, but I somehow doubt that will solve this issue.
Yup, we could do scheduled restart of the QlikView Services but that seems like a bit of kludge.
Any other suggestions of areas to look at ?
Best Regards, Bill
Something isn't right with your setup. Do you have two Management services running? The reason I ask is that the settings are different in the two logs.
In the logfast you have
2013-10-07 14:16:43) Information: Allocating new QlikView Engine. Current usage count=1 of 8 (of type non-reader).
In the LogSlow you have
(2013-10-07 10:15:49) Information: Allocating new QlikView Engine. Current usage count=2 of 4 (of type non-reader).
Somehting isn't right here.
Thanks for you suggestion, but we are running no loop & reduce tasks.
Well spotted re the number of Max number of simultaneous QlikView engines for distribution having been increased from 4 to 8. It was me that did this, and so I did not give 2 log files suitable for direct comparison.
So now attached from this morning are Slow & Fast Log Files and the Task History, which show the task at 6:14 AM taking 1 min 26 sec and then at 7:14 AM , after restarting the QlikView Distribution Service, taking 32 sec.
23-Oct-13 8:14:28 AM Succeeded 34 sec 23-Oct-13 7:41:53 AM Succeeded 26 sec 23-Oct-13 7:14:50 AM Succeeded 32 sec 23-Oct-13 6:14:42 AM Succeeded 1 min 26 sec 23-Oct-13 5:14:39 AM Succeeded 1 min 23 sec 23-Oct-13 4:14:42 AM Succeeded 1 min 23 sec 23-Oct-13 3:15:16 AM Succeeded 1 min 27 sec
All Tasks show the same slowness, then fast after restart symptom. This is just the one I have chosen to use for diagnostics.
Best Regards, Bill M
Document logs attached, one from before restarting the QV Distribution Service and the other after.
The 1:20 PM taking 56 seconds was before the restart and the 1:27 PM talking 31 seconds after the restart.
Execution Started Status Duration 24-Oct-13 1:27:36 PM Succeeded 31 sec 24-Oct-13 1:20:32 PM Succeeded 56 sec 24-Oct-13 1:15:29 PM Succeeded 56 sec
Best Regards, Bill
I logged this as a Support Case with QlikTech and it transpires this a QlikView Bug :
Found in: 11.00.11440
Corrected in: Ver11.20/SR3
Title: QlikView - Publisher- Loop and Reduce/Distribute: Tasks taking longer to run over time
Describe Expected Behaviour
Would expect task run times to be relatively consistent
Describe Actual Behaviour
Over time task runs take longer to complete. After restarting QDS the task takes 34 minutes, the next time it took 3hrs. This was not an issue in v10 for customer.
Steps to reproduce:
Test server is 2003 server OS (as is customers servers) (all patches including the hot fix for pinned objects as been applied) and can be access here via RDP session: xx.xx.xx.xx
Tasks are listed under default Start.qvw in status pain, run that and they will kick off and continue till stopped
Included in this folder are QVWs for testing and a word doc outlining the task setup.
To set this up I have a Start task that you can do just a reload on, it's trigger is the successful completion of the other 5 QVWs
Each of the 5 QVWs has a task set up to loop and reduce then do a server distribution (keep in mind they have seen this with folder distributions as well).
Kick the tasks off see what happened, to my amazement the first task ran in 45 mins, however the next task took an hour, then next 39min, then next an hour, the next 1hr 38 mins, then next 1hr 42
3/27/2013 16:39:59.5902855 Information Task Execute Duration=00:45:36.7621605
3/27/2013 17:45:27.9597914 Information Task Execute Duration=01:03:49.0997172
3/27/2013 18:27:30.4382710 Information Task Execute Duration=00:39:02.6457510
3/27/2013 19:30:26.4289207 Information Task Execute Duration=01:02:11.5214314
3/27/2013 21:11:13.5372717 Information Task Execute Duration=01:38:24.9511912
3/27/2013 22:55:57.4733434 Information Task Execute Duration=01:42:33.8883644
No adjustments were made during these runs, I had one task with a warning which stopped the tasks from keeping up the cycle. QDS was never restarted during this time.
Best Regards, Bill