Publisher tasks steadily get slower until Windows Services restarted

[Anonymous]

Hi

We are running QlikView Publisher, 11.0 SR2 [11.0.11414.0].

Our Published Tasks steadily get slower and slower over a few days, until the Windows Service is restarted and then they speed up. Then again they steadily get slower and slower over a few days, until the Windows Service is restarted.  This cycle repeats ad infinitum.

Attached is the log when it was slow, LogSlow.txt when it took 2 minutes and 32 seconds.

And also the log after the Service restart when it was fast, LogFast.txt when it took 0 minutes and 38 seconds.

Any ideas what to look for ?

Feel free to request any more information.

Best Regards,     Bill

.

[Anonymous]

My guess is that it is memory - it "fills up" with cache data.  Useful to speed up calculations, but takes to much of RAM after some time.

If you can set up Publisher on a separate box, not with Server, the Publisher task problem will be resolved.  But the problem on QVS remains.  See QlikView Server Performance tab in QMC, maybe you want to reduce Working set.

Another approach is just to restart QVS daily (weekly).  In this case you loose the advantage of calculation caching.

Regards,

Michael

[Anonymous]

Michael

Many thanks for your suggestions. 

I have checked RAM usage, we have 192 GBytes and it never exceeds 30 GBytes being used.  I will monitor RAM usage on going though.  We are only just starting to roll out dashboards, hence why the current over provision of RAM.

We have plans in place for a separate dedicated physical Publisher Server, but I somehow doubt that will solve this issue.

Yup, we could do scheduled restart of the QlikView Services but that seems like a bit of kludge.

Any other suggestions of areas to look at ?

Best Regards,     Bill

[Anonymous]

Bill, I can't add anything, just a recommendation to monitor - which you're going to do without my advice anyway...

[Bill_Britt]

Hi Bill,

Something isn't right with your setup. Do you have two Management services running? The reason I ask is that the settings are different in the two logs.

In the logfast you have

2013-10-07 14:16:43) Information: Allocating new QlikView Engine. Current usage count=1 of 8 (of type non-reader).

In the LogSlow you have

(2013-10-07 10:15:49) Information: Allocating new QlikView Engine. Current usage count=2 of 4 (of type non-reader).

Somehting isn't right here.

Bill

[#6]

Are you running a lot of loop and reduce distribution tasks?

[Anonymous]

Andreas

Thanks for you suggestion, but we are running no loop & reduce tasks.

Bill B

Well spotted re the number of Max number of simultaneous QlikView engines for distribution having been increased from 4 to 8.  It was me that did this, and so I did not give 2 log files suitable for direct comparison.

So now attached from this morning are Slow & Fast Log Files and the Task History, which show the task at 6:14 AM taking 1 min 26 sec and then at 7:14 AM , after restarting the QlikView Distribution Service, taking 32 sec.

23-Oct-13 8:14:28 AM	Succeeded	34 sec
23-Oct-13 7:41:53 AM	Succeeded	26 sec
23-Oct-13 7:14:50 AM	Succeeded	32 sec
23-Oct-13 6:14:42 AM	Succeeded	1 min 26 sec
23-Oct-13 5:14:39 AM	Succeeded	1 min 23 sec
23-Oct-13 4:14:42 AM	Succeeded	1 min 23 sec
23-Oct-13 3:15:16 AM	Succeeded	1 min 27 sec

All Tasks show the same slowness, then fast after restart symptom.  This is just the one I have chosen to use for diagnostics.

Best Regards,     Bill M

[#8]

Please provide document logs from a slow and fast run, that way we can see if the exact same amount of data is being processed.

From the latest task log indicates that more data is being processed as the calculated memory to be used is bigger in the slower run.

[Anonymous]

Andreas

Document logs attached, one from before restarting the QV Distribution Service and the other after.

The 1:20 PM taking 56 seconds was before the restart and the 1:27 PM talking 31 seconds after the restart.

Execution Started	Status	Duration
24-Oct-13 1:27:36 PM	Succeeded	31 sec
24-Oct-13 1:20:32 PM	Succeeded	56 sec
24-Oct-13 1:15:29 PM	Succeeded	56 sec

Best Regards,     Bill

[Anonymous · Accepted solution]

Dear All

I logged this as a Support Case with QlikTech and it transpires this a QlikView Bug :

Found in: 11.00.11440

Corrected in: Ver11.20/SR3

Title: QlikView - Publisher- Loop and Reduce/Distribute: Tasks taking longer to run over time

Describe Expected Behaviour

Would expect task run times to be relatively consistent

Describe Actual Behaviour

Over time task runs take longer to complete. After restarting QDS the task takes 34 minutes, the next time it took 3hrs. This was not an issue in v10 for customer.

Steps to reproduce:

Test server is 2003 server OS (as is customers servers) (all patches including the hot fix for pinned objects as been applied) and can be access here via RDP session: xx.xx.xx.xx

Tasks are listed under default Start.qvw in status pain, run that and they will kick off and continue till stopped

Included in this folder are QVWs for testing and a word doc outlining the task setup.

To set this up I have a Start task that you can do just a reload on, it's trigger is the successful completion of the other 5 QVWs

Each of the 5 QVWs has a task set up to loop and reduce then do a server distribution (keep in mind they have seen this with folder distributions as well).

Kick the tasks off see what happened, to my amazement the first task ran in 45 mins, however the next task took an hour, then next 39min, then next an hour, the next 1hr 38 mins, then next 1hr 42

3/27/2013 16:39:59.5902855 Information Task Execute Duration=00:45:36.7621605

3/27/2013 17:45:27.9597914 Information Task Execute Duration=01:03:49.0997172

3/27/2013 18:27:30.4382710 Information Task Execute Duration=00:39:02.6457510

3/27/2013 19:30:26.4289207 Information Task Execute Duration=01:02:11.5214314

3/27/2013 21:11:13.5372717 Information Task Execute Duration=01:38:24.9511912

3/27/2013 22:55:57.4733434 Information Task Execute Duration=01:42:33.8883644

No adjustments were made during these runs, I had one task with a warning which stopped the tasks from keeping up the cycle. QDS was never restarted during this time.

Best Regards,     Bill