QlikView Server 11 Reload Tasks Hang in Running State

[Anonymous]

Good day everyone,

My company recently upgraded from QlikView Server 10 SP4 with Publisher (all-in-1) to QlikView Server 11 SP1 with Publisher (all-in-1). Shortly afterwards, we found that some of our reload tasks were hanging in a "running" state, even after finishing. After troubleshooting, speaking with our QlikView provider and reviewing the QlikView community, we found that we are being affected by a bug in QlikView 11 SP1.

In this bug, QlikView Server reload tasks will run, and complete. However, the task may remain in a running state until the QDS is restarted. If the QDS is not restarted, the next time a task which remains in a running state is triggered, it will not trigger as the QDS believes that the task has not finished from the previous run time.

To counter this issue, my company has created a simple Windows Scheduled Task to initiate a restar of the QlikView Distribution Service (QDS) after business hours. The windows task calls the Windows PowerShell application with an argument that performs the restart. This solution has been helping us combat this bug until it is resolved.

We have created a help document for other QlikView Server Admins who may also be experiancing the same issue.

QlikView Community document:

http://community.qlik.com/docs/DOC-3152

Thank you all

-Travis Dirmyer

[#2]

Thank you for your quick reply.

For more details: We had uninstalled QV11 SR1 and then installed QV11 SR2, but we did not delete old tasks.

Should we recreate all the tasks after installation?

Also, we have recently clustred our servers: 2 QVS and 1 (QDS,QMS). We had upgrated (uninstall/install) only the QV Publisher server. Should we upgrade also the QVSs?

Thank you.

[Anonymous]

I have confirmed that this bug still exists in QlikView Server 11 SR2.

My deployment was an in-place upgrade from QVS 11 SR1 to QVS 11 SR2. We experianced this issue under SR1, however, it was fixed by Qliktech by supplying a fixpack. This fix was un-done in SR2 as we are once again experiancing hanging reload tasks.

The log script shows that the QVW was reloaded successfuly. However, the task in the QMC stays running.

Thank you all.

-Travis

[rothtd]

For what it's worth, I had the bug in 11SR1 and did an inplace upgrade to 11SR2. I have no signs of the bug - I upgraded to SR2 the day it was released (so I should have seen the bug by now). I never had the fix pack installed - which it sounds is the difference between me and you guys.

[#5]

Hi,

Any news for this subject?

On the other side, i'm also noticing when QVDistributionService.exe gone beyond 1GB of memory in task manager, it become less functional. some task will not even start, and i had to restart the service.

Is this also known bug?

Thanks.

[#6]

Hi,

A few months ago, we have moved to QlikView Server 11 SR2 and the bug disappeared. But, unfortunately, for RAM problems ( probably) , the hanging task problem has recently emerged again.

So, i'm searching for a temporary solution, like executing a script to check the task status before running it (If he is Running, we have to modify its status to Waiting).. Any Idea please???

[Anonymous]

Hello Sanaba321,

I setup one of my test servers with QVS 11SR2 and had task hang on that machine. In addition, I have been able to reproduce this problem in QVS 11.2 SR1.

I put in a little time working on this. However, in my deployment, if we have 8 engines for distribution setup and the timeing for reload tasks triggers in such a way that 10 or 11 tasks are triggered (only 8 still process), then one task will either fail or will not connect to QVW resulting in hang in a running state. I say it that way, because I find no logs or evidence that the .QVW related to the task was even touched by the task. This is different than the behavior we have seen before where the QVW related to the task would finish, but the server task stays green and doesn't complete.

thank you.

[#8]

Has anyone test the new 11.2R2 that just came out a couple weeks ago to see if it resolved this issue yet?

[Anonymous]

I am testing QVS 11.2 SR2 and am working with our support agent. It seems to happen as we increase our parallel reloading. As we task the QDS with more tasks running in parallel, they seem to error for odd reasons.

Has anyone encountered these issues with QVS 11, 11SR1, 11SR2,  11.2SR1, 11.2SR2?

Reload problems.

We have issues with QVS11.2 SR2 when running QVW reload tasks in parallel with Publisher. This problem has persisted from QVS 11SR1 to QVS11SR2 and still exists in QVS11.2SR2. Our Main production system (QVS 11 SR1) has 2 CPUs 8Cores, enabling us to run 7 Engines for distribution (7-way parallel based on descriptions below) however can only run in 6-way parallel as tasks become unstable and fail randomly with higher than 6 engines for distributions. We thought there were some hardware limitations prohibiting us from increases our parallel processing. However, an alternate server which has QVS11.2 SR2 4CPUs and 16 Cores, experiences problems running in 8 and 9-way parallel which is only 2-3 engines higher.

Reloading in Parallel
We use process where tasks are created to reload in sequence via dependencies (upon success). Extract and Transformation are configured in this “trees” or “column” style approach with a maximum of N-trees/columns (N-way parallel) where N is 1 less than the max number of CPU cores (8 cores, 7-way parallel). According to QlikTech deployment information, the QlikView Distribution Service can be configured with X-Distribution Cores where X is 1 less (-1) than the maximum number of non-hyper threaded CPU cores up to a maximum of 9.

Our QVS11.2 SR2 QlikView server was configured with 9, 8 and 7 distribution engines and reloads have been tested many times. QlikView Reload failures occur on 9, 8 and 7 distribution engines with 9, 8, 7-way parallel configuration. The errors which occur are the same and occur at a higher frequency as the distribution engines go up.

Failure Type – Infinite loading - Hung Task
Periodically, a QlikView Task will run until its task expiration time in an “Infinite Loading” loop. According to the server log, the task starts up and opens the QVW. It acts like it is reloading the QVW but isn’t. The QVW is not being accessed by the server, but the server thinks it is accessing the file. The server log for this task shows an entry every few seconds with “Loading…”, this is different than the “reloading…” task log statement from a normal task trigger. There is NO document log for this event. The last update date of the .QVW remains unchanged. No record of access takes place.
• The QlikView Server has full / administrator access to all file stores where objects reside.
• Reload task begins as normal, but then enters an Infinite Loading loop
• Cannot reproduce on-demand

QlikView Release Notes:
QlikView Server, QlikView Publisher and Management Consoles· “Due to a limitation associated with using Microsoft’s COM objects we recommend that you limit the number of QlikView Engines (QlikView Enterprise Management Console>> System>> Setup>> Distribution Services>> Advanced tab) to a maximum of 9 or the number of processor cores available on the host server -1, whichever is lower.” If you have more than 9 processor cores, and wish to run more Engines, contact Support for information regarding a registry change to the Desktop Memory Heap settings on the server.

[#10]

Hello

Anybody have and can place "Support information regarding a registry change" ?

Best regards

Hugo

[#11]

Travis,  I also see that issue, using SR2, just with QVS, no publisher at a customer.  I had one task reloading every 15 minutes, then a group of others dependent on every 24 hours.  It seems they have gotten messed up, even restarting the machine brings them all back to 'error' state.  Once I kick off the 15 m task again, it will start on its own.  I have changed the 24 hour task to be offset (they would have once a day been triggered at the same time).

Hopefully all will fix itself and come back to normal tomorrow.