27 Replies Latest reply: Aug 14, 2015 2:46 PM by Bill Britt RSS

    QlikView Server 11 Reload Tasks Hang in Running State

    Travis Dirmyer

      Good day everyone,

       

      My company recently upgraded from QlikView Server 10 SP4 with Publisher (all-in-1) to QlikView Server 11 SP1 with Publisher (all-in-1). Shortly afterwards, we found that some of our reload tasks were hanging in a "running" state, even after finishing. After troubleshooting, speaking with our QlikView provider and reviewing the QlikView community, we found that we are being affected by a bug in QlikView 11 SP1.

       

      In this bug, QlikView Server reload tasks will run, and complete. However, the task may remain in a running state until the QDS is restarted. If the QDS is not restarted, the next time a task which remains in a running state is triggered, it will not trigger as the QDS believes that the task has not finished from the previous run time.

       

      To counter this issue, my company has created a simple Windows Scheduled Task to initiate a restar of the QlikView Distribution Service (QDS) after business hours. The windows task calls the Windows PowerShell application with an argument that performs the restart. This solution has been helping us combat this bug until it is resolved.

       

      We have created a help document for other QlikView Server Admins who may also be experiancing the same issue.

       

      QlikView Community document:

      http://community.qlik.com/docs/DOC-3152

       

      Thank you all

       

      -Travis Dirmyer

        • Re: QlikView Server 11 Reload Tasks Hang in Running State
          Jakob Fabian

          Hi Travis,

           

          while restarting the QDS fixes the problem, it's not the solution. Patch Release 11.00.11311 fixes your issue:

          "Bug 46252: Publisher in QV 11 Sr1 - Tasks status on QMC stay on running but they succeeded in the logs".

           

          For further reference, see below

          http://community.qlik.com/message/239623#239623

           

          Regards, Jakob

          • Re: QlikView Server 11 Reload Tasks Hang in Running State
            Travis Dirmyer

            I have confirmed that this bug still exists in QlikView Server 11 SR2.

             

            My deployment was an in-place upgrade from QVS 11 SR1 to QVS 11 SR2. We experianced this issue under SR1, however, it was fixed by Qliktech by supplying a fixpack. This fix was un-done in SR2 as we are once again experiancing hanging reload tasks.

             

            The log script shows that the QVW was reloaded successfuly. However, the task in the QMC stays running.

             

            Thank you all.

            -Travis

            • Re: QlikView Server 11 Reload Tasks Hang in Running State
              Nick Gan

              Hi,

               

              Any news for this subject?

               

              On the other side, i'm also noticing when QVDistributionService.exe gone beyond 1GB of memory in task manager, it become less functional. some task will not even start, and i had to restart the service.

               

              Is this also known bug?

               

               

              Thanks.

              • Re: QlikView Server 11 Reload Tasks Hang in Running State

                Hi,

                 

                A few months ago, we have moved to QlikView Server 11 SR2 and the bug disappeared. But, unfortunately, for RAM problems ( probably) , the hanging task problem has recently emerged again.

                 

                So, i'm searching for a temporary solution, like executing a script to check the task status before running it (If he is Running, we have to modify its status to Waiting).. Any Idea please???

                  • Re: QlikView Server 11 Reload Tasks Hang in Running State
                    Travis Dirmyer

                    Hello Sanaba321,

                     

                    I setup one of my test servers with QVS 11SR2 and had task hang on that machine. In addition, I have been able to reproduce this problem in QVS 11.2 SR1.

                     

                    I put in a little time working on this. However, in my deployment, if we have 8 engines for distribution setup and the timeing for reload tasks triggers in such a way that 10 or 11 tasks are triggered (only 8 still process), then one task will either fail or will not connect to QVW resulting in hang in a running state. I say it that way, because I find no logs or evidence that the .QVW related to the task was even touched by the task. This is different than the behavior we have seen before where the QVW related to the task would finish, but the server task stays green and doesn't complete.

                     

                    thank you.

                      • Re: QlikView Server 11 Reload Tasks Hang in Running State

                        Has anyone test the new 11.2R2 that just came out a couple weeks ago to see if it resolved this issue yet?

                          • Re: QlikView Server 11 Reload Tasks Hang in Running State
                            Travis Dirmyer

                            I am testing QVS 11.2 SR2 and am working with our support agent. It seems to happen as we increase our parallel reloading. As we task the QDS with more tasks running in parallel, they seem to error for odd reasons.

                             

                            Has anyone encountered these issues with QVS 11, 11SR1, 11SR2,  11.2SR1, 11.2SR2?


                            Reload problems.

                            We have issues with QVS11.2 SR2 when running QVW reload tasks in parallel with Publisher. This problem has persisted from QVS 11SR1 to QVS11SR2 and still exists in QVS11.2SR2. Our Main production system (QVS 11 SR1) has 2 CPUs 8Cores, enabling us to run 7 Engines for distribution (7-way parallel based on descriptions below) however can only run in 6-way parallel as tasks become unstable and fail randomly with higher than 6 engines for distributions. We thought there were some hardware limitations prohibiting us from increases our parallel processing. However, an alternate server which has QVS11.2 SR2 4CPUs and 16 Cores, experiences problems running in 8 and 9-way parallel which is only 2-3 engines higher.

                            Reloading in Parallel
                            We use process where tasks are created to reload in sequence via dependencies (upon success). Extract and Transformation are configured in this “trees” or “column” style approach with a maximum of N-trees/columns (N-way parallel) where N is 1 less than the max number of CPU cores (8 cores, 7-way parallel). According to QlikTech deployment information, the QlikView Distribution Service can be configured with X-Distribution Cores where X is 1 less (-1) than the maximum number of non-hyper threaded CPU cores up to a maximum of 9.

                            Our QVS11.2 SR2 QlikView server was configured with 9, 8 and 7 distribution engines and reloads have been tested many times. QlikView Reload failures occur on 9, 8 and 7 distribution engines with 9, 8, 7-way parallel configuration. The errors which occur are the same and occur at a higher frequency as the distribution engines go up.

                            Failure Type – Infinite loading - Hung Task
                            Periodically, a QlikView Task will run until its task expiration time in an “Infinite Loading” loop. According to the server log, the task starts up and opens the QVW. It acts like it is reloading the QVW but isn’t. The QVW is not being accessed by the server, but the server thinks it is accessing the file. The server log for this task shows an entry every few seconds with “Loading…”, this is different than the “reloading…” task log statement from a normal task trigger. There is NO document log for this event. The last update date of the .QVW remains unchanged. No record of access takes place.
                            • The QlikView Server has full / administrator access to all file stores where objects reside.
                            • Reload task begins as normal, but then enters an Infinite Loading loop
                            • Cannot reproduce on-demand

                            QlikView Release Notes:
                            QlikView Server, QlikView Publisher and Management Consoles· “Due to a limitation associated with using Microsoft’s COM objects we recommend that you limit the number of QlikView Engines (QlikView Enterprise Management Console>> System>> Setup>> Distribution Services>> Advanced tab) to a maximum of 9 or the number of processor cores available on the host server -1, whichever is lower.” If you have more than 9 processor cores, and wish to run more Engines, contact Support for information regarding a registry change to the Desktop Memory Heap settings on the server.