15 Replies Latest reply: Mar 29, 2016 3:29 AM by Marcus Sommer RSS

    Server-Task delaying

    Marcus Sommer

      Hi,

       

      some of my task-chains which runs daily triggered through the qv server (no publisher available) needs to be to delayed on sunday for 5 hours.

       

      For this I have created some start-qvw's which check the weekday and delay it then - the weekday checking worked without a problem but the delaying from following-tasks is only 60 minutes (3 task-chaines) or 90 minutes (2 task-chaines).

       

      Are there any timeout-settings or could it be that there are then too many parallel-tasks (how many are possible within the server - two?) because of this waiting some of the task-chains are now overlapping eachother?

       

      How could it be handled? Especially if I don't want to use solely windows-task triggered macro-routines (which I have used previously).

       

      This is the routine - but I think they isn't the issue:

       

      if num(weekday(today())) = 6 then

          // hours * minutes * seconds * 1000 (for milliseconds)

           sleep 5 * 60 * 60 * 1000; // 5 hours delay on sundays (values in milliseconds)

      else

          sleep 50;

      end if

       

      I hope someone could help me. Many thanks.

       

      - Marcus

        • Re: Server-Task delaying
          Marcus Sommer

          Anyone has an idea or a hint for me?

           

          - Marcus

            • Re: Server-Task delaying
              Peter Cammaert

              Are your delay QVWs being aborted? Or do they run the full delay that you specified in the Timeout input field on the reload tab? Note that this Timeout value includes everything: all retries and all task setup/shutdown delays.

                • Re: Server-Task delaying
                  Peter Cammaert

                  Sorry Marcus, ignore my previous post.

                   

                  Better check the QV Help article on statement SLEEP. The very first sentence will be very enlightening...

                   

                  You can fix this problem by embedding a SLEEP statement in a FOR loop.

                   

                  Peter

                    • Re: Server-Task delaying
                      Marcus Sommer

                      Many thanks, this is definitely a point and I will try it with such a for-loop!

                       

                      Do you know how many tasks the server could parallel execute even they are in waiting-loop? I think I have read somewhere that there are limitations at least by the server (or even by publisher?).

                       

                      - Marcus

                        • Re: Server-Task delaying
                          Peter Cammaert

                          Yes there are limits to the number of parallel reload jobs you can have running (you can start many more but they will be queued). The limit can be modified. Best practice is to not set this number to anything higher than the number of cores in your server minus one. Reloads (especially JOINs) may cause a core to go to 100% load, and you don't want all cores to be in this state as Windows will become unresponsive (the reloads will succeed though )

                           

                          See QMC->System->Setup->QlikView Servers->your servername->Performance->Reload limits.

                            • Re: Server-Task delaying
                              Marcus Sommer

                              How does this queueing work? Will they be executed considering to their start-time and after the first few task-chaines are finished the next one within the queue started although is origin start-time was already for hours or will they be skipped?

                               

                              - Marcus

                                • Re: Server-Task delaying
                                  Peter Cammaert

                                  Yes indeed. If you have two cores available for reloads, and four tasks should start at the same time, two of them will be delayed until Reload engines become available. As one of the first two reload tasks is completed, the reload engine will be freed up and the next task will allocate it for script execution. As far as I know, the mechanism is first-come, first-served-based.

                                   

                                  The funny thing is that the 4 tasks-2 cores setup will show 4 green play buttons in the QMC Status screen, so that you know that 4 tasks are due to be completed. But only 2 of them are actually running simultaneously.

                                   

                                  Best,

                                   

                                  Peter

                                    • Re: Server-Task delaying
                                      Marcus Sommer

                                      Many thanks - then it should work only with the measure that I applied the for-loop to extend the max. sleep-time. I see then hopefully the next Sunday against ...

                                       

                                      - Marcus

                                        • Re: Server-Task delaying
                                          Peter Cammaert

                                          Good luck

                                           

                                          Peter

                                            • Re: Server-Task delaying
                                              Marcus Sommer

                                              The first two tasks worked (after implementing the hours-loop) fine with each 5 hours delaying to the normal start-time. But all then followings tasks failed with these error:

                                               

                                              27.09.2015 06:30:01 Error Failed to allocate a QlikView Engine.
                                              27.09.2015 06:30:01 Error No QlikView Engine was available after waiting for 45 minutes

                                               

                                              For me it looked that only two task would work parallel and/or waiting tasks will be aborted after 45 minutes. Are there further settings to influence this? What are the max. Limits for parallel tasks, waiting times and so on?

                                               

                                              Other ideas to delay tasks?

                                               

                                              - Marcus

                                                • Re: Server-Task delaying
                                                  Peter Cammaert

                                                  Hi Marcus,

                                                   

                                                  I temporarily switched over from the bbq to a portable qlikview server (someone else is doing the dishes ) to bring you some good news: yes, there is a setting to improve your situation, but it is hidden in a configuration file and the QMC won't be of any help.

                                                   

                                                  Open C:\Program Files\QlikView\Distribution Service\QVDistributionService.exe.config. Notepad will do just fine, but you can use any text editor.

                                                  Scroll down until you encouter a setting called "QvbWaitTimeoutMinutes". Overthere you'll find your value of 45 minutes. Increase this value so that it takes the maximum delay into account: the 5-hour wait time+the actual reload delay+any margin that you think is suitable.

                                                  Save and restart the QDS.

                                                   

                                                  Note that this is a global setting. From now on, all queued tasks will wait for a free QVB engine for a very long time before giving up.

                                                   

                                                  Alternatives? As far as I can see you are fighting a limitation of the standard Publisher "built into" QlikView Server: every task has only one trigger with a single set of specifications and those turn out not to be very flexible at all. Possible solutions:

                                                   

                                                  • Buy a Publisher license (too obvious and probably not very helpful...) With a full Publisher, you can give a task any number of triggers with any type of settings. For example running a task at a fixed time 6 days every week, and at another time on sunday only.
                                                  • Or set your reload tasks to be triggered by "an external event", and use Windows Task Scheduler in combination withQMSEDX Enhanced to trigger a single task at different times every other day. More complex but very flexible.

                                                   

                                                  Best,

                                                   

                                                  Peter

                                                    • Re: Server-Task delaying
                                                      Marcus Sommer

                                                      Many thanks. I will have a closer look in the next week on these setting but I think I will give it a try.

                                                       

                                                      Maybe with an future extension of our environment I could get a publisher but it's no topic for the next time. External trigger per EDX would mean I had to use only those trigger then within the server is only one trigger for an application available and I think the disadvantages are greater then the benefits (here in my case, not in general).

                                                       

                                                      - Marcus

                                                      • Re: Server-Task delaying
                                                        Marcus Sommer

                                                        The ninth task-chain failed after six hours (delaying was five hours) with the following error-message:

                                                         

                                                        04.10.2015 13:25:10 Warning The QlikView Engine is Reloading, it will be killed (Please ignore logged warnings/errors about the kill).
                                                        04.10.2015 13:25:11 Error The task "Controls/CheckWeekDay_Log+Meta.qvw" failed. Exception: QDSMain.Exceptions.TaskFailedException: Task execution failed with errors to follow. ---> QDSMain.Exceptions.ReloadFailedException: Reload failed ---> System.Threading.ThreadAbortException: Der Thread wurde abgebrochen. bei System.Threading.WaitHandle.WaitOneNative(SafeHandle waitableSafeHandle, ....


                                                        But I believe my problem is solved - I have done both increasing the qvbwaittimeoutminutes and the number of parallel qvb-tasks - and I think that with a little bit fine-tuning it will run without any error. And even if the last chain with my logging- and meta-data failed it's no drama - nobody needs them on a sunday.

                                                         

                                                        Many thanks for your help.

                                                         

                                                        - Marcus

                                                        • Re: Server-Task delaying
                                                          Keith Doyle

                                                          Note that we have run into this problem as well, and have been advised by QlikView support that bumping the QvbWaitTimeoutMinutes parameter up much more than the default 45 minutes can risk having the load run and be unable to write out the QVW at the end because of a Windows lock that expires on the file--- apparently, when the task is triggered, QV sets a lock on the QVW so that it isn't touched until it can get to it-- and Windows can timeout the lock  after 45 minutes, causing the reload write-out of the qvw to fail.   So the idea of setting the value to 5-hours would seem to be erroneous.   We're looking into instead governing the task triggers to depend on when the engines are available.