7 Replies Latest reply: Apr 14, 2016 5:31 AM by Johan Lindell RSS

    Alert when a task is delayed or stuck

    Ariel Topaz

      Hi

      I would like to know if there a option to set an alert on QMC when task is delayed from normal or if the task stuck

       

      TNX

        • Re: Alert when a task is delayed or stuck
          Johan Lindell

          You could create a doucment that checks the file structure of C:\ProgramData\QlikTech\DistributionService\1\Log and look for the file "_InUse.!!!" in each subfolder. If found, look at the file-time and if that's too old, fire an Alert in the Document.

           

          Johan

            • Re: Alert when a task is delayed or stuck
              Ariel Topaz

              Hi Johan

              thanks for the help

               

              one Question

              in the folder where the "_InUse.!!!"  file

              i have 2 log files

              1. TaskLog

              2. DocumentLog

               

              the TaskLog shows the running duration only when the task is finished

              For example

              Task Execute Duration=00:00:29.9693968

               

              in the DocumentLog i cant find mention for runtime


              how can i know how much time the task is running during the run?

                • Re: Alert when a task is delayed or stuck
                  Johan Lindell

                  Hi Ariel,

                   

                  When the Document is reloading, there should be an "_InUse.!!!"-file in the folder. The timestamp of this file should be when the job started, so Now () - FileTime ('.......\_InUser-!!!') should give you the duretion so far.

                   

                  /Johan

                    • Re: Alert when a task is delayed or stuck
                      Ariel Topaz

                      OK TNX

                      I thought there is a value in the log like "Task Execute Duration".

                      Another Q

                      I want to compare the average run time to the Current runtime of the task

                      were i can find the average running time for this task?

                       

                      TNX again

                        • Re: Alert when a task is delayed or stuck
                          Johan Lindell

                          Ariel,

                           

                          Try this script. It will give you all the executions found in the folder of the log files. Then you'll get the data you need and can build charts in the QlikView-application to compare the last with averages.

                           

                          Possibly you could also install Goverance Dashboard, and look into the qvd(s) / qvx(s) it is producing and build your own application on this.

                           

                          Good luck!

                          Johan

                           

                           

                           

                           

                           

                          IF Len('$(vFolderDistributionService)') = 0 then

                            LET vFolderDistributionService = 'C:\ProgramData\QlikTech\DistributionService';

                          ENDIF

                           

                           

                          SUB DoDirTasks (Root)

                            for each File in filelist ('$(Root)' & '\*.xml')

                           

                            LET TaskId = Replace(Replace(SubField('$(File)', '\', SubStringCount('$(File)', '\') + 1), 'Task_', ''), '.xml', '');

                           

                            IF not IsNull(TableNumber('ExistingTasks')) Then

                            isFileAnExistingTask:

                            NoConcatenate

                            LOAD *

                            Resident ExistingTasks

                            Where TaskId = '$(TaskId)';

                            ENDIF

                           

                            IF Len('$(vLimitFileName)') = 0 or NoOfRows('isFileAnExistingTask') > 0 then

                           

                            QlikViewServerTasks_Recipients:

                            LOAD ObjectType AS RecipientObjectType,

                               ID AS RecipientID,

                               SubField(ID, '\', 1) AS Domain,

                               SubField(ID, '\', 2) AS [Group name],

                               '$(TaskId)' AS TaskId

                            FROM [$(File)] (XmlSimple, Table is [DistributeTask/Resources/Resource/AddRecipient]);

                           

                            QlikViewServerTasks_Resource:

                            LOAD Type,

                               Path AS TargetPath,

                               DistributionMethod,

                               ID AS ResourceID,

                               MountPoint,

                               '$(TaskId)' AS TaskId

                            FROM [$(File)] (XmlSimple, Table is [DistributeTask/Resources/Resource]);

                           

                            QlikViewServerTasks_ScheduleTrigger:

                            LOAD RecurrenceType,

                               RepeatEvery,

                               HourStart,

                               Enabled AS Enabled_Trigger,

                               ID AS TriggerId,

                                 '$(TaskId)' AS TaskId

                            FROM [$(File)] (XmlSimple, Table is [DistributeTask/StartTriggers/ScheduleTrigger]);

                           

                            QlikViewServerTasks_TaskInfo:

                            LOAD *,

                            Upper (Container) as CONTAINER;

                            LOAD ID,

                               Generation,

                               Name AS TaskName,

                               Enabled AS Enabled_Task,

                               SendAlertMail,

                               AllowPluginClient,

                               AllowMobileClient,

                               AllowZFClient,

                               AllowPDFGeneration,

                               DownloadAccess,

                               ExportAccess,

                               NameTemplate,

                               SubField([SourceDocument/FileName], '\', SubStringCount([SourceDocument/FileName], '\') + 1) as SourceFileName,

                               [SourceDocument/FileName] as SourceFullFileName,

                            If (Index(SubField(Replace([SourceDocument/FileName], '$(vBaseSourcePath)' & '\', ''), '\', 3), '.qvw') > 0,

                            Null(),

                            SubField(Replace([SourceDocument/FileName], '$(vBaseSourcePath)' & '\', ''), '\', 3)) as Folder,

                            If (Index(SubField(Replace([SourceDocument/FileName], '$(vBaseSourcePath)' & '\', ''), '\', 3), '.qvw') > 0,

                            Null(),

                            SubField(Replace([SourceDocument/FileName], '$(vBaseSourcePath)' & '\', ''), '\', 2)) as Container,

                            If (Index(SubField(Replace([SourceDocument/FileName], '$(vBaseSourcePath)' & '\', ''), '\', 3), '.qvw') > 0,

                            Null(),

                            SubField(SubField(Replace([SourceDocument/FileName], '$(vBaseSourcePath)' & '\', ''), '\', 1), '.', 2)) as Environment,

                               '$(TaskId)' AS TaskId,

                               'Distribute task' as [Task type]

                            FROM [$(File)] (XmlSimple, Table is [DistributeTask]);

                            // End of [Task_e075c5e5-d47e-4b5e-87e2-ee46db3e7a82.xml] LOAD statements

                           

                           

                            QlikViewServerTasks_TaskInfo:

                            LOAD ID,

                               Generation,

                               Name AS TaskName,

                               Enabled AS Enabled_Task,

                               SendAlertMail,

                               Null() as AllowPluginClient,

                               Null() as AllowMobileClient,

                               Null() as AllowZFClient,

                               Null() as AllowPDFGeneration,

                               Null() as DownloadAccess,

                               Null() as ExportAccess,

                               Null() as NameTemplate,

                               Null() as SourceFileName,

                               Null() as SourceFullFileName,

                               Null() as Folder,

                               Null() as Container,

                               Null() as Environment,

                               Null() as CONTAINER,

                               '$(TaskId)' AS TaskId,

                               'External program task' as [Task type]

                            FROM [$(File)] (XmlSimple, Table is [ExternalProgramTask]);

                            // End of [Task_e075c5e5-d47e-4b5e-87e2-ee46db3e7a82.xml] LOAD statements

                            ENDIF

                           

                            IF not IsNull(TableNumber('isFileAnExistingTask')) Then

                            DROP Table isFileAnExistingTask;

                            ENDIF

                           

                            next File

                            for each Dir in dirlist ('$(Root)' & '\*' )

                            call DoDirTasks ('$(Dir)')

                            next Dir

                          end sub

                           

                           

                           

                           

                          sub DoDirTaskExecutionHistory (Root)

                            for each File in filelist ('$(Root)' & '\*.xml')

                           

                            LET TaskId = Replace(SubField('$(File)', '\', SubStringCount('$(File)', '\') + 1), 'Task_', '');

                           

                          // QlikViewServerTasks_TaskExecutionHistory:

                          // LOAD ObjectType AS RecipientObjectType,

                          //    ID AS RecipientID,

                          //    SubField(ID, '\', 1) AS Domain,

                          //    SubField(ID, '\', 2) AS [Group name],

                          //    '$(TaskId)' AS TaskId

                          // FROM [$(File)] (XmlSimple, Table is [DistributeTask/Resources/Resource/AddRecipient]);

                          //

                          // Directory;

                          // // Start of [TaskExecutionHistory_e075c5e5-d47e-4b5e-87e2-ee46db3e7a82.xml] LOAD statements

                            QlikViewServerTasks_TaskExecutionHistoryItems:

                            LOAD TaskId,

                               TaskLogId,

                               TaskStatus,

                               Date(Date#(StartTimeStamp, '$(vDateTimeFormatInFiles)'), 'YYYY-MM-DD hh:mm:ss') AS StartTimeStamp,

                               Date(Date#(StartTimeStamp, '$(vDateTimeFormatInFiles)') - DayStart(Date#(StartTimeStamp, '$(vDateTimeFormatInFiles)')), 'hh:mm:ss') AS StartTime,

                               Date(DayStart(Date#(StartTimeStamp, '$(vDateTimeFormatInFiles)')), 'YYYY-MM-DD') AS StartDate,

                               Date(MonthStart(Date#(StartTimeStamp, '$(vDateTimeFormatInFiles)')), 'YYYY-MM-DD hh:mm:ss') AS StartMonth,

                               Date(Date#(EndTimeStamp, '$(vDateTimeFormatInFiles)'), 'YYYY-MM-DD hh:mm:ss') AS EndTimeStamp,

                               If (Num(DayStart(Today())-DayStart(Date#(StartTimeStamp, '$(vDateTimeFormatInFiles)')), '0') < 7, 'Last week', Null()) as LastWeek,

                               Timestamp#(Duration, 'hh:mm:ss.fffffff') AS Duration,

                               Duration * 24 * 60 AS Duration_Minutes,

                               1 As TaskExecutionCount

                            FROM [$(File)] (XmlSimple, Table is [Root/TaskExecutionHistoryItems/TaskExecutionHistoryItem])

                            Where Exists (TaskId);

                            // End of [TaskExecutionHistory_e075c5e5-d47e-4b5e-87e2-ee46db3e7a82.xml] LOAD statements

                           

                           

                            next File

                            for each Dir in dirlist ('$(Root)' & '\*' )

                            call DoDirTaskExecutionHistory ('$(Dir)')

                            next Dir

                          end sub

                            • Re: Alert when a task is delayed or stuck
                              Ariel Topaz

                              Hi Johan

                              I have one question

                              the script that you send seek for .XML file

                              this is the only file with this ending

                              "DistributionReport.XML".


                              In addition all the files in this folder

                              "C:\ProgramData\QlikTech\DistributionService\1\Log\"


                              are very small ( about 1Kb)

                              when i reload the model I don't receive any data



                                • Re: Alert when a task is delayed or stuck
                                  Johan Lindell

                                  Hi Ariel,

                                   

                                  Sorry I was a bit quick copying the script. Missed the two lines that call the actual load of the files and one variable for the time format. Please try this tested and working app.

                                   

                                  Johan

                                   

                                  Missed these lines in the beginning:

                                   

                                  IF Len('$(vDateTimeFormatInFiles)') = 0 then

                                    LET vDateTimeFormatInFiles = 'YYYY-MM-DD hh:mm:ss';

                                  ENDIF

                                   

                                  And this at the end:

                                   

                                  Directory;

                                   

                                  FOR each File in filelist ('$(vFolderDistributionService)\$(vLimitFileName)')

                                    ExistingTasks:

                                    LOAD Replace(Replace(Name, 'Task_', ''), '.xml', '') AS TaskId

                                    FROM '$(File)' (XmlSimple, Table is [Root/Entry]);

                                    // End of [TaskDetails.xml] LOAD statements

                                  NEXT File

                                   

                                   

                                  call DoDirTasks ('$(vFolderDistributionService)' & '\Tasks')

                                  call DoDirTaskExecutionHistory ('$(vFolderDistributionService)' & '\TaskExecutionHistory')