Qlik Community

Qlik Support Discussions

Discussion Board for collaboration related to Qlik Support.

Highlighted
amithmurali
Contributor

Operations monitor reload task failed due to Connection error

Dear Team,

One of our clients operations monitor failed to reload from the task scheduler. We had recently upgraded the server to November 2018 patch 1. Everything was working fine and this issue  came up from past few days. 

Thanks in Advance. 


Please find the script log below. 

  • 2019-02-10 08:22:59      Execution started.
  • 2019-02-10 08:22:59      Qlik Sense Server Version     12.244.6
  • 2019-02-10 08:22:59      CPU Target                    x64
  • 2019-02-10 08:22:59      Operating System              Windows Server 2012 R2 Standard  (64 bit edition)
  • 2019-02-10 08:22:59      Wow64 mode                    Not using Wow64
  • 2019-02-10 08:22:59      Language                      0409 English English
  • 2019-02-10 08:22:59      Country                       USA United States United States
  • 2019-02-10 08:22:59      MDAC Version                  xxxxxxxxxxxxxxxx
  • 2019-02-10 08:22:59      MDAC Full Install Version     xxxxxxxxxxxxxxxx
  • 2019-02-10 08:22:59      PreferredCompression          2
  • 2019-02-10 08:22:59      EnableParallelReload          1
  • 2019-02-10 08:22:59      ParallelizeQvdLoads           1
  • 2019-02-10 08:22:59      EnableFlushLog                0
  • 2019-02-10 08:22:59      UserLogfileCharset            65001
  • 2019-02-10 08:22:59      OdbcLoginTimeout              -1
  • 2019-02-10 08:22:59      OdbcConnectionTimeout         -1
  • 2019-02-10 08:22:59      LongestPossibleLine           1048576
  • 2019-02-10 08:22:59      WebFileTimeout                0
  • 2019-02-10 08:22:59      WebFileUserAgent              Mozilla/5.0
  • 2019-02-10 08:22:59      DontUseInternetCache          1
  • 2019-02-10 08:22:59      ScriptWantsDbWrite            false
  • 2019-02-10 08:22:59      ScriptWantsExe                false
  • 2019-02-10 08:22:59      ICU_FOR_CHARACTER_CLASSIFICATION true
  • 2019-02-10 08:22:59      ICU_FOR_CODE_PAGES            true
  • 2019-02-10 08:22:59      ICU_FOR_COLLATION             true
  • 2019-02-10 08:22:59      ICU_FOR_LOCALE_INFO           true
  • 2019-02-10 08:22:59      ICU_FOR_SUPPORT_INFO          true
  • 2019-02-10 08:22:59      ICU_FOR_TIME_ZONE_INFO        true
  • 2019-02-10 08:22:59      LogFile CodePage Used:        65001
  • 2019-02-10 08:22:59      UseMemoryFillPattern          1
  • 2019-02-10 08:22:59      ScriptMode                    Standard (limits system access)
  • 2019-02-10 08:22:59      SequentialThreadCount         8
  • 2019-02-10 08:22:59      NumberOfCores (effective)     2
  • 2019-02-10 08:22:59      ReloadCodebase                Bnf
  • 2019-02-10 08:22:59      Reload Executed By            UserDirectory=INTERNAL; UserId=sa_scheduler
  • 2019-02-10 08:22:59      Process Executing             Qlik Sense Server
  • 2019-02-10 08:22:59      Process ID                    11144
  • 2019-02-10 08:22:59      Grpc version                  5.0.0
  • 2019-02-10 08:22:59      Protobuf version              3.4.0
  • 2019-02-10 08:22:59 0003 LET yr                                              = year(ReloadTime())
  • 2019-02-10 08:22:59 0004 SET copyright = 'Copyright 1993-2019 Qliktech International AB'
  • 2019-02-10 08:22:59 0006 REM *** *** *** *** *** *** *** *** ****** *** *** *** *** *** *** ******
  • 2019-02-10 08:22:59 0007 *** *** *** *** *** *** *** *** *** *** *** *** ***
  • 2019-02-10 08:22:59 0008 *** *** *** *** *** ****** *** *** ******
  • 2019-02-10 08:22:59 0009 *** *** *** *** *** ****** *** *** *** *** *** *** *** *** *** *** *** *** ***
  • 2019-02-10 08:22:59 0011 SET db_v_file_override           =             0
  • 2019-02-10 08:22:59 0013 REM *** *** *** *** *** *** *** *** ****** *** *** *** *** *** *** *** *** *********
  • 2019-02-10 08:22:59 0015 Let ReloadStartTime                  = now(1)
  • 2019-02-10 08:22:59 0016 Set ahora =
  • 2019-02-10 08:22:59 0016 SET msg =
  • 2019-02-10 08:22:59 0016 SET skipped=0
  • 2019-02-10 08:22:59 0016 SET loaded =0
  • 2019-02-10 08:22:59 0016 SET textFile =
  • 2019-02-10 08:22:59 0017 SET app_name                                                            = 'Operations Monitor'
  • 2019-02-10 08:22:59 0018 SET app_version                                                         = '7.8.0'
  • 2019-02-10 08:22:59 0019 Let comp                                                                        = ComputerName()
  • 2019-02-10 08:22:59 0020 LET EngineVer = PurgeChar(EngineVersion(),chr(39))
  • 2019-02-10 08:22:59 0021 LET startMsg_1 = 'Reloading Operations Monitor 7.8.0 from AHQQLIK01 running QIX Engine version 12.244.6'
  • 2019-02-10 08:22:59 0022 LET startMsg                                                = 'Reloading Operations Monitor 7.8.0 from AHQQLIK01 running QIX Engine version 12.244.6. ' & If(db_v_file_override=2,'Database logs chosen',if(db_v_file_override=1,'File logs chosen','Default log source selected (will check database first, then files)'))
  • 2019-02-10 08:22:59 0023 TRACE Reloading Operations Monitor 7.8.0 from AHQQLIK01 running QIX Engine version 12.244.6. Default log source selected (will check database first, then files)
  • 2019-02-10 08:22:59 0023 Reloading Operations Monitor 7.8.0 from AHQQLIK01 running QIX Engine version 12.244.6. Default log source selected (will check database first, then files)
  • 2019-02-10 08:22:59 0024
  • 2019-02-10 08:22:59 0025 SET monthsOfHistory                               = 3
  • 2019-02-10 08:22:59 0026 LET cutoffDate                                                             = AddMonths(today(1),-3,1)
  • 2019-02-10 08:22:59 0027 Let LastReloadTime                                   = timestamp(alt(LastSuccessfulReloadStartTime,cutoffDate))
  • 2019-02-10 08:22:59 0028 Let lastReloadCompare                            = num(LastReloadTime)-1
  • 2019-02-10 08:22:59 0030 LET serverLogFolder                                 = 'lib://ServerLogFolder/'
  • 2019-02-10 08:22:59 0031 LET archivedLogsFolder                           = 'lib://ArchivedLogsFolder/'
  • 2019-02-10 08:22:59 0033 LET baseFileName                                    = 'governanceLogContent_7.8.0'
  • 2019-02-10 08:22:59 0034 LET baseTableName                                  = 'lib://ServerLogFolder/governanceLogContent_7.8.0'
  • 2019-02-10 08:22:59 0035 LET serviceFileName                                = 'governanceServiceLog_7.8.0'
  • 2019-02-10 08:22:59 0036 LET serviceTableName                             = 'lib://ServerLogFolder/governanceServiceLog_7.8.0'
  • 2019-02-10 08:22:59 0037 LET time_range_FileName                     = 'governance_time_range_ops_7.8.0'
  • 2019-02-10 08:22:59 0038 LET time_range_TableName = 'lib://ServerLogFolder/governance_time_range_ops_7.8.0'
  • 2019-02-10 08:22:59 0039 LET date_time_FileName                       = 'governance_date_time_ops_7.8.0'
  • 2019-02-10 08:22:59 0040 LET date_time_TableName    = 'lib://ServerLogFolder/governance_date_time_ops_7.8.0'
  • 2019-02-10 08:22:59 0041 LET monitorAppStatsFile                         = 'lib://ServerLogFolder/Operations_Monitor_Reload_Stats_7.8.0.txt'
  • 2019-02-10 08:22:59 0043 SET hideprefix                                                             = 'log'
  • 2019-02-10 08:22:59 0044 SET firstReload                                            = 0
  • 2019-02-10 08:22:59 0047 SET TimeFormat                         = 'hh:mm:ss'
  • 2019-02-10 08:22:59 0048 SET DateFormat                          = 'YYYY-MM-DD'
  • 2019-02-10 08:22:59 0049 SET TimestampFormat             = 'YYYY-MM-DD hh:mm:ss'
  • 2019-02-10 08:22:59 0051 Let vLast4Hours =       Num(timestamp(Now(1)-1/6))
  • 2019-02-10 08:22:59 0052 Let vLast24Hours =    Num(timestamp(Now(1)-1))
  • 2019-02-10 08:22:59 0053 Let vLast72Hours =    Num(timestamp(Now(1)-3))
  • 2019-02-10 08:22:59 0055 SUB verify_database
  • 2019-02-10 08:22:59 0140 SUB logList
  • 2019-02-10 08:22:59 0169 SUB defineFields
  • 2019-02-10 08:22:59 0278 SUB mappingLoads
  • 2019-02-10 08:22:59 0303 SUB load_base_table (nombre, archivo, timestamp_field)
  • 2019-02-10 08:22:59 0355 SUB multiNodeConfig
  • 2019-02-10 08:22:59 0386 SUB logFolderList
  • 2019-02-10 08:22:59 0429 SUB loadFiles (fdr,iter)
  • 2019-02-10 08:22:59 0481 SUB load_database_logs
  • 2019-02-10 08:22:59 0646 SUB calendarization
  • 2019-02-10 08:22:59 0731 SUB calendarization_add
  • 2019-02-10 08:22:59 0814 SUB concat_tables (concatToTable, incrementalTable, concatField)
  • 2019-02-10 08:22:59 0841 SUB store_files (nombre, archivo)
  • 2019-02-10 08:22:59 0864 SUB serviceLog
  • 2019-02-10 08:22:59 0909 SUB reloadSummary
  • 2019-02-10 08:22:59 0995 SUB sessionSummary
  • 2019-02-10 08:22:59 1073 SUB exportingSummary
  • 2019-02-10 08:22:59 1162 SUB monitor_app_stats_incremental // Use this to append new 'status' entry to table 
  • 2019-02-10 08:22:59 1176 SUB monitor_app_reload_stats (sev, obj, message, description)
  • 2019-02-10 08:22:59 1211 SUB QRS
  • 2019-02-10 08:22:59 1238 SUB monitor_apps_REST_user
  • 2019-02-10 08:22:59 1320 SUB monitor_apps_REST_app
  • 2019-02-10 08:22:59 1449 SUB monitor_apps_REST_appobject
  • 2019-02-10 08:22:59 1543 SUB monitor_apps_REST_task
  • 2019-02-10 08:22:59 1925 SUB supportingLogic
  • 2019-02-10 08:22:59 1983 SUB finalize
  • 2019-02-10 08:22:59 2022 CALL monitor_app_reload_stats('INFO','Operations Monitor', startMsg,'Reload Start')
  •   2019-02-10 08:22:59 1178 TRACE Working on Monitor App Reload Stats
  •   2019-02-10 08:22:59 1178 Working on Monitor App Reload Stats
  •   2019-02-10 08:22:59 1179
  •   2019-02-10 08:22:59 1180 IF description = 'Reload Start' THEN
  •     2019-02-10 08:22:59 1182 IF FileSize('lib://ServerLogFolder/Operations_Monitor_Reload_Stats_7.8.0.txt') > 0 THEN
  •       2019-02-10 08:22:59 1183 monitor_app_reload_stats:
  •       2019-02-10 08:22:59 1184       Load * From 'lib://ServerLogFolder/Operations_Monitor_Reload_Stats_7.8.0.txt' (txt, utf8, embedded labels, delimiter is '\t', msq)
  •       2019-02-10 08:22:59               7 fields found: Log Entry, Log Timestamp, Log Severity, Host, Description, Log Message, Object,
  •       2019-02-10 08:22:59      20,600 lines fetched
  •     2019-02-10 08:22:59 1185 ELSE
  •     2019-02-10 08:22:59 1191 Let appMonitorStatsRowsInit = NoOfRows('monitor_app_reload_stats')
  •     2019-02-10 08:22:59 1192 CALL monitor_app_stats_incremental                          // Add start message
  •       2019-02-10 08:22:59 1163 Concatenate (monitor_app_reload_stats)
  •       2019-02-10 08:22:59 1164     Load
  •       2019-02-10 08:22:59 1165       RowNo() as [Log Entry],
  •       2019-02-10 08:22:59 1166       timestamp(now(1)) as [Log Timestamp],
  •       2019-02-10 08:22:59 1167       'INFO' as [Log Severity],
  •       2019-02-10 08:22:59 1168       'AHQQLIK01' as Host,
  •       2019-02-10 08:22:59 1169       'Reload Start' as Description,
  •       2019-02-10 08:22:59 1170       'Reloading Operations Monitor 7.8.0 from AHQQLIK01 running QIX Engine version 12.244.6. Default log source selected (will check database first, then files)' as [Log Message],
  •       2019-02-10 08:22:59 1171       'Operations Monitor' as Object
  •       2019-02-10 08:22:59 1172     AutoGenerate (1)
  •       2019-02-10 08:22:59               7 fields found: Log Entry, Log Timestamp, Log Severity, Host, Description, Log Message, Object,
  •       2019-02-10 08:22:59      20,601 lines fetched
  •       2019-02-10 08:22:59 1174 ENDSUB
  •   2019-02-10 08:22:59 1194 ELSEIF description = 'Status Message' THEN
  •   2019-02-10 08:22:59 1208 ENDSUB
  • 2019-02-10 08:22:59 2024 CALL verify_database
  •   2019-02-10 08:22:59 0057 TRACE Verifying logging database.
  •   2019-02-10 08:22:59 0057 Verifying logging database.
  •   2019-02-10 08:22:59 0058
  •   2019-02-10 08:22:59 0059 REM *** *** *** *** *** *** *** *** *** *** *** ***
  •   2019-02-10 08:22:59 0061 IF db_v_file_override = 1 THEN
  •     2019-02-10 08:22:59 0072 LET db_check_time = timestamp(now(1)-0.01)
  •     2019-02-10 08:22:59 0073 Set errormode = 0
  •     2019-02-10 08:22:59 0075 LIB CONNECT TO 'QLogs'
  •    2019-02-10 08:23:00      Connected.
  •     2019-02-10 08:23:00 0078 LET tempErrorDetails = ScriptErrorDetails
  •     2019-02-10 08:23:00 0079 IF Len(tempErrorDetails) > 0 THEN
  •     2019-02-10 08:23:00 0093 db_check:
  •     2019-02-10 08:23:00 0094     SELECT "id"
  •     2019-02-10 08:23:00 0095     FROM "public"."log_entries"
  •     2019-02-10 08:23:00 0096     WHERE "entry_timestamp" >= '2019-02-10 12:08:35'
  •     2019-02-10 08:23:00      Error: Connector reply error: Unable to connect to database:
  •     2019-02-10 08:23:00          Connection failed:
  •     2019-02-10 08:23:00      Please check the values for Username‚ Password‚ Host‚ and other properties.
  •     2019-02-10 08:23:00 0100 LET tempErrorDetails = ScriptErrorDetails
  •     2019-02-10 08:23:00 0101 IF Len(tempErrorDetails) > 0 THEN
  •       2019-02-10 08:23:00 0102 trace ERROR: Connector reply error: ***
  •       2019-02-10 08:23:00 0102 ERROR: Connector reply error: ***
  •       2019-02-10 08:23:00 0103
  •       2019-02-10 08:23:00 0103 CALL monitor_app_reload_stats('WARN','', tempErrorDetails, 'Status Message')
  •         2019-02-10 08:23:00 1178 TRACE Working on Monitor App Reload Stats
  •         2019-02-10 08:23:00 1178 Working on Monitor App Reload Stats
  •         2019-02-10 08:23:00 1179
  •         2019-02-10 08:23:00 1180 IF description = 'Reload Start' THEN
  •           2019-02-10 08:23:00 1195 CALL monitor_app_stats_incremental                   // Add status message
  •             2019-02-10 08:23:00 1163 Concatenate (monitor_app_reload_stats)
  •             2019-02-10 08:23:00 1164     Load
  •             2019-02-10 08:23:00 1165       RowNo() as [Log Entry],
  •             2019-02-10 08:23:00 1166       timestamp(now(1)) as [Log Timestamp],
  •             2019-02-10 08:23:00 1167       'WARN' as [Log Severity],
  •             2019-02-10 08:23:00 1168       'AHQQLIK01' as Host,
  •             2019-02-10 08:23:00 1169       'Status Message' as Description,
  •             2019-02-10 08:23:00 1170       'Connector reply error: ***' as [Log Message],
  •             2019-02-10 08:23:00 1171       '' as Object
  •             2019-02-10 08:23:00 1172     AutoGenerate (1)
  •             2019-02-10 08:23:00        7 fields found: Log Entry, Log Timestamp, Log Severity, Host, Description, Log Message, Object,
  •             2019-02-10 08:23:00      20,602 lines fetched
  •             2019-02-10 08:23:00 1174 ENDSUB
  •         2019-02-10 08:23:00 1197 ELSEIF description = 'Reload Finish' THEN
  •         2019-02-10 08:23:00 1208 ENDSUB
  •       2019-02-10 08:23:00 0104 tempErrorDetails =
  •       2019-02-10 08:23:00 0105 TRACE Could not validate active database logging. Sourcing from file logs instead.
  •       2019-02-10 08:23:00 0105 Could not validate active database logging. Sourcing from file logs instead.
  •       2019-02-10 08:23:00 0106
  •       2019-02-10 08:23:00 0106 SET db_v_file = 1
  •       2019-02-10 08:23:00 0107 LET baseTableName = 'lib://ServerLogFolder/governanceLogContent_7.8.0_file'
  •       2019-02-10 08:23:00 0108 LET lastReloadCompare = If(LastReloadSource=1,lastReloadCompare,cutoffDate)
  •       2019-02-10 08:23:00 0110 TRACE Last Reload Compare time = 43505.467997685. CutoffDate = 2018-11-10.
  •       2019-02-10 08:23:00 0110 Last Reload Compare time = 43505.467997685. CutoffDate = 2018-11-10.
  •       2019-02-10 08:23:00 0111
  •       2019-02-10 08:23:00 0111 DisConnect
  •       2019-02-10 08:23:00 0112 SET errormode=1
  •     2019-02-10 08:23:00 0113 EXIT SUB
  • 2019-02-10 08:23:00 2026 REM *** *** *** ********* *** *** *** ***
  • 2019-02-10 08:23:00 2027 CALL load_base_table ('LogContent', 'lib://ServerLogFolder/governanceLogContent_7.8.0_file','LogTimeStamp')
  •   2019-02-10 08:23:00 0305 TRACE Checking for base qvd
  •   2019-02-10 08:23:00 0305 Checking for base qvd
  •   2019-02-10 08:23:00 0306
  •   2019-02-10 08:23:00 0306 Let ts_field = '"LogTimeStamp"'
  •   2019-02-10 08:23:00 0309 Let baseFileSize = FileSize('lib://ServerLogFolder/governanceLogContent_7.8.0_file.qvd')
  •   2019-02-10 08:23:00 0311 IF  baseFileSize > 0 THEN           // Yes - QVD exists = not first load
  •     2019-02-10 08:23:00 0313 trace Incremental reload of LogContent table (i.e. not first reload)
  •     2019-02-10 08:23:00 0313 Incremental reload of LogContent table (i.e. not first reload)
  •     2019-02-10 08:23:00 0314
  •     2019-02-10 08:23:00 0314 Let firstReload = 0
  •     2019-02-10 08:23:00 0316 LogContent:
  •     2019-02-10 08:23:00 0317         NoConcatenate
  •     2019-02-10 08:23:00 0318     Load * FROM [lib://ServerLogFolder/governanceLogContent_7.8.0_file.qvd] (qvd)
  •     2019-02-10 08:23:00 0319         WHERE LogTimeStamp >= '2018-11-10'
  •     2019-02-10 08:23:00                 51 fields found: Id, LogEntryPeriodStart, LogTimeStamp, Service, Hostname, Message, Export Count, Export Store Count, Description, ProxyPackageId, RequestSequenceId, _date_time_link, _proxySessionPackage, Reload from Hub, TaskExecution, TaskId, UserId, ObjectId, ObjectName, ProxySessionId, Context, Command, Result, App Name, ProductVersion, Sequence#, qmc_change, QMC Resource Type, SecurityClass, ClientHostAddress, Severity, ActiveDocSessions, ActiveDocs, ActiveUsers, CPULoad, LoadedDocs, Selections, VMCommitted, VMAllocated, VMFree, VMPctCommitted, Cache Hits, Cache Lookups, Cache Bytes Added, Session Duration, Session Selections, Session Count, Session CPU Spent (ms), Reload CPU Spent (ms), Session KBytes Sent+Received, Reload KBytes Sent+Received,
  •     2019-02-10 08:23:00      Error: I/O error  governanceLogContent_7.8.0_file.qvd
  •     2019-02-10 08:23:00      Execution Failed
  •     2019-02-10 08:23:00      Execution finished.

 

Community Browser