Deprecation of Centralized Logging in Qlik Sense Enterprise on Windows

[Jamie_Gregory]

Hello Qlik Users!

We would like to announce the deprecation of the Centralized Logging service in Qlik Sense Enterprise on Windows by the end of 2021.  

We will remove the Centralized Logging service to make Qlik Sense Enterprise on Windows more reliable and performant.

Logging will continue to be available in file format. We are updating the monitoring applications to support this change and provide you with the same insights.  

With Qlik Sense Enterprise May 2021 and on, a fresh installation of Qlik Sense Enterprise on Windows will not deploy Centralized Logging.  

We will provide a firm date for the removal of the service later this year.

As always, feel free to contact us if you have any questions.

Thank you for choosing Qlik! 

Kind regards,

Qlik Global Support 

[jpjust]

Hi Sonja - We are planning to upgrade from Nov 2021 to Feb 2022 version and as I understand there won't be centralized logging. When we do the upgrade, the upgrade provides with updated monitoring app's that will have the updated script?

Thanks

[Sonja_Bauernfeind]

@jpjust  Are you currently using centralised logging for your monitoring apps?

The monitoring apps already allow for either the use of centralized logging or flat files, so if you have not been using centralized logging nothing will change.

Here are the instructions for how to configure it to use file logging: Configuring the Monitoring apps.

All the best,
Sonja 

[jpjust]

Ok Thanks Sonja. We use monitoring apps and we allow use of centralized logging to grab information from the log files from the Postgres instead of flat files. So I think I have to change the code a bit to route  to flat files instead.

Can you also point me to the correct qliksense upgrade (Feb 2022) steps document when you have a moment?

Thanks

[Sonja_Bauernfeind]

Hello again @jpjust !

I can recommend these two links: Upgrading Qlik Sense and Qlik Sense Upgrade Guide - Step-by-Step plus common issues and tips included.

All the best,
Sonja 

[jpjust]

Thank you Sonja for the information.

[jbaechle]

For anyone who is waiting for the Sense System Performance Analyzer to be updated in accordance with the deprecation of centralized logging it looks like you should stop waiting according to this:  https://community.qlik.com/t5/Security-Governance/Qlik-Sense-System-Performance-Analyzer-App/td-p/18...

[IuliaM]

Hello Everyone,

I am a Qlik Sense Admin, returnign to work after a 6 months gap (maternity leave) and I have indeed found that with my latest upgrade the centralized loggig is gone. 

the issue I have is that I had self-created an app that sends email notifications in case of failure to the app owner but also our team email address. this app ran just seconds after a faild task and was taking information from the logging DB. this is clearly wokring anymore except for the fact that it send an email when somethign fails, it does not include the correct task name.

does anyone have a different solution for email notifications or can anyone point me to the correct log folder where I can find information about task executions that I can load into the app so that I adjust my solution?

Many Thanks!

Iulia 

[SusanSewell]

We make use of the "monitor_apps_REST_Task" connection that is used by the monitor apps  for this purpose.  The lastExectionResult section will give you the log location and the failures are those with status = 8 ("FinishedFail").  You can send your email for the most recent failure for the given app id with the correct task/app names and log file attached.

[jpjust]

Hi Susan - Using which tool do you send email alerts with the log attached when an reload task fails?

If you don't mind, can you please share the solution?

Thanks very much!!

[SusanSewell]

For sending the emails, I wrote an app using the SMTP connector in the Qlik Web Connectors package. This app is chained to the tasks on failure.   As far as using the REST connector to get the task info,  here is a snippet from the app that builds the table with the failed tasks (sorry - it's a bit long)

//////////////////////////////////////////////////////////////
// TASKS - use the REST APIs to get failed task info
LIB CONNECT TO 'monitor_apps_REST_task';

RestConnectorMasterTable:
SQL SELECT
"id" AS "id_u6",
"createdDate",
"modifiedDate",
"modifiedByUserName",
"isManuallyTriggered",
"name" AS "name_u3",
"taskType",
"enabled",
"taskSessionTimeout",
"maxRetries",
"privileges" AS "privileges_u6",
"schemaPath",
"__KEY_root",
(SELECT
"id" AS "id_u0",
"name" AS "name_u0",
"appId",
"publishTime",
"published",
"savedInProductVersion",
"migrationHash",
"availabilityStatus",
"privileges" AS "privileges_u0",
"__KEY_app",
"__FK_app"
FROM "app" PK "__KEY_app" FK "__FK_app"),
(SELECT
"id" AS "id_u3",
"nextExecution",
"privileges" AS "privileges_u3",
"__KEY_operational",
"__FK_operational",
(SELECT
"id" AS "id_u2",
"executingNodeName",
"status",
"startTime",
"stopTime",
"duration",
"fileReferenceID",
"scriptLogAvailable",
"scriptLogLocation",
"scriptLogSize",
"privileges" AS "privileges_u2",
"__KEY_lastExecutionResult",
"__FK_lastExecutionResult",
(SELECT
"id" AS "id_u1",
"detailsType",
"message",
"detailCreatedDate",
"privileges" AS "privileges_u1",
"__FK_details"
FROM "details" FK "__FK_details")
FROM "lastExecutionResult" PK "__KEY_lastExecutionResult" FK "__FK_lastExecutionResult")
FROM "operational" PK "__KEY_operational" FK "__FK_operational"),
(SELECT
"id" AS "id_u4",
"name" AS "name_u1",
"privileges" AS "privileges_u4",
"__FK_tags"
FROM "tags" FK "__FK_tags")
FROM JSON (wrap on) "root" PK "__KEY_root";

[app]:
LOAD [id_u0] AS [id_u0],
[name_u0] AS [name_u0],
[appId],
[publishTime],
[published],
[savedInProductVersion],
[migrationHash],
[availabilityStatus],
[privileges_u0] AS [privileges_u0],
[__KEY_app],
[__FK_app] AS [__KEY_root]
RESIDENT RestConnectorMasterTable
WHERE NOT IsNull([__FK_app]);

[details]:
LOAD [id_u1] AS [id_u1],
[detailsType],
[message],
[detailCreatedDate],
[privileges_u1] AS [privileges_u1],
[__FK_details] AS [__KEY_lastExecutionResult]
RESIDENT RestConnectorMasterTable
WHERE NOT IsNull([__FK_details]);

[lastExecutionResult]:
LOAD [id_u2] AS [id_u2],
[executingNodeName],
[status],
[startTime],
[stopTime],
[duration],
[fileReferenceID],
[scriptLogAvailable],
[scriptLogLocation],
[scriptLogSize],
[privileges_u2] AS [privileges_u2],
[__KEY_lastExecutionResult],
[__FK_lastExecutionResult] AS [__KEY_operational]
RESIDENT RestConnectorMasterTable
WHERE NOT IsNull([__FK_lastExecutionResult]);

[operational]:
LOAD [id_u3] AS [id_u3],
[nextExecution],
[privileges_u3] AS [privileges_u3],
[__KEY_operational],
[__FK_operational] AS [__KEY_root]
RESIDENT RestConnectorMasterTable
WHERE NOT IsNull([__FK_operational]);

[tags]:
LOAD [id_u4] AS [id_u4],
[name_u1] AS [TagValue],
[privileges_u4] AS [privileges_u4],
[__FK_tags] AS [__KEY_root]
RESIDENT RestConnectorMasterTable
WHERE NOT IsNull([__FK_tags]);

[root]:
LOAD [id_u6] AS [id_u6],
[createdDate],
[modifiedDate],
[modifiedByUserName],
[isManuallyTriggered],
[name_u3] AS [RootName],
[taskType],
[enabled],
[taskSessionTimeout],
[maxRetries],
[privileges_u6] AS [privileges_u6],
[schemaPath],
[__KEY_root]
RESIDENT RestConnectorMasterTable
WHERE NOT IsNull([__KEY_root]);

DROP TABLE RestConnectorMasterTable;

/////////////////////////////////////////////////
// lists of all tasks
TASKS: NoConcatenate
Load
[createdDate] AS TaskCreateDate,
[modifiedDate] as TaskModifiedDate,
[modifiedByUserName] as TaskModifiedBy,
[isManuallyTriggered] as TaskManual,
[RootName] AS TaskName,
[taskType] as TaskType,
[enabled] as TaskEnabled,
[taskSessionTimeout] as TaskTimeout,
[maxRetries] as TaskRetries,
[schemaPath] as TaskSchema,
[__KEY_root] as TaskKey
resident root;
left join (TASKS)
Load
[name_u0] as AppName,
[id_u0] as AppID,
[__KEY_root] as TaskKey
resident app;

inner join (TASKS)
Load
[__KEY_operational] as OperationalKey,
[executingNodeName],
[status],
[startTime],
[stopTime],
[duration],
[fileReferenceID],
[scriptLogAvailable],
[scriptLogLocation],
SubField( [scriptLogLocation], '\', -1) as [LogFileName],
[scriptLogSize],
[__KEY_lastExecutionResult] as LastExecuteKey
resident lastExecutionResult
// "finishFail" in the time window
where [status]=8
AND [stopTime] > $(vLookBackWindow);