Deprecation of Centralized Logging in Qlik Sense Enterprise on Windows

[Jamie_Gregory]

Hello Qlik Users!

We would like to announce the deprecation of the Centralized Logging service in Qlik Sense Enterprise on Windows by the end of 2021.  

We will remove the Centralized Logging service to make Qlik Sense Enterprise on Windows more reliable and performant.

Logging will continue to be available in file format. We are updating the monitoring applications to support this change and provide you with the same insights.  

With Qlik Sense Enterprise May 2021 and on, a fresh installation of Qlik Sense Enterprise on Windows will not deploy Centralized Logging.  

We will provide a firm date for the removal of the service later this year.

As always, feel free to contact us if you have any questions.

Thank you for choosing Qlik! 

Kind regards,

Qlik Global Support 

[Pheaxx]

it's not so much if the data is not there, its more the time we need to put into changing the back-end of our monitoring tool to use the files instead of the DB. 

[mountaindude]

Got some news relating to this topic that I'd like to share. 

By now many of us have seen that new installations of Qlik Sense Enterprise no longer includes log db.
Still, having real-time access to warnings and errors is critical for running large Sense environments.

@rva_be-terna pointed out that tools like Butler SOS (butler-sos.ptarmiganlabs.com) pull data from log db - what will happen to them?

I can only speak for Butler SOS, but here is what's currently planned:

First, it will (of course) remain open source. Free to use and contribute to.
It will also keep the same real-time Sense metrics as before (number of users/sessions/apps, what apps are loaded, CPU load, RAM usage etc as today. It will also integrate with the same tools - Prometheus, InfluxDB, Grafana, MQTT etc.

The data pulled from log db will no longer be available, but work is ongoing to pull this data from the log files instead (I outlined this concept in an earlier post in this thread). 
The result is that Butler SOS will get very close to the same (maybe even exactly the same, remains to be seen) log events as today, but from the Sense log files instead. 

That's good, but there is more: 
Because the log db only included data from some log files, this new solution will allow us to tap into log event sources previously not available in log db. 
Hard to know right now how useful that will be - but having flexibility is usually a good thing.

If you are interested in this topic I invite you over to the Butler SOS home over at GitHub, and specifically a discussion about what log events are useful, missing, overrated etc.
Basically a crowd sourced requirement specification for future log events support in Butler SOS.

Direct link to the GitHub discussion/post: https://github.com/ptarmiganlabs/butler-sos/discussions/241

Cheers,
Göran

[lucienorrin]

When will a new version of the Sense System Performance Analyzer app show up?
I just installed Nov 2021 and it's broken out of the box with no logging DB.

[mountaindude]

Adding to my previous post earlier in this thread:

Open source tool Butler SOS version 7.0 adds real-time insight into warnings and errors from the log files.

It's not a full replacement for the now deprecated Qlik Sense log db, but it does address some of the features that log db did well.

Butler SOS also adds some new features not present in log db, like browser based real-time view into warning and error messages in the Sense log files.

This page explains the concept: https://butler-sos.ptarmiganlabs.com/docs/examples/qliksense_monitoring_grafana8/#warnings--errors

Butler SOS basically gets events whenever there is a warning, error or fatal message in the log files. These events are stored in a time-series database, after which they can be visualised in tools like Grafana - creating dashboards and takes such as the ones at the link above.

This particular table shows that very detailed information is available in Grafana. Previously you would have to dig through log files to get this info, now it's available in your web browser - you can even create various kinds of alerts based on these events!

Butler SOS is open source. https://butler-sos.ptarmiganlabs.com

[Thileg92]

We need to develop a Qlik dashboard on Qlik usage over a rolling period of one year , no need of real time metrics. It should be fed with data from the logs and we were thinking of using Centralized Logging data as a main source. Do you know a convenient solution for collecting the Qlik log data and save them in a db? is there any plan for replacing Centralized Logging soon?

[elyMalkiAcn]

Hi,

a Qlik Sense logging appender would be useful if more documented. Possible useful with integration with SysLog protocol, that is in most of enterprise platform. Any update on this side ?

BR,

Ely

[TusharDev_Qlik]

@Jamie_Gregory  we are planning to upgrade to either Qliksense May 2021 or Nov 2021. What changes exactly the deprecation of Centralized logging brings. I am not able to understand fully what will be the major difference between old and this new version going forward ? what precautions we should take while upgrading?

[Sonja_Bauernfeind]

Hello @TusharDev_Qlik 

Unless you had previously used centralised logging (which feeds log data into a database, rather than writing it to flat .log files), you will not be seeing any impact at all. If you have been using centralised logging, then you will now need to work with the .log files created for log reading/auditing.

All the best,
Sonja 

[TusharDev_Qlik]

@Sonja_Bauernfeind Thank you for your response. Can I get any detail document on this or how can I check If centralized logging has been used in our environment or not?

[Sonja_Bauernfeind]

Hello again @TusharDev_Qlik,

This should help: How To Verify if Qlik Sense Is Running With Centralized Logging Service . It documents how you can verify if you are running it currently or not.

All the best,
Sonja