Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW

Sending e-mail alerts in Qlik Sense Enterprise Task Failures

No ratings
cancel
Showing results for 
Search instead for 
Did you mean: 
Clever_Anjos
Employee
Employee

Sending e-mail alerts in Qlik Sense Enterprise Task Failures

Last Update:

Apr 26, 2018 7:31:47 PM

Updated By:

Clever_Anjos

Created date:

Apr 26, 2018 7:31:47 PM

There´s a common request to any Qlik Sense Enterprise Administrator:

"Is there any way to be informed by e-mail when a reload task fails?"  Yes, there is!

Before that, let´s have some technical small talk (you can jump to session "Creating your config xml" if you want)

Qlik Sense Enterprise relies on log4Net a very mature Apache Project  that is very reliable and powerful. It, besides a lot of features  , has the concept of Appender. That concept means that the application being logged (Qlik Sense in our case) does not to take care where the log is being stored, log4net config files handles this.

You can change a config file and choose if your log are meant to be stored into a database, a flat file, a hadoop data lake and you can even send an e-mail for each error message.   Qlik Sense Enterprise help to Appenders

1 - Creating your config xml

Pick your favorite text editor and write your config file according this https://logging.apache.org/log4net/release/config-examples.html

No, wait!

I´m posting here a functional config file, you can copy and paste it.

I´m supposing you´re going to use Gmail as the SMTP but is up to you chose any working SMTP that you want

<?xml version="1.0"?>

<configuration>

    <!-- Mail appender-->

    <appender name="MailAppender" type="log4net.Appender.SmtpAppender">

        <filter type="log4net.Filter.LevelRangeFilter">

            <param name="levelMin" value="WARN" />

            <param name="levelMax" value="ERROR" />

        </filter>

        <filter type="log4net.Filter.DenyAllFilter" />

        <evaluator type="log4net.Core.LevelEvaluator">

            <param name="threshold" value="ERROR"/>

        </evaluator>

        <param name="to" value="admin@domain.com" /> <!-- please use your admin e-mail address here -->

        <param name="cc" value="someone.else@domain.com" />

        <param name="from" value="yourgmailaccount@gmail.com" /> <!-- you´re supposed to change this -->

        <param name="subject" value="Qlik Sense Task Fail!!!" />

        <param name="smtpHost" value="smtp.gmail.com" />

        <param name="port" value="587" />

        <param name="EnableSsl" value="true" />

        <param name="Authentication" value="Basic" />

        <param name="username" value="yourgmailaccount@gmail.com" /> <!-- you´re supposed to change this -->

        <param name="password" value="yourpasswordforGmail" /> <!-- you´re supposed to change this -->

        <param name="bufferSize" value="0" />

        <!-- Set this to 0 to make sure an email is sent on every error -->

        <param name="lossy" value="true" />

        <layout type="log4net.Layout.PatternLayout">7

<!-- You can change below code if you want to pretify your e-mail message -->

            <param name="conversionPattern" value="%newline%date %-5level %newline%property{TaskName}%newline%property{AppName}%newline%message%newline%newline%newline" />

        </layout>

    </appender>

    <!--Send mail on task failure-->

    <logger name="System.Scheduler.Scheduler.Slave.Tasks.ReloadTask"> <!-- Filter what kind of exceptionname is beeing captured -->

        <appender-ref ref="MailAppender" />

    </logger>

</configuration>

2 - Saving your config xml

You must save your file with LocalLogConfig.xml name and under C:\ProgramData\Qlik\Sense\Scheduler (at least until version April2018). Anything different from this will cause your configuration not work.

Capture.PNG


3 - Restarting your Scheduler Service


Check if any task is running before restarting the service!!

Capture.PNG



4 - Test

  • Create an app and create an very "silly always fail" script like Store notable into [lib://neverland/notable.qvd](qvd)
  • Schedule your app as usual (under QMC/Apps/[your app]/More options button/Create New Reload Task

    (https://help.qlik.com/en-US/sense/April2018/Subsystems/ManagementConsole/Content/create-reload-tasks...)

Get a cup of coffee and wait for your task fail and your alert e-mail arrives at your inbox.



This blog post was written with fundamental contribution of ile‌ that sent me a functional config file


Comments
Clever_Anjos
Employee
Employee

@mgb I need to test, any particular version are you interested in?

ChrisEvans
Contributor III
Contributor III

@Clever_Anjos  - Have you gotten this to work on Windows Server 2019 and 2022? 

Clever_Anjos
Employee
Employee

Hi @ChrisEvans , never tried. Are you facing any issues?

ChrisEvans
Contributor III
Contributor III

Hi @Clever_Anjos  - We have 3 environments. Only the 2016 server is working. The 2019 and 2022 Servers are not working. Any ideas

mountaindude
Partner Ambassador
Partner Ambassador

@ChrisEvans 
I have Log4Net appenders working also on Windows server 2019 and 2022.
Can't see what would be different - the appenders are a Log4Net feature that isn't really connected to the Windows server version being used.

But there may of course be differences when it comes to firewall settings, permissions etc. But just for the record - it's working also on those newer Windows versions.

 

That said, the alert emails you get will be very basic. 

That is why I created the Butler tool some 6-7 years ago.
Open source, free to use and modify if so desired.
Documentation here: https://butler.ptarmiganlabs.com/
Discussion forums here: https://github.com/ptarmiganlabs/butler/discussions

By now it's a pretty capable tool that monitors failed reloads, sends messages to Slack/Teams, enables parameter passing between reload tasks, secure and controlled file system access, start reload tasks via a simple REST API and much more. It's basically a multi-purpose tool that makes life easier for both Sense admins and developers.

Focusing on Butler's failed reload monitor feature, it uses Log4net to detect the failed reloads (or other Sense events of interest). The information in the Sense log file is sent to Butler, where it is refined and then forwarded to any combination of destinations.

An alert email can look like this:

mountaindude_0-1682951749348.png

Note the nice formatting of the email and the inclusion of the reload log in the alert email.

"Refined" above means (among other things) getting the script log of the failed reload. This is usually a huge time saver as the alert message will show right away where the script failed. Various task and app metadata is also retrieved and included in the alert message.
Bottom line is that you get a much more complete picture of what happened compared to using a plain Log4Net SMTP appender.

"Destinations" above are the various places to which Butler can send alert messages.
The full list is 

  • Email
  • Slack
  • MS Teams
  • Webhook
  • InfluxDB
  • New Relic
  • Signl4
  • MQTT

Butler also lets you control (using custom properties on the reload task) which tasks should generate alert messages.
You can also specify per reload task to what email address alerts should be sent.
Quite a lot of flexibility thus! 

More about Butler's alert emails here: https://butler.ptarmiganlabs.com/docs/concepts/alert-emails/
More about Butler's Teams/Slack messages: https://butler.ptarmiganlabs.com/docs/concepts/alerts-slack-teams/
Butler can optionally extract and save the reload logs of all failed reloads in a directory on disk: https://butler.ptarmiganlabs.com/docs/concepts/failed-reload-logs/

dana
Creator II
Creator II

Hi @Clever_Anjos 

Is there a way to find out the reason an email isn't sent?

Some sort of a log ?

The Sense server version is  is May 2022.

Thanks!

mountaindude
Partner Ambassador
Partner Ambassador

@dana There is no specific log for the Log4Net appenders. 


When the filter pattern in an appender matches part of a row in the log file(s) the appender is configured to look at, the appender will trigger and do whatever it is supposed to do (send an email, write to another log file, send a UDP message etc). 

This can indeed make it tricky to configure those log appender XML config files if you are not familiar with them.

So you just have to be very careful when creating that XML file, making sure it's correct both with respect to XML syntax (which I personally find to be a mess...) and contents (i.e. that it really looks for the text you are interested in etc).
Usually the log appenders will be picked up by Sense without any restarts, but to be on the safe side you can also try restarting the Sense service that the appender deals with.

Clever_Anjos
Employee
Employee

@dana , you get a perfect answer from @montaindude 

0 Likes
NirupamaSatya
Contributor
Contributor

Hi Mick,

 

This really helped us in configuring the failure alert but one thing we wanted to know is that,if we want to configure same way for success tasks too...could you please help us in setting it up in the same way?

 

Thanks,

Nirupama

mountaindude
Partner Ambassador
Partner Ambassador

@NirupamaSatya 

It is quite doable to create an XML appender file for Log4net, capturing successful reload tasks instead of failed ones.

Remember - what the XML appender does is to look for certain strings in the Sense log files.
If the string you look for relate to a failed reload task, then you get alerts when reload tasks fail.
If you search for a string that's written to Sense logs when reload tasks succeed, you get a reload-succeeded message.

If I was to do this I would extend the existing XML you have and add sections to capture successful reload tasks.
BUT, and this is important: You would then capture ALL successful reloads. Which could be a lot. 
If you plan on sending an email for each successful reload.... bad idea.
You could probably refine the XML appender to only capture some successful reload tasks... but you'd still get those rather basic emails that Log4Net create for you.
 

Over to the shameless self-promotion then.

The Open Source Butler tool that I mentioned in an earlier post in this thread also handles successful reloads these days. This is documented at the Butler documentation site.

If you let Butler store the successful-reload-tasks-data in InfluxDB, you can then use Grafana to create charts like this, showing the distribution of how long all successful reloads take.
Which can be super useful when optimising a busy Qlik Sense environment!

mountaindude_0-1707143775791.png

Bottom line is that if you want to get high-value insights like the above in real-time (rather than retrospectively using a Sense app, which is what the Operations Monitor does), you need some kind of add-on tool that builds on top of the raw info provided by Sense itself.
Butler and its sibling Butler SOS are such tools. 

More examples about how to use Butler (and it's sibling tools) are found over at ptarmiganlabs.com, for example this post that outline how Butler can monitor and alert on Windows services, as well as the successful-reload-task example mentioned above.

If you're interested in these topics you can sign up for notifications when new posts on ptarmiganlabs.com are published.

Version history
Last update:
‎2018-04-26 07:31 PM
Updated by: