Qlik Community

Qlik Sense Enterprise Documents & Videos

Documents & videos about Qlik Sense.

Announcements
BARC’s The BI Survey 19 makes it official. BI users love Qlik. GET REPORT

Qlik Sense task failure report with Nprinting

Partner
Partner

Qlik Sense task failure report with Nprinting

Background:

As there is no built in task failure report in Qlik Sense, we decided to create one, with Nprinting. Together with IIS, we can provide a user-friendly interface for troubleshooting task failures, only using products from Qlik and Microsoft. 

These days we can trigger NPrinting jobs from Qlik Sense QMC, although is not always reliant in the notification for sysadmins. In order to combat this, we are reading data directly from the NPrinting API, and fetching error messages from the NPrinting scheduler logs to aid the search for erros. Most of the information is _only_ available from the past 31 days and if there was an error, not a success. The intention here is to aid the system administrator to find errors more easily, but as the information is not 100% consistent, if there is a log fil missing - a link missing or similar, one can always search for it directly in the NPrinting WebConsole, QMC Tasks or locally on disk.

The reading from logfiles and APIs is made in a copy of the original "Reload Monitor" with some slight modifications. It is bundled with newer versions of Qlik Sense. "Reload Failures" is the new name of our modified app, and a couple of new tabs within the script is added with some alterations, along with some new sheets for the front-end.

The actual Qlik Sense app (Reload Failures) runs a chained task  (SYSADMIN Reloads Failures 24h summary) at the end of our task chain, or on a set time after a nightly reload schedule. Our tasks thus should triggers both on a successful or a failed reload parent in the task chain, so we can use this to send out a task failure report. This means we will reload and check if there are any errors every 24 hours, irrelevant if there has been any task success or failure. Naturally, this task failure report being reloaded can be adjusted to be independent of the task chain, or to be run after each reload chain (see below). The actual NPrinting report for task failures is using conditions, meaning we haven't sent out an email just yet.

After the Qlik Sense task is run, we run another task (SYSADMIN NPTrigg_ADMIN Task failures on SERVERNAME past24), which triggers the NPrinting API to start an NPrinting task, which is the job which sends out an email. The report being used reads data from the QS app, parent in the task chain.

The jobs in Nprinting utilizes parts of the added script sections to trigger a report only if there is an error, built on the copy of “Reload Failures”. In NPrinting this is based on "conditions".

In the email that get sent out, we link to the individual logfiles, directly from the email - same links as in the QS app. This is hosted by IIS on the same server, but different port.

The API and logfiles from NPrinting allows us to compile and read errors from both products, with user-friendly links to the actual logfiles. Unfortunately NPrinting logs only outputs IDs and filenames to its tasks if there are errors, but technically that is fine - for a task failure report.

 

What we have in the end is a task failure report in email, covering both NPrinting and Qlik Sense, with some neat links hosted by IIS directly to script files, both Qlik Sense logs, and NPrinting:

2019-08-07 13_22_00-Task failure report on BIPORTAL - past 24h - Message (HTML).png

2019-08-07 14_24_32-Task failure report on BIPORTAL - past 24h - Message (HTML).png

 

Requirements:

  • Nprinting February 2019 or later
    • Firewall ports to API
    • Connectivity to Qlik Sense
  • Qlik Sense February 2019 or later
    • Firewall ports to IIS (see below)

Recommendations:

  • IIS (same server or separate)
  • Patience

 

News (2019-08-07):

Since we started with this feature, Qlik has added API support to it's product. This gives the benefit to:

  • Read reload failures from logs and then immediately send out an alert email.
  • Read more information from NPrinting system.

The first one will allow us to use a task chain, as reported by Gianluca Perin, see here. We have since added status check for the task in the actual report, for both NPrinting and Qlik Sense, meaning if you trigger your reports to NPrinting from Qlik Sense scheduling, you can read them both in your task failure report.

This is thus what a trigger can look like, where the QS task is first, followed by a chain with an NP trigger:

2019-08-07 13_15_31-Clipboard.png

An example is included in this document.

 

Here is what you need to do in Qlik Sense ("SERVERQS"):

Download and import the "Reload Failures" copy attached. All sections named "EDIT" within the script contains modified code.

Follow this guide to add new libraries for:

  • NPrinting server GET
  • NPrinting server POST
  • Also add one for NPrinting logfiles

You need to update the corresponding variables and ensure that connectivity is working to both NPrinting server and QS server.

Schedule a reload at the end of a chain.

 

Setting up local IIS access to logfiles ("SERVERQS"):

This allows both the email being sent out, and the QS app, to be able to access logfiles directly.

Install IIS on the server through Roles and Features.

2019-08-07 14_03_08-mRemoteNG - confCons.xml - QlikSense-MS557 DEV.png

In order not to create a port conflict, we add a new port (444) to IIS. We bind a certificate to it, which ideally is the same one as Qlik Sense use. No need to use port 80 really, in our case we use it to redirect users to HTTPS. If you don't know how to set up and bind a certificate in Qlik Sense, have a look here. We are going to use the same certificate in this case, but that is not necessary.

Right click on Default Web Site, and "Add Virtual Directory". Add both ArchivedLogFolder and ServerLogFolder, e.g.:

2019-08-07 14_08_30-mRemoteNG - confCons.xml - QlikSense-MS557 DEV.png

Ensure that authentication is set to "Windows Authentication" as we don't want any people not being allowed to read the logfiles access it. You can set the proper rights within Windows "Security" tab on the folder itself.

2019-08-07 14_10_29-mRemoteNG - confCons.xml - QlikSense-MS557 DEV.png

 

Within Nprinting ("SERVERNP"):

Go to the Management Console of Nprinting.

Add a new app, e.g. "Task failure report", and a new connection to your new copy "Reload Failures".

The connection should include the proxy address to the server https://servername.domain.local/ and a Sense App ID. You need to be able to extract objects from your Qlik Sense server, see the help site for more info.

No filters are needed.

Conditions:

Add a new condition, call it "Failures over the past 24 hours"

Settings:

Variable, vFailures24hNew, >_ , Constant , 1

If you are using "since last run", you can use the variable "vFailuresSinceLastRun" in the same way.

 

Report:

Import the NPrinting report attached to this document called "Task failure HTML past 24h...", which is an Excel template.

You can choose to import the similar "Task failure HTML past run" if you wish to run the job more often.

 

As you can see above, by restricting data to only show what we have within our filter, only actual failures will be included. The reload summary table and reload details are official tables within the app.

 

Publish task:

Create a new NPrinting publish task, which you can name "Task failures on SERVERNAME".

Include the report as an embedded report to the email (no reports attached) and send to the relevant users and/or to the newsstand or unified hub. No filters are needed but add the condition above ("Failures over the past 24 hours") to be set as "true". That way you will only get emails if there have been any failures during the past 24 hours.

All text that is needed is included in the email.

If you want the app to reload more often, you may include the "since last run" report in another email report.

 

The text should pick up the appid to the official "Reload Failures" as soon as it is reloaded. Address to the Task sheet within the app is easily accessible for the administrator. 

 

Note that if upgrading, remember to make a copy of the new "Reload Failures" and include your "task-failure script", in order to stay up to date.

 

Good luck!

Attachments
Comments
deepti_singh
Contributor II

Hi Erik,

I am trying your approach and ran into an issue. I created a copy of the Operations Monitor and added a tab with the recommended script. When I reload it, I get an error " Error: Table 'ReloadSummary' not found", though I can see the table ReloadSummary in the data model as shown in the screenshot mentioned below.

I checked for the spelling, case etc.

Can you please suggest what I am doing wrong?

Thanks,

Deepti

0 Likes
Partner
Partner

Hi there Deepti,

Which version are you using? I have only tested this with 3.2.3 of Qlik Sense Server.

Is the table drop beforehand?

Is your added table at the very end of the script?

Add it as a Section at the end(bottom) of your copy of Operations Monitor.

Cheers

0 Likes
deepti_singh
Contributor II

Hi Erik,

I am using Qlik Sense 3.2.0 and NPrinting 17.3. I added the script recommended by you at the end of the script of the Operations Monitor.

I did a quick test where I reloaded a copy of the Operations Monitor dashboard (without adding the script) that ran today and encountered the same error. Looks like everyday when the dashboard runs (after all the tasks run), it runs successfully. But when I make a copy of it and reload it throws the error "Error: Table 'ReloadSummary".

Maybe we can take this offline (I can share my emai id) so that I can share the two log files (the one that ran successfully and the one that fails to reload without even adding the script)?

Thanks,

Deepti

0 Likes
Partner
Partner

Hmm. Then it might not have something to do with my script.

Can you just add the script log here?    

0 Likes
deepti_singh
Contributor II

Sure Erik.




Since I cannot upload files here, please find below the scripts for the two log files:


[Log files removed by moderator due to size]

0 Likes
deepti_singh
Contributor II

Erik,

It looks like the copy of the Operations Monitor (OM) dashboard (without your script) was failing because it was running at the same time as the original OM dashboard. But when I changed the schedule, it runs fine.

Now I added your script in the copy of the OM dashboard and it fails due to the same error mentioned above. Looks like the script is throwing an error now.

0 Likes
Partner
Partner

Hi Deepti,

As you can see on line 1643 it already fails before my script.

From what I can see in the beginning you are using an older version. Are you reloading on Desktop? Please try the server.


2017-08-18 08:39:50 0009 LET startMsg     = 'Reloading Operations Monitor 3.1.0 from (servername) running version 3.2.0'

2017-08-18 08:39:50 0010 TRACE Reloading Operations Monitor 3.1.0 from (servername) running version 3.2.0

2017-08-18 08:39:50 0010 Reloading Operations Monitor 3.1.0 from (servername) running version 3.2.0

Whereas on mine it says:

2017-08-24 06:45:07 0020 LET startMsg = 'Reloading Operations Monitor 3.3.4 from ACTUALSERVERNAME running version 3.2.0. ' & If(db_v_file_override=2,'Database logs chosen',if(db_v_file_override=1,'File logs chosen','Default log source selected (will check database first, then files)'))

2017-08-24 06:45:07 0021 TRACE Reloading Operations Monitor 3.3.4 from ACTUALSERVERNAME running version 3.2.0. Default log source selected (will check database first, then files)

2017-08-24 06:45:07 0021 Reloading Operations Monitor 3.3.4 from ACTUALSERVERNAME running version 3.2.0. Default log source selected (will check database first, then files)

2017-08-24 06:45:07 0022

For some reason yours doesn't find the servername nor uses a newer version. Please try to upgrade. Also ensure that there have been some reloads the past 24 hours.

0 Likes
Version history
Revision #:
3 of 3
Last update:
‎2019-08-07 08:52 AM
Updated by:
 
Contributors