Qlik Community

Qlik NPrinting Documents

Documents related to Qlik NPrinting.

Announcements
Document boards are being consolidated, this board no longer allows NEW documents READ MORE

NPrinting Governance Dashboard

cancel
Showing results for 
Search instead for 
Did you mean: 
JonnyPoole
Employee
Employee

NPrinting Governance Dashboard

Update June 2021***

NPrinting Governance Dashboard Version2 is out.

To upgrade , simply deploy the new QVF  and reload the app

https://github.com/eapowertools/NPrintingGovernanceDashboard/releases/tag/v2.0

If and only if you have altered the engine.config file(s) on your NPrinting Engine computer(s) to change the default 'resolvers-count' value,  then go to the 'deployment summary' sheet and enter the 'resolvers-count' value under 'Available Resolvers'.  This value sets a baseline which the new concurrency metric can evaluate to ensure your NPrinting environment is not core constrained. 

Please enjoy and post comments / suggestion in github preferably, or in the community comments below.  

This has been tested with NPrinting May 2021 and Qlik Sense May 2021 but it should work with NPrinting April 2020+ .  If you have an old Qlik Sense version it will still load but some of the visuals might not work.  I'm using some newer UI features in Qlik Sense. 

 

Update Summary:

- Updated Deployment Summary Sheet

- Updated Execution Analysis Sheet

- New Report Performance Sheet

- Various UI cleanups and additional filters.  More standardized format

- New configurable variable (vMonthsToLoad) to control how many months of data is loaded from the NPrinting repository.

-New configurable variable (vResolvers) which is set to the number of logical engine processors. 

 

Update Details:

Deployment Summary:

- Configurable variable "vResolvers ".  By default this is set to the # logical engine processors. If you have changed  the "content-resolvers" property in the engine.config file(s)  to increase/decrease the number of resolvers in your NPrinting environment,  set vResolvers to the sum of this value in all your engine.config files.  

-New KPI "Peak  Connection Concurrency".  This is the number of unique connections used at any one time by executing publish tasks, subscriptions, on-demand requests, or metadata reload requests. It does not include preview requests.  If this number exceeds 60% of your resolver count, queuing is expected to occur and reports will take longer to run. The metric will turn red indicating it is time to add logical engine processors to your NPrinting Engine(s) to a maximum of 16 logical processors per engine. 

-New KPI "Peak Execution Concurrency". This is the number of concurrent execution requests at any one time . It is the sum of executing publish tasks + executing subscription requests + executing onDemand Requests + executing Metadata reload requests. If multiple reports are running in a single publish task, it counts as just '1' in this resolver . Does not include 'preview' requests.  

 

Report Delivery:

-New filters added

 

Data Connections:

-New filters added 

-Section Access field (yes/no ) added 

 

Qlik Lineage:

-New filters added

-New KPIs added:  Complex reports , Medium reports, simple reports

 

App Content:

-new filters added

 

Task Recipients:

-new filters added

 

Execution Analysis:

- New time chart:  "Active Connections by Type Over Time".  Shows concurrently used connections over time , stacked bar charts splits the type of request using the connection  (Publish Task, Subscription,OnDemand,Metadata Reload) . A Y Axis limit line shows the peak.  It turns red if it exceeds 60% of the vResolvers variable

-New table "Days when Concurrency Exceeded". Shows the number of days where the peak concurrency connections exceeds 60% of vResolvers.  

- New time chart:  "Executions by Type Over Time".  Shows concurrent executions over time , stacked bar charts splits the type of request   (Publish Task, Subscription,OnDemand,Metadata Reload) . A y axis limit line shows the peak over time.

-New table "Peak Concurrency by Day". Shows peak concurrency by day.

Report Performance:

-Container with 4 distribution plots showing execution length for publish tasks, subscriptions,ondemand requests and metadata reloads.  Colored by status (green = success,  red = failed)

-Container with 4 tables showing detailed executions

- 4 KPIs showing  "Average Publish Task Length", "Average OnDemand Length", "Average Subscription Length", "Average Metadata Reload Length" . 

Users:

-New Filters added

Execution Log Messages:

-New Filters added

 

Load Script / Model:

vResolvers variable set to the sum of logical processors on each NP engine found

- section access field added to connections

- New master date table intervalmatched to executions. 

- vMonthsToLoad variable determines how many trailing months to generate in the master date table (default is 3 months)

 

 

 

 

 

 

 

 

//********************************************************************************************//

Hello, 

I've released the NPrinting Governance Dashboard at the following location. 

https://github.com/eapowertools/NPrintingGovernanceDashboard/releases/tag/v1.0

It is a Qlik Sense App that loads a large amount of information from the NPrinting PostgreSQL repository to help administrators better visualize and understand their NPrinting Deployments. 

summary.JPG

Flow.JPG

 

The app will load and display details about the NPrinting environment whether you use Qlik Sense as the back end or QlikVIew.  But the app itself is a Qlik Sense App. 

recipents.JPG

Requires the use of NPrinting April 2020 and later.  Earlier versions of NPrinting are not supported. 

Sheets include:

1. Deployment Summary

     - Summarizes the version , features deployed and amount of content deployed

2. Report Delivery Overview   

   -   Shows directional report names and volume to desitnations and recipients per month

3.  Data Connections

   -   Over view of all data connections and their status

4.  Qlik Lineage

   -   Shows the lineage from Qlik App -> App Object  -> Report as well as a score of the reports complexity vs executions

5. App Content

    -  Aggregates a list of all tasks, reports, filters etc.. by NPrinting App . Also shows dependencies between all objects (  Tasks -> Reports -> Connections ,   Report Filters -> Filters -> Connections ,  Task Conditions ->  Conditions -> Connections etc...) 

6. Task Recipients

   - Shows a grid of all groups and recipients and which tasks / subscriptions they are configured to receive

7. Execution Analysis

    - Lists all Publish Tasks, Subscriptions, and OnDemand requests showing a range of each execution time

8. Users 

    -  Lists all users and the groups, roles, tasks, subcriptions, ondemand requests that they own or belong to

9. Execution Log Messages 

     -  Shows all messages from the repository about executions.  Does not load information from the logs themselves.

 

Labels (2)
Comments
JonnyPoole
Employee
Employee

@Steven_Haught    thats a good enhancement request.  i'll jot it down for if/when i look to expand the app.   Off the top of my head these are three that i've heard that would take "some effort" to include. 

1 - A QVW version

2 - A future schedule / execution map 

3 - Integration with Nov 2020 Audit Trail feature

JonnyPoole
Employee
Employee

@rafalcqlik   .  I have thought about it but was concerned about scope creep.  It may be that the audit trail feature just needs a whole 2nd Qlik Sense app to analyze the new audit logs and do them justice.  The new audit logs  are prepared from the brand new postgresql NP audit database.    In Qlik Sense there isn't any one analyzer app that does everything and the NP Governance Dashboard has tons of tables and fields in it already.  

0 Likes
JonnyPoole
Employee
Employee

Hi just a heads up everyone.  

With the release of NPrinting November 2020 , there was a minor change in the NPrinting repository schema that causes an error when reloading the NPrinting Governance Dashboard. 

A patch was added to the github repository.  

Please download the latest NPrinting Governance Dashboard when you upgrade to NPrinting November 2020.  https://github.com/eapowertools/NPrintingGovernanceDashboard/releases/tag/v1.0

There was also a bug with the 'complexity' analysis page and all reports showed as 'complex'  .   That has also been fixed in the patch released yesterday  (11/24/2020). 

cheers, 

rafalcqlik
Contributor III
Contributor III

Thanks!

One more thing I'd like to ask to be updated. In Execution Analysis sheet, there is that chart with average execution times. Time format on X axis is set to 'mm:ss'. Could you please change it to 'hh:mm:ss' in next release? We have few reports that take more than one hour to execute and it looks weird without hh part 🙂 Not a big deal and very easy to update in our environment but maybe there are more people that have such cases. 

Thank you!

0 Likes
JonnyPoole
Employee
Employee

Yep  .  i've seen a fair bit of hour long reports too. I made the update in github. I also noticed the very same measure was using time() which isn't right either.  The 4 usages of the measure on this sheet have all been updated to interval( <expression>,'hh:mm:ss') 

0 Likes
santhoshkumar
Contributor III
Contributor III

@JonnyPoole  Thanks for this wonderful dashboard. Currently this dashboard gives me last 30 days data. Is there a way i can extract the historic data by changing any date parameters? or extracting data from different tables?

JonnyPoole
Employee
Employee

@Santo   the app doesn't filter anything out.  The 30 day history is most likely tied to NPrinting.    For on-demand at least you can go past 30 days with this configuration option:  https://help.qlik.com/en-US/nprinting/November2020/Content/NPrinting/On-Demand/Admin-On-Demand.htm

i'll check if there are other internal settings that are limitting NP to 30 days

santhoshkumar
Contributor III
Contributor III

@JonnyPoole  Thanks for your reply. That article helps. when i queries the public.on_demand_request table it was having only 30 days data. I believe the settings needs to be corrected.

 

Please share your feedback if something comes up

JonnyPoole
Employee
Employee

So there is an "executions-dtf" switch that can be enabled to control history in days. 

<add key="executions-dtk" value="365" />

Here is a sample of a scheduler.config file with the setting added. 

JonnyPoole_0-1610390069197.png

keep in mind that this almost certainly going to make your NPrinting repository larger.  

Since its not a used setting i'm not sure the performance impact as more history then normal accumulates in the repository.  Its not a big repository in the first place but i would advise pausing and thinking about why you may need this ? 

For instance in November 2020 there is a new 'audit' feature that dumps more granular audit activity about NPrinting into a brand new audit database.  We will have more experience with accumulating detailed logging through this out of box feature then turning on a hidden setting that almost nobody knows about.  

So my recommendations for more history are to:

1)  investigate the new audit database capabilities .  it can even be used to develop a new audit logging Qlik Sense App.  You can size the audit database to be as big as you want. You may need more C: space on the NP server for this. 

2) Enable the setting above but not before resizing your NP Server component (this is not an engine featuer) on the machine running NP Server specifically.  You may need more C: disk space to accommodate more growth in the postgresql instance and later in the year more CPU .  

 

santhoshkumar
Contributor III
Contributor III

@JonnyPoole  thanks for the details. super helpful.

one doubt.

may be not relevant to this main topic but still asking.

is it possible to integrate the "plan monitor" (nprinting url/admin/plan) data in this dashboard?

Simply to track and understand how stressed the cpu cores are when the concurrency loads/ondemand goes up. how much waiting happening,etc..

0 Likes
Version history
Last update:
‎2021-06-29 02:04 PM
Updated by:
Contributors