Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW

Qlik NPrinting Governance Dashboard - Version 3 Now Available

100% helpful (1/1)
cancel
Showing results for 
Search instead for 
Did you mean: 
JonnyPoole
Employee
Employee

Qlik NPrinting Governance Dashboard - Version 3 Now Available

Last Update:

May 18, 2022 6:54:02 PM

Updated By:

Frank_S

Created date:

Sep 17, 2020 12:43:20 AM

To install or upgrade your existing Qlik NPrinting governance dashboard, simply deploy the new QVF  and reload the app: NPrinting Governance Dashboard - version3

If and only if you have altered the engine.config file(s) on your NPrinting Engine computer(s) to change the default 'resolvers-count' value,  then go to the 'deployment summary' sheet and enter the 'resolvers-count' value under 'Available Resolvers'.  This value sets a baseline that the new concurrency metric can evaluate to ensure your NPrinting environment is not core constrained. 

Please enjoy and post comments/suggestions in GitHub preferably, or in the community comments below.  

 

Version 3.0 (4/12/2022)

Update Details:

-Fixed a bug introduced by NP May 2021 SR3 where the reload fails in the load script:
< Field 'id' not found FROM "nprinting"."public"."task_execution" >

-Updated background colours on the "Task Recipients" Pivot Table to green

 

Version 2.0 

Update Details:

Deployment Summary Sheet:

  • Configurable variable "vResolvers ".  By default this is set to the # logical engine processors. If you have changed  the "content-resolvers" property in the engine.config file(s)  to increase/decrease the number of resolvers in your NPrinting environment,  set vResolvers to the sum of this value in all your engine.config files.  

-New KPI "Peak  Connection Concurrency".  This is the number of unique connections used at any one time by executing publish tasks, subscriptions, on-demand requests, or metadata reload requests. It does not include preview requests.  If this number exceeds 60% of your resolver count, queuing is expected to occur and reports will take longer to run. The metric will turn red indicating it is time to add logical engine processors to your NPrinting Engine(s) to a maximum of 16 logical processors per engine. 

-New KPI "Peak Execution Concurrency". This is the number of concurrent execution requests at any one time . It is the sum of executing publish tasks + executing subscription requests + executing onDemand Requests + executing Metadata reload requests. If multiple reports are running in a single publish task, it counts as just '1' in this resolver . Does not include 'preview' requests.  

Report Delivery Sheet:

-New filters added

Data Connections Sheet:

-New filters added 

-Section Access field (yes/no ) added 

Qlik Lineage Sheet:

-New filters added

-New KPIs added:  Complex reports , Medium reports, simple reports

App Content Sheet:

-new filters added

Task Recipients Sheet:

-new filters added

Execution Analysis Sheet:

  • New time chart:  "Active Connections by Type Over Time".  Shows concurrently used connections over time , stacked bar charts splits the type of request using the connection  (Publish Task, Subscription,OnDemand,Metadata Reload) . A Y Axis limit line shows the peak.  It turns red if it exceeds 60% of the vResolvers variable

-New table "Days when Concurrency Exceeded". Shows the number of days where the peak concurrency connections exceeds 60% of vResolvers.  

  • New time chart:  "Executions by Type Over Time".  Shows concurrent executions over time , stacked bar charts splits the type of request   (Publish Task, Subscription,OnDemand,Metadata Reload) . A y axis limit line shows the peak over time.

-New table "Peak Concurrency by Day". Shows peak concurrency by day.

Report Performance Sheet:

-Container with 4 distribution plots showing execution length for publish tasks, subscriptions,ondemand requests and metadata reloads.  Colored by status (green = success,  red = failed)

-Container with 4 tables showing detailed executions

  • 4 KPIs showing  "Average Publish Task Length", "Average OnDemand Length", "Average Subscription Length", "Average Metadata Reload Length" . 

Users Sheet:

-New Filters added

Execution Log Messages:

-New Filters added

Load Script / Model:

- vResolvers variable set to the sum of logical processors on each NP engine found

  • section access field added to connections

  • New master date table intervalmatched to executions. 

  • vMonthsToLoad variable determines how many trailing months to generate in the master date table (default is 3 months)

 

Upgrade:

For anybody upgrading you will need to re-edit the data source in the load script and if and only if you have changed the resolvers-count setting in the NPrinting engine.config file .... then after reloading the app , go to the deployment summary sheet and update the vResolvers variable to be equivalent to the same value as 'resolvers-count' in the engine.config. If you have multiple NP engines, sum the number together and set vResolvers to the sum of the values found in the engine.config files.

 

Labels (1)
Comments
Ruggero_Piccoli
Support
Support

Great job! Thanks!

Lech_Miszkiewicz
Partner Ambassador/MVP
Partner Ambassador/MVP

Love it!!! Such a great app.

My comments below and possible room for further improvements which are not currently in but (i think) will not be too hard to add:

  • Add Filter details page - this app pulls in information regarding filters giving you info which filter is used and where, but does not expose values/expressions and types (is_advanced_search, is_numeric) of the filters applied. From my experience this is very valuable information allowing you to quickly apply search on "Value" column in "filter_field_value" table to give you full overview of filters which use particular field from Qlik app. I guess this should by quite simple change as it only requires you to add below code to your script + aliases to show user friendly name and/or small transformation to put is_numeric, is_advanced_search, evaluate into single column so its more user friendly... Obviously I can just add it by myself, but throwing an idea on how to enrich this app for all!
    • LOAD id,
      value,
      display_value,
      evaluate,
      is_numeric,
      is_advanced_search,
      filter_field_id,
      If(is_advanced_search='True','Advanced Search',
      If(evaluate='True','Evaluate',
      If(is_numeric='True','Numeric Value','Value'))) as Type,
      upper("filter_field_id") as FilterFieldID;
      SELECT "id",
      "value",
      "display_value",
      "evaluate",
      "is_numeric",
      "is_advanced_search",
      "filter_field_id"
      FROM "public"."filter_field_value";
  • Comment/Question regarding "Complexity markup on Qlik Lineage page: I understand the formula which calculates complexity is quite straight forward. My question is: Are the parameters used in the formula really considered by Qlik as valid when estimating report complexities? I mean all my reports (also those ones which I would consider as extremely simple)  will be complex as quite frankly it is very easy to go over those thresholds. This is just a question out of my curiosity - thats all.

Anyway - thanks very much for such a great effort. I really appreciate this app!!! 

cheers

JonnyPoole
Employee
Employee

hi @Lech_Miszkiewicz   .  Thank you for the feedback.  Yes the field level would be a whole new level of governance and oversight and powerful.  While it would be relatively easy to add the ability to drill into fields for filters,  I was contemplating and ultimately left out (for now) the idea of associating fields to not just filters, but reports, connections, conditions maybe tasks, and ultimately users. That would allow a lot lineage and dependency questions to be answered.   i feel like that is real target to harness the associative magic of the engine but i know it wouldn't be a simple fact table add,  i would need to add new fact type records for most of the fact tables in the model and new link table record tyupes for the link tables.   I've done a lot of that already and the structure is there (i think) to expand the model.  But i'll wait to see what other feedback there is before doing that.  Already the style of the app is one that will likely need updates with every version if/when the repository expands so i'll have to set aside time to keep up with things.  Anyways really appreciate the feedback there.    

For complexity,  i know and agree that most reports will come out as "complex" . To answer your , question its 'non-official' from a qlik standpoint and the rationale has to do with the # of content resolver requests that would be generated by the report and content resolution is the biggest part of report execution which is , basically waiting for NPrinting to issue and retrieve all the data requests from QV or QS   .  I do feel like a report with even a single level that iterates 3-4 objects can result in often dozens of separate associative queries (or content resolver requests). When you take into account all the objects, selection states that are driven by levels, pages, and cycles, it easily becomes a lot of requests. Having 5 objects or a single multiplier (level/page etc..) is enough to bump up the complexity  .  A 4 core  NP engine out won't have enough cores to handle 5 concurrent content resolver requests , and an 8 core engine won't be able to handle  10 requests.  So if you want to run a complex report (10 requests+)  quickly you should have a 12 core engine or engines.    Otherwise ,  and probably in spite of, there will be serial processing most of the time .  So even mildly complex reports can take a bit more time and that's where the designation comes in.  Hope that helps a bit.   if you feel there is another driver of complexity besides this kind of estimation of performance  / load please share .    Once again thank you very much

marcginqo
Partner - Creator
Partner - Creator

Hi @JonnyPoole,

Really great work, very helpful. Will give it a test drive for sure.

Marc

mikaelsc
Specialist
Specialist

it was written there all along 🙂

mikaelsc_0-1600864969542.png

 

 

Thanks ! 

haty
Contributor III
Contributor III

This is awesome! Thanks Jonny

I got an error (copy below) when reloading. There was no public.smtp_destination_to_to table in my database. I just commented out that part of the code and now works perfectly. We are running Qlik NPrinting February 2020 (Version: 20.0.2.0)

The following error occurred:
Field 'smtp_destination_id' not found
The error occurred here:
SQL SELECT "smtp_destination_id",
"email"
FROM "nprinting"."public"."smtp_destination_to_to"

rzenere_avvale
Partner - Specialist II
Partner - Specialist II

Hey @haty ,

I had the same 'issue' on a November 2019, but from my point that should be expected since one of the requirements is having "NPrinting April 2020 and later"
Just as you, I commented that section and everything worked smoothly

Riccardo

JonnyPoole
Employee
Employee

Correct.  In April 2020, new fields were introduced into the repository to support dynamically resolved additional email addresses in the publish task generated recipient emails.   

https://help.qlik.com/en-US/nprinting/September2020/Content/NPrinting/DistributionSchedulesAutomatio...

It's that enhancement that is responsilble for the load error in < APril 2020 versions.    

Commenting out may allow the load script to complete but i'm not able to comment on accuracy of the data .   

Steven_Haught
Creator III
Creator III

Has anyone attempted to build a scheduler off of this data? Is it possible to create a table that shows potential trigger times? This would be the next biggest piece of help in a dashboard such as this! Really great work though, and thank you for the detailed document on how to implement. 

rafalcqlik
Contributor III
Contributor III

Hi @JonnyPoole 

Great application, we have that deployed and is working perfectly.

November 2020 version introduced Audit Trail feature. Are You planning to extend Governance dashboard by adding details from Audit Trail?

Version history
Last update:
‎2022-05-18 06:54 PM
Updated by: