Qlik NPrinting Governance Dashboard - Version 3 Now Available

[JonnyPoole]

To install or upgrade your existing Qlik NPrinting governance dashboard, simply deploy the new QVF  and reload the app: NPrinting Governance Dashboard - version3

If and only if you have altered the engine.config file(s) on your NPrinting Engine computer(s) to change the default 'resolvers-count' value,  then go to the 'deployment summary' sheet and enter the 'resolvers-count' value under 'Available Resolvers'.  This value sets a baseline that the new concurrency metric can evaluate to ensure your NPrinting environment is not core constrained. 

Please enjoy and post comments/suggestions in GitHub preferably, or in the community comments below.  

Version 3.0 (4/12/2022)

Update Details:

-Fixed a bug introduced by NP May 2021 SR3 where the reload fails in the load script:
< Field 'id' not found FROM "nprinting"."public"."task_execution" >

-Updated background colours on the "Task Recipients" Pivot Table to green

Version 2.0 

Update Details:

Deployment Summary Sheet:

• Configurable variable "vResolvers ".  By default this is set to the # logical engine processors. If you have changed  the "content-resolvers" property in the engine.config file(s)  to increase/decrease the number of resolvers in your NPrinting environment,  set vResolvers to the sum of this value in all your engine.config files.  

-New KPI "Peak  Connection Concurrency".  This is the number of unique connections used at any one time by executing publish tasks, subscriptions, on-demand requests, or metadata reload requests. It does not include preview requests.  If this number exceeds 60% of your resolver count, queuing is expected to occur and reports will take longer to run. The metric will turn red indicating it is time to add logical engine processors to your NPrinting Engine(s) to a maximum of 16 logical processors per engine. 

-New KPI "Peak Execution Concurrency". This is the number of concurrent execution requests at any one time . It is the sum of executing publish tasks + executing subscription requests + executing onDemand Requests + executing Metadata reload requests. If multiple reports are running in a single publish task, it counts as just '1' in this resolver . Does not include 'preview' requests.  

Report Delivery Sheet:

-New filters added

Data Connections Sheet:

-New filters added 

-Section Access field (yes/no ) added 

Qlik Lineage Sheet:

-New filters added

-New KPIs added:  Complex reports , Medium reports, simple reports

App Content Sheet:

-new filters added

Task Recipients Sheet:

-new filters added

Execution Analysis Sheet:

• New time chart:  "Active Connections by Type Over Time".  Shows concurrently used connections over time , stacked bar charts splits the type of request using the connection  (Publish Task, Subscription,OnDemand,Metadata Reload) . A Y Axis limit line shows the peak.  It turns red if it exceeds 60% of the vResolvers variable

-New table "Days when Concurrency Exceeded". Shows the number of days where the peak concurrency connections exceeds 60% of vResolvers.  

• New time chart:  "Executions by Type Over Time".  Shows concurrent executions over time , stacked bar charts splits the type of request   (Publish Task, Subscription,OnDemand,Metadata Reload) . A y axis limit line shows the peak over time.

-New table "Peak Concurrency by Day". Shows peak concurrency by day.

Report Performance Sheet:

-Container with 4 distribution plots showing execution length for publish tasks, subscriptions,ondemand requests and metadata reloads.  Colored by status (green = success,  red = failed)

-Container with 4 tables showing detailed executions

• 4 KPIs showing  "Average Publish Task Length", "Average OnDemand Length", "Average Subscription Length", "Average Metadata Reload Length" . 

Users Sheet:

-New Filters added

Execution Log Messages:

-New Filters added

Load Script / Model:

- vResolvers variable set to the sum of logical processors on each NP engine found

• section access field added to connections

• New master date table intervalmatched to executions. 

• vMonthsToLoad variable determines how many trailing months to generate in the master date table (default is 3 months)

Upgrade:

For anybody upgrading you will need to re-edit the data source in the load script and if and only if you have changed the resolvers-count setting in the NPrinting engine.config file .... then after reloading the app , go to the deployment summary sheet and update the vResolvers variable to be equivalent to the same value as 'resolvers-count' in the engine.config. If you have multiple NP engines, sum the number together and set vResolvers to the sum of the values found in the engine.config files.

[louiswinter]

@JonnyPoole We found that most of our reports were not associating to the tasks they are on.  After some investigation we found that the variable "vTaskReportsExist" isn't being set.  In your .qvf the variable is already exists with a value of 1 so the data is loaded.  When we run this in the QlikView load the variable doesn't get set, so section 5b didn't create any records.

UPDATE:
Even after setting vTaskReportsExist to 1 we found more reports not associated to tasks.  I ended up setting all of the variables used in the load to 1 (basically bypassing the logic to determine if the records exist) and now everything seems to be associating correctly.  I'm not sure what other variable(s) aren't being set but there must be at least one more.

[Senor_Dai]

hi @JonnyPoole ,  Just trying to configure this.

Is this the correct line to change to port 4992 ?

Many thanks!

Dai

[JonnyPoole]

@Senor_Dai   looks right.    remember to backup the file prior to the change.  Also remove the preceding '#' character to ensure the change is picked up (and not commented out) 

the instructions are here    https://github.com/eapowertools/NPrintingGovernanceDashboard/releases/download/v1.0/NPrinting.Govern...

[JonnyPoole]

@louiswinter    I'll check it out.    

[JonnyPoole]

@Louis  the usage of  vTaskReportsExist has been removed from the script and replaced with vPublishTaskReportsExist  which is a variable that existed all along.  I haven't located any other flaws with the conditional checks but i know I have limited Q/A capacity.  so please feel free to send me a direct message in github or community and we can solve any other gaps.  

[Louis]

@JonnyPoole
wrong
louis I think
@louiswinter

[louiswinter]

@Louis Got it, thanks.

[mjht]

@JonnyPoole  - I am trying to load the governance dashboard and getting an error in the ConnectionContent Dimension section and I am not sure why.  I did a limited load of 100 records and it finished so I don't know what the issue is.  I would attach the load script output with all the errors but I don't know how to attach a file.  At the end of the load it says 

Finished with error(s) and/or warning(s)
3 forced error(s)

Here is an example of the error ---

ReportXMLContent << property/class/properties/property
Lines fetched: 16
The following error occurred:
Cannot open file: '***'
The error occurred here:
Concatenate (ReportXMLContent)
LOAD
distinct
'Connection' as "ReportRecordType",
'89B16443-2CF1-428D-A0AF-89036AFE6680' as [ReportID],
upper(PurgeChar(property%Table,'{}')) as [ReportConnectionID]
From_Field ('SingleReportXML','XML') (XmlSimple,table is ***)
where name='DatabaseConnectionID'

[mjht]

I fixed it ---

I looked at my "nprinting"."public"."report" table in nprinting database and not all reports have data in the "entities_xml" field.  That is causing the error.  So I modified the script to skip the reports where the field was blank or null.  That worked.  I tried it with 4 reports, 3 had data in "entities_xml" and 1 did not - running with this fix eliminated the Cannot open file: '***'  error that occurs for the report that does not have data in "entities_xml".

In the Connection Content Dimension section I modified the load of Reports table to add

          if(len(trim("entities_xml")) > 0,1,0)  as  ReportHasXML

Then in the table below added a where clause so it only loads the records that have the field.

ReportXML:
LOAD
// 'Report' as "Record Type",
// "app_id" as [AppID],
ReportID,
ReportEntitiesXML,
// "title" as Title,
RowNo()-1 as RecordNumber
resident Reports
where ReportHasXML = 1;

There is another location in the Connection Content Dimension section where I had to put the XML check -  after // AppObjectsInReports the table ReportXML is created again for Qlik Entity reports:

where ReportTypeID<>'QLIKENTITY'
and ReportHasXML = 1;

[JonnyPoole]

@mjht   thank you.   Yes that does sound familiar.  The entitiesXML entry stores the repository's copy of the metadata tree that appears on the left side of the designer window when you have a report template open.  

I will PROBABLY have a version 2.0 soon for the NP Governance Dashboard .  I've added some diagnostic displays which will be handy for an administrator to look back and view execution concurrency over the past month.