6 Replies Latest reply: Jan 15, 2016 2:51 PM by Tyler Waterfall RSS

    Governance Dashboard – Performance Problems

    Thomas Rothauski

      Hello everybody,

       

      I installed the Governance Dashboard on one of our QV-Servers and changed the config to MultiCluster. The Problem is that the creation task run and run and run ... (several days...) is that normal?!?

       

      #Server1Server2Server3Server4Server5Server6Server6Server8Server9Server10Server11
      App.-Paths (Count)55555555555
      Logfile-PathCluster1Cluster2Cluster3Cluster3Cluster4Cluster5Cluster5Cluster5Cluster5Cluster5Cluster5
      Audit (2=NO)22222222222
      Publisher-PathQVD1QVD1QVD1QVD1QVD1QVD1QVD1QVD1QVD1QVD1QVD1

       

       

      If I try to start with only 2 Servers (and 10 App.) the duration is between (2 hours – 4 hours). Sometimes the task will finish successfully and sometimes it finish not successfully.. (the config is always identical..)

       

      Does anyone have any ideas how to solve that problem..?

       

      MultiCluster Script

       

      // 1. How many distinct will be scanned?

      SET vHowManyServers = 11;

       

      // 2. How many Aggregators are needed?

      SET vHowManyAggregators = 3;

       

      //3. How many years of server log history is needed?

      SET vHistoricalYears = 1;

       

      SET vMultiCluster_ProfilePath_1  = 'Governance_1.1_prod\profiles\X1';

      SET vMultiCluster_FilePath_1a   = '..._PROD';

      SET vMultiCluster_FilePath_1b   = '..._PROD';

      SET vMultiCluster_FilePath_1c   = '..._PROD';

      SET vMultiCluster_FilePath_1d   = '..._PROD';

      SET vMultiCluster_FilePath_1e   = '..._PROD';

      SET vExcludeFiles_or_Folders_1a  = '...';

      SET vExcludeFiles_or_Folders_1b  = '...';

      SET vExcludeFiles_or_Folders_1c  = '...';

      SET vExcludeFiles_or_Folders_1d  = '...';

      SET vExcludeFiles_or_Folders_1e  = '...';

      SET vMultiCluster_ServerPath_1   = '\\..\Cluster1\LogFiles';

      SET vMultiCluster_HasAudit_1   = 2; (2 = NO right?)

      SET vMultiCluster_PublisherPath_1  = '\\..\QlikTech\DistributionService';

      SET vMultiCluster_QVPRPath_1   = '...';

      SET vMultiCluster_PubDateTimeFormat_1  = 'DD.MM.CCYY HH:MI:SS';

       

      ....

       

      // Aggregator 1

      SET vMultiCluster_ProfilePath_101  = 'Governance_1.1_prod\profiles\AGG1';

      SET vMultiCluster_FilePath_101a  = '$(vMultiCluster_ProfilePath_1)';

      SET vMultiCluster_FilePath_101b  = '$(vMultiCluster_ProfilePath_2)';

      SET vMultiCluster_FilePath_101c  = '$(vMultiCluster_ProfilePath_3)';

      SET vMultiCluster_FilePath_101d  = '$(vMultiCluster_ProfilePath_4)';

      SET vMultiCluster_FilePath_101e  = '$(vMultiCluster_ProfilePath_5)';

      SET vClusterName_101a = 'X1';

      SET vClusterName_101b = 'X2';

      SET vClusterName_101c = 'X3';

       

      SET vClusterName_101d = 'X4';

      SET vClusterName_101e = 'X5';

       

      // Aggregator 2

      SET vMultiCluster_ProfilePath_102  = 'Governance_1.1_prod\profiles\AGG2';

      SET vMultiCluster_FilePath_102a  = '$(vMultiCluster_ProfilePath_6)';

      SET vMultiCluster_FilePath_102b  = '$(vMultiCluster_ProfilePath_7)';

      SET vMultiCluster_FilePath_102c  = '$(vMultiCluster_ProfilePath_8)';

      SET vMultiCluster_FilePath_102d  = '$(vMultiCluster_ProfilePath_9)';

      SET vMultiCluster_FilePath_102e  = '$(vMultiCluster_ProfilePath_10)';

      SET vClusterName_102a = 'X6';

      SET vClusterName_102b = 'X7';

      SET vClusterName_102c = 'X8';

      SET vClusterName_102d = 'X9';

      SET vClusterName_102e = 'X10';

       

      // Aggregator 3

      SET vMultiCluster_ProfilePath_103  = 'Governance_1.1_prod\profiles\AGG3';

      SET vMultiCluster_FilePath_103a  = '$(vMultiCluster_ProfilePath_11)';

      SET vClusterName_103a = 'X11';

       

      // GLOBAL Aggregator

      SET vMultiCluster_ProfilePath_201  = 'Governance_1.1_prod\profiles\GLOBAL';

      SET vMultiCluster_FilePath_201a  = '$(vMultiCluster_ProfilePath_101)';

      SET vMultiCluster_FilePath_201b  = '$(vMultiCluster_ProfilePath_102)';

      SET vMultiCluster_FilePath_201c  = '$(vMultiCluster_ProfilePath_103)';

       

      QVS-Logging Properties (identical on all Servers..):

       

      Logging

      [x] Enable Session Logging

      [x] Enable Performance Logging Every 15 Minutes

      [x] Enable Event Logging

      [ ] Enable Audit Logging

       

      Event Log Verbosity

      [x] Medium

       

      Split Files

      [x] Daily

       

      All Log-Files are stored on the same File-Share

       

      File-Size (QVS-Log-Files)

      • For one day: ~1Mbyte (Performance_, Events_, Sessions_,)
      • For 365 days: ~260 MByte (Performance_, Events_, Sessions_)
      • There are still some Audit_ Log-Files available which pump up the folder size to 20,2 GByte

       

      Trigger (Data Loading – Reproduction QVW)

       

      • It differs from (3 Minutes to once a day)
        • Re: Governance Dashboard – Performance Problems
          Tyler Waterfall

          This is intense!

          But, it looks like you have:

          55 distinct file folders

          5 server clusters (5 paths where the server logs are stored)

          1 Publisher (1 path where the publisher & qvpr logs are stored)

           

          Depending on the network bandwidth and speed, I am not surprised by the performance.

          However, are all 55 file paths unique? Meaning are there parents folders you could reference instead of 55 different folders?

           

          If so, you should be able to configure it to reload 5 "servers", each of which has one of the server log paths, one of which has the publisher path, and then the file folders divided up among the five.

           

          (For the question on Audit = ? use 0 instead of 2, though it shouldn't matter.)