0 Replies Latest reply: Nov 14, 2017 3:29 PM by Justin Dallas RSS

    Is this a bad loading idea?

    Justin Dallas

      Hello Folks,

       

      I'm at a location that has a Qlik Sense installation.  It is a single node, and there are no options currently to add another node no matter how small.  Because this node is small, and expectations of the QS system are increasing, my loads are running into resource contention problems.  Whereby a dashboard will try to read a file that is currently being written to by one of the many dataload apps.

       

      To resolve this I've thought about doing this.

       

      Current issues: The dataload  (creators of the qvd files) process is spread amongst 6 files.  The file whose load time is believed to be the longest is used as the Event Trigger for the later sections (Repository, Application).  However, sometimes this LongLoadApplication isn't the last one to load, meaning the other applications are triggered before their data is ready

       

      1. Group the loadings into logical hierarchies.  Repository, Datamodel, Application levels.

      2. These groupings are triggered by shim Apps loading i.e,

       

          1. RepositoryReloadTriggerApp    - 8:00AM; triggers the following apps

                - GetAppServerData1

                - GetSQLData2

                - GetExternalData3

       

          2. DataReloadTriggerApp - 8:30AM; triggers the following apps which depend on the above.

                - CreateDataModel1

                - CreateAccountingModel2

                - CreateLogisticsDataModel

       

           3. AppReloadTriggerApp - 9: 00AM; triggers the following apps to reload which depend on the datamodel

                -EndUserApp

                -LostSheepApp

       

      The function of this structure is to mimic a Java CountdownLatch which will wait for concurrent threads/processes to complete before continuing.

       

      So what do yall think of this idea?