5 Replies Latest reply: Feb 21, 2018 3:23 AM by Ivan Bozov RSS

    Data load

    Love Qliksense

      Hi all,

      Loading data in my app takes almost 25 min from script. How can I make it faster?

       

       

       

      Thanks

      In advance

        • Re: Data load
          Shilpan Patel

          There are multiple factors that may impact load time.

           

          1. Data volume

          2.CPU and RAM power of the machine on which reload runs

          3. Script itself may cause increased load time

          4. # of sources and tables

          • Re: Data load
            Chennaiah Nallani

            Implement incremental load and avoid Synthetic tables and circuital loop.

            • Re: Data load
              vishal waghole

              1. Make sure that you will only load required fields.

              2. Avoid synthetic keys and circular loop

              3. Use joining keys in number data types, if you have in character then convert it by using autonumber function.

               

              Regards,

              Vishal

              • Re: Data load
                Ahmar ansari

                1-Avoid if condition when load data from data source

                2-Don't put more than one where condition when load data directly from data source

                3-Put condition checking or If condition after Table Resident

                4-Avoid left join and Right join better to use Left keep and Right keep its faster as compare to Left and Right join

                5-If you have multiple condition checking at the time of data load so try to avoid if function,you can also use pick function with wildmatch/match doing same things with faster processing .

                6-Try to use mapping table and applymap in your script its faster than join.

                7-Comments and exclude field which are not in used.

                 

                I hope above point is help you.

                 

                Regards

                Ahmar

                • Re: Data load
                  Ivan Bozov

                  Also if you have historical data that is not changing you can save it as QVD file an load from that file instead of using Excel or DB as a source. QVD loads are 10-100 times faster.