5 Replies Latest reply: Mar 6, 2012 12:01 PM by Badr Harfoush RSS

    Incremental Load Clarification

      Hi guys,

       

      I have looked through a lot of topics on incremental load and also read some QlikView documentation on this, but I need more clarity.

      To put my questions in context, I do not connect directly to a database, but I spit out csv files and load them into my application.

       

      My issue lies with the fact that I would like to clearly understand the following:

       

      1. Is incremental load only useful for adding new data, say for example for the most recent month to historic data so that you have just one compact qvd for everything?

      2. Hypothetically speaking, if I have 12 months of info loaded into my QlikView app which takes 2 hours for a full reload, with the use of incremental load, is it possible to only add information for a current month which will make me have 13 months of info directly to my qvw file, so that I don't need to do a full reloaod of the 13 months?

       

      If point 2 is actually possible, can someone please just use a simple inline load to demonstrate this concept as I have already looked a lot of posts on incremental load and some applications using incremental load and I am still not sure how to implement this for point 2 above.

       

      Any help would be greatly appreciated.

       

      Thanks.

        • Incremental Load Clarification
          José Andrés López

          Hi,

           

          In Mr. Rob Wunderlich's blog you will find a fantastic example about incremental load app.

           

          Download the Qlikview Cookbook at http://robwunderlich.com/downloads/

           

          I hope it helps.

          Kr.

          JaL

          • Incremental Load Clarification
            Johannes Sunden

            Hi zebadguy,

             

            You could potentially use Partial Reload for question 2.

            If you run an if statement and use the IsPartialReload() function to check if you're doing a partial reload, you can then do an ADD LOAD to just add new data to the existing table, and then maybe run a full reload on a weekly basis or something.

             

            However, if it takes 2 hours to run the reload, I would suggest using incremental reload, store the tables into QVD files, and then use these for your user documents, and they should load quickly. The actual QVD generator document you can schedule to run nightly for example.

            • Re: Incremental Load Clarification

              Hi!

              Plz find the attachment .

              This is a very basic method to append the new data into the Qvd storing old/historic data.

              • Incremental Load Clarification

                Thank you all for your answers. I have found them useful, but my main goal is to achieve this.

                Scenario:

                I have 12 months of info loaded into my QlikView app which takes 6 hours for a full reload and this will continue to increase over time. With the use of incremental load, I want to be able to only add information for every current month to the application directly. I am fine with concatenating and getting a qvd that will have both the recent and historic data. That is not my problem. My problem is being able to load only one month of info for the current month directly to my qvw file, so that I don't need to do a full reload always?

                 

                I hope I have provided enough information.

                 

                Any help would be greatly appreciated.

                 

                Thanks