3 Replies Latest reply: Aug 17, 2017 6:45 PM by Rob Wunderlich RSS

    Using partial load instead of incremental load

    Aniruddhya Dutta

      Hi All,

       

      I have a table where I have designed incremental load. The table has a volume of around 10 million records.  I have an insert update logic running in my incremental load.So its picking up new/modified records and concatenating them into the history qvd every 5 mins. In this way my QVD file is growing in size every hour.

       

      Once incremental load finishes it tigers my dashboard. Now the problem is every time my dash board runs it has to read through all the days of data i.e the history data in the qvd along with the new ones that came in.Its taking a huge time which business can't afford.

       

      Can anyone suggest me some otimization solution?

       

      while researching I found we can do partial reload but some how I cannot releate to it since all the examples are talking  about reading a new table. But what if I can use the partial reload option in my application so that it don't have to read the records it has already stored gone through?

       

       

      I am using qliksense 3.2. (I think partial reload option is not native but that's a different question)