5 Replies Latest reply: Oct 30, 2012 5:09 AM by Vegar Lie Arntsen RSS

    Increamental load

      Hi

       

      I have just started reading about Increamental reload, and I understand some parts about, but I am not really sure how it works, can some one help me with a simple example of increamental load on the attached file?

       

      Many thanks in advance

       

      Kind regards

      Edzi

        • Re: Increamental load
          Vegar Lie Arntsen

          I added a sheet to your datasource.

          1 I load the id of new data

          2 I load the old data except the rows that are to be updated

          3 I add the new rows to your data.

            • Re: Increamental load

              Hi

               

              I am loading data from the SQL database, and I only want to load new data that has been added to the table, the issue is that, i am loading millions of rows to the model and i want to avoid loading all the data when doing a reload but only loading the new rows

               

              Thanks

              Edzi

                • Re: Increamental load
                  Vegar Lie Arntsen

                  Then you don't need the first where NOT EXISTS() check.

                   

                  I believe you have to load all data into a reload, but your old transaction-file.qvd will be faster than a SQL-request?

                  However you could do a partial reload if you are working on the QlikView-desktop.

                   

                  Table:

                  //Load your old data except the rows you want to do incrementel update on

                  LOAD Id,

                       Name

                  FROM

                  Test.xlsx

                  (ooxml, embedded labels, table is Sheet1);

                   

                  //Load your new rows when using partial reload to the Table

                  add only LOAD

                  Id,

                  Name

                  FROM

                  Test.xlsx

                  (ooxml, embedded labels, table is Sheet2);

                   

                   

                  Running script using ctrl-shift-R will only add your Sheet2-data to the qvw-table "Table".

                    • Re: Increamental load

                      Hi

                       

                      On my case the ADD ONLY LOAD will not work since the data i want is only the new added data on sheet1, I want to add the Data that was inserted on sheet1 after I did the first reload. what I see now is that if I do the reload on the same sheet, all the data is add again, plus the new rows.

                       

                      So is there a way to load only the new data but on the same sheet?

                       

                      Thanks

                      Edzi

                        • Re: Increamental load
                          Vegar Lie Arntsen

                          You need to know which data is new data. I am assuming the Id is unique for each row.

                          If I understand correctly.

                          Day one: You load sheet1 containing id 1,2 and 3

                          Day two, three etc.: The sheet1 contains id 1,2,3 and 4, but you only want to load 4 into the app.

                           

                          Day one, the initial load of data.

                          Trans:

                          LOAD Id,

                               Name

                          FROM

                               Test.xlsx

                               (ooxml, embedded labels, table is Sheet1)

                          ;

                          store Trans into Trans.qvd;

                           

                          Day two, three etc., only new rows will be added to the master Transaction file.

                          Trans:

                          LOAD

                               Id,

                               Name

                          From

                               Trans.qvd (qvd);

                          concatenate LOAD

                               Id,

                               Name

                          FROM

                               Test.xlsx

                               (ooxml, embedded labels, table is Sheet1)

                          WHERE

                               NOT Exists(Id)

                          ;