topic Reg:How to perform Incremental Load in QlikView

Reg:How to perform Incremental Load

Mon, 22 Mar 2010 05:11:02 GMT

Hi All,

I want to do Incremental Load.Thing is I would like to know how to perform Incremental load?

So please share the qvw file or share some doc.Client using Sql Server as Data base

Reg:How to perform Incremental Load

Mon, 22 Mar 2010 06:09:14 GMT

Hi , you can do like this

Let ThisExecTime = Date(Today()-1,'YYYY-MM-DD 00:00:00');

ODBC CONNECT TO v;

Paye:

LOAD ID,

DATIME,

……….

………….

……….

………

…….

………

BCBRANCH;

SQL SELECT *

FROM "RLIFE_DW".dbo."TEAM_PAYE_NEW"

WHERE DATIME > '$(LastExecTime)'

AND DATIME <= '$(ThisExecTime)';

Concatenate LOAD * FROM D:\Qview\LifeAsia\QVD\Paye.qvd (qvd)

WHERE NOT EXISTS(ID);

If ScriptErrorCount = 0 then

STORE Paye INTO Paye.QVD;

Let LastExecTime = ThisExecTime;

End If

Where

ID is a primary key

DATIME is a modified date time

above example is of append and update type of increamental load

Rahul

Reg:How to perform Incremental Load

Anonymous — Thu, 27 May 2010 19:56:19 GMT

When do you establish LastExecTime to begin with?

Reg:How to perform Incremental Load

Fri, 28 May 2010 06:25:38 GMT

Hi Rahul,

Thanks for your kind reply.Since I have not used Incremental Load,I am having small query on this.Here you used variable

'$(LastExecTime)' where we have to define and what is the input for that.Please let me know the same.

Thanks & Regards

Jai

Reg:How to perform Incremental Load

Fri, 28 May 2010 07:12:14 GMT

Hi Jai,

Last Execution time should be RloadTime().

-Peterson

Reg:How to perform Incremental Load

Sun, 03 Apr 2011 22:01:33 GMT

Hello Rahul,

I am also trying to the same thing as you mentioned here. Did you find a solution for the problem? Please let me know if you did. I also have something like a TimeID instead of actual Date and Time... so I am not sure how to plan the incremental load as without that I will have to reload all the QVDs everyday which will take more than 2 hrs...

Please help.

Thanks,

ANDY

Reg:How to perform Incremental Load

Mon, 04 Apr 2011 05:12:14 GMT

The Qlikview Reference Manual has a whole section on this subject. For version 9 it is Book 1, Part IV, pages 505-508. It outlines how to handle combos of insert, update, and delete in the data set. Perhaps this goes without saying (for everyone but me), but in addition to limiting the data, one also has to combine the limited data set with the historical data set via a concatenate statement. Again, the manual covers the different scenarios one could encounter.

Reg:How to perform Incremental Load

Mon, 04 Apr 2011 05:39:32 GMT

Hi All,

I am uploading a sample file for incremental load for database Oracle.

Please see and tell if its helpful.

Raman

Reg:How to perform Incremental Load

Mon, 04 Apr 2011 09:24:33 GMT

Hello Ramandeep,

Thank you for the file, and I have seen that before. My question is that I have something called a Timeperiod ID and nothing in the Date table, so was wondering how can I do a reload from the DB to QVD with that? I think we need to have something like the last updated time Field in the DB to do an incremental load isnt it? I have sales, orders tables which my client needs to have 3 years data and everytime I reload the QVD its doing a full reload which is taking about 2-3 hours and then an incremental load of QVW which takes about 15 minutes... Is there a solution to avoid the complete reload of the QVDs with just the Time period ID (Primary Key)?

Thanks,

ANDY

Reg:How to perform Incremental Load

Mon, 04 Apr 2011 10:04:08 GMT

Hi Andy,

If you have salesorder Tables then you might be having a CreationDate in that table I guess it is a must.

Before using the script above you have to do once a full upload. Save as a Table.qvd. With the Script above you can implement the Incremental load and load only new data to the already present data in Table.Qvd . This is done by the concatenate Load and the Dates.

Regards

Sravan