topic Re: Incremental Load in QlikView

Incremental Load

Anonymous — Wed, 25 Nov 2020 16:16:04 GMT

I have following sales data going back to year 2014. The actual model i have takes an hour to reload as it reloads all the data. I want to reduce the reload time by applying incremental reload. How do i accomplish this. Basically, the incremental reload should be the last sales date.

Attached herewith is the data.

Kind regards

Nayan

Re: Incremental Load

prma7799 — Tue, 26 Sep 2017 07:07:39 GMT

Please check this

https://www.analyticsvidhya.com/blog/2014/09/qlikview-incremental-load/

QlikView Incremental Load

Incremental load in Qlikview

Re: Incremental Load

prma7799 — Tue, 26 Sep 2017 07:09:58 GMT

Also go through this

Re: Incremental Load

Anonymous — Tue, 26 Sep 2017 07:18:21 GMT

Thank you. Will let you know if I'm successful.

Kind regards

Nayan

Re: Incremental Load

prma7799 — Tue, 26 Sep 2017 07:21:22 GMT

Ok ... Good.

Re: Incremental Load

its_anandrjs — Tue, 26 Sep 2017 07:26:21 GMT

Check the attached file

First initially load all data and then incremental load data with max date by finding max date by variables and then comment initial load table.

MaxDateTab:

LOAD Date(Max(Date)) as MaxDate FROM SalesData.qvd(qvd);

LET vMaxDate = Peek('MaxDate',0,'MaxDateTab');

//Incremental Load

Data:

LOAD Date,

[Sales Amount]

FROM

[Sales Data.xlsx]

(ooxml, embedded labels, table is Sheet1)

Where Date > '$(vMaxDate)';

Concatenate(Data)

LOAD * FROM SalesData.qvd(qvd);

Store Data into SalesData.qvd;

DROP Table Data;

Re: Incremental Load

Anonymous — Tue, 26 Sep 2017 07:35:14 GMT

Thank you Anand for your reply, i will let you know.

By the way, on the Incremental Load Script, will it be better to use an " Where Exist" expression as it will only look at the max date reload data and as compared to just a "Where" Expression, where it will load all data and then reduce to a max date. Am I correct?

Kind regards

Nayan

Re: Incremental Load

its_anandrjs — Tue, 26 Sep 2017 08:50:45 GMT

Exactly but in your requirement you are want to load the data with Maxdate which is Max date for one day, Right. so in this you have to find only Max Date and load your latest available dates from the table and concatenate with the previous (History Qvd).

Re: Incremental Load

Anonymous — Tue, 26 Sep 2017 09:10:52 GMT

Thank you for the feedback. Will the reload time be shorter?

Kind regards

Nayan

Re: Incremental Load

Anonymous — Tue, 26 Sep 2017 09:31:11 GMT

Hi Anand

The reload time is shorter.

Kind regards

Nayan

Re: Incremental Load

its_anandrjs — Tue, 26 Sep 2017 17:05:23 GMT

When you load data by this statement Where Date > '$(vMaxDate)'; this will read only max records based on the available data if it is few rows and abusively there are few rows then it will take less time to load.