topic Re: Increamental load Issue in QlikView

Increamental load Issue

aniketsr — Wed, 13 Nov 2013 10:35:46 GMT

Hi,

I have a table of 73L Record which takes around 1Hr to reload i.e data extraction from Database to QVD.

I Tried loading the data using increamental load wherein i used a date Column which is the transaction date.

But as we use the transaction date in the where clause the Reload becomes more slow and for 20 Mins there is not data fetched.

kindly suggest with any other steps or ways in which the above requirement can be met or else guide with any step if iam missing in the incremental load.

Thanks

Re: Increamental load Issue

marcus_malinow — Wed, 13 Nov 2013 10:44:55 GMT

Hi Aniket,

it sounds very much like you need to create an index in your database on your date column. That should speed things up fairly dramatically.

Marcus

Re: Re: Increamental load Issue

Anonymous — Wed, 13 Nov 2013 10:53:09 GMT

Aniket

When you say 73L is the L Lakh, and how many in millions is 73L ?

You also say ..we use the transaction date in the where clause the Reload. Is this where in the SQL database statement or in a QlikView Load Statement ? Pasting in your load script would help us understand.

Best Regards, Bill Markham

Re: Increamental load Issue

Bill_Britt — Wed, 13 Nov 2013 13:03:32 GMT

Hi Aniket,

This is a little old, but I think it will still apply.

Bill

Re: Increamental load Issue

Wed, 12 Mar 2014 05:41:43 GMT

Hi,

You can try this.

Hope it will help you.

Thanks.

Re: Increamental load Issue

sunilkumarqv — Wed, 12 Mar 2014 05:54:40 GMT

Hi Aniket,

Please follow steps like below and see attached .

we can do in various senarios , Typically below on of that .

//**************************Incremental Load**********************//

//Initial Reload & generate Qvd

ProdMaster:

LOAD ProductID,

CategoryName,

ProductName,

LastUpdateDate

FROM

Sample_Data.xls

(biff, embedded labels, table is [Sample_Data$]);

Store ProdMaster into [..\Data\Qvds\ProdMaster.Qvd];

DROP Table ProdMaster;

//exit script;

Max_LastUpdateDate:

LOAD

Max(ProductID) As Max_ProductID

FROM

[..\Data\Qvds\ProdMaster.Qvd]

(qvd);

//exit script;

Let vMaxProductID = peek('Max_ProductID',0,'Max_LastUpdateDate');

Drop Table Max_LastUpdateDate;

Complete_Data:

LOAD ProductID,

CategoryName,

ProductName,

LastUpdateDate

FROM

[..\Data\Qvds\ProdMaster.Qvd]

(qvd);

Concatenate

LOAD ProductID,

CategoryName,

ProductName,

LastUpdateDate

FROM

Sample_Data.xls

(biff, embedded labels, table is [Sample_Data$])

where ProductID>'$(vMaxProductID)';

Store Complete_Data into [..\Data\Qvds\ProdMaster.Qvd];