topic Re: Incremental load in QlikView

Incremental load

Mon, 12 May 2014 10:16:15 GMT

Hi All,

I have 5 fields A, B, C, D, E in that fields time stamp will not be available. First time i have 100 records for this i used full load but next time 50 records are added. Now i have to do incremental load without time field.

Could you please anyone give me some suggestion on this with detailed descrption.

Thanks,

Vamsi

Re: Incremental load

ashfaq_haseeb — Mon, 12 May 2014 10:18:25 GMT

If you have primary key use that instead.

Regards

ASHFAQ

Re: Incremental load

rohan_mulay — Mon, 12 May 2014 10:34:06 GMT

Hello Reddy,

If you have an ID field u can take max(ID). Also, u can try using rowno() to get the previously loaded last record.

refer following code:

Master:

ID,

Name,

Description,

RowVersion

from Master.qvd;

Max:

load max(ID) as maxID

resident Master;

Let vMax = peek('maxID',0,'Max')

Drop table Max;

Transaction:

join(Master)

Load

ID,

Name,

Description,

RowVersion

From Transaction.qvd

where ID >'$(vMax)';

Re: Incremental load

raghvendrasingh — Mon, 12 May 2014 11:08:20 GMT

Hi VV Reddy,

Please Find below qvw for Incremental load.hope this will help u.

Regards,

Raghvendra singh

Re: Incremental load

Anonymous — Mon, 12 May 2014 11:14:33 GMT

Hi v v Reddy,

you can apply incremental load by using the id or primary key of the table , suppose we conside the id then

firstly you load the table by using order by id and make qvd then by using the peek ('id',-1,'tableName') you can get the last value of the table,

let vLastvalue = peek ('id',-1,'tableName')

set vUpdated_value = $(vLastvalue )

load the table where vUpdated_value >= id

then concatenate this data with the old table

and then store it into the same qvd .

Hope this would be help you.

Re: Incremental load

Thu, 15 May 2014 07:22:23 GMT

Hi all,

Thanks for suggestions. Present i'm using Primary key and working fine.

Thanks,

Re: Incremental load

stevedark — Thu, 15 May 2014 07:40:48 GMT

Just to comment on the correct answer here, finding the Max ID from a QVD actually takes quite a chunk of time (as a full non optimised load is required).

If you are loading from a SQL based data source I would suggest using the database to get the max value and then persist that to a separate QVD for quick access next time.

Also, if all the columns in two loads are identical, and you want to add one set of values to another you will want to be using a CONCATENATE statement rather than a JOIN. Again, this is much more performant.

I've written a post on incremental loading that covers a number of techniques you may find useful:

http://www.quickintelligence.co.uk/qlikview-incremental-load/

Steve