topic Re: incremental performance in QlikView

incremental performance

Nagaraju_KCS — Wed, 25 Nov 2020 16:16:04 GMT

Hi Community,

I have done incremental load for only for new records.

Full reload and Incremental load taking same time....

How to improve performance....???

SET vQvdFile='E:\Ledg_Entry.qvd';

SET vTableName='Ledger_Entry';

LET vQvdExists = if(FileSize('$(vQvdFile)') > 0, -1, 0);

IF $(vQvdExists) THEN

maxdateTab:

LOAD Date(max("Posting Date")-1,'YYYY-MM-DD') as maxdate

FROM $(vQvdFile) (qvd);

LET vIncrementalExpression = 'Where Convert("Posting Date",SQL_CHAR) >=' & Chr(39) & peek('maxdate') & Chr(39);

DROP table maxdateTab;

ELSE

LET vIncrementalExpression = '';

END IF

Cust_Ledger_Entry:

LOAD "Entry No",

"Customer No",

"Posting Date",

"Document Type";

SQL SELECT *

FROM "Ledger Entry"

$(vIncrementalExpression);

IF $(vQvdExists) THEN

CONCATENATE ($(vTableName))

LOAD * FROM $(vQvdFile) (qvd);

End IF

STORE $(vTableName) INTO $(vQvdFile);

drop table $(vTableName);

Both of taking same time (Full and Incremental Load).

Any other method to fetch new records..??

Thanks in Advance...

Re: incremental performance

Nagaraju_KCS — Tue, 31 May 2016 05:21:53 GMT

I don't have any Primary_Key.

I am using Views.

Re: incremental performance

sujeetsingh — Tue, 31 May 2016 05:39:29 GMT

Can you just come up with more words sir ?

Re: incremental performance

Nagaraju_KCS — Wed, 01 Jun 2016 00:42:25 GMT

Sorry for Delay..

I am fetching data from SQL Views, i don't have any primary key in that.

Tables have huge data, almost i have 20 views..

it was full loading time more than 1 hour.

So, I did Incremental load for fetching new data only.

It's working fine.. but it's taking same time.

Did i anything wrong??

How to reduce reload time....

Please any suggestions???

Re: incremental performance

marcus_sommer — Wed, 01 Jun 2016 07:24:42 GMT

I'm not sure if there are further issues but your pick from the max-date isn't correct then outside from a load you need to specify all parameters also instead of:

peek('maxdate') you need: peek('maxdate', 0, 'maxdateTab')

- Marcus