Skip to main content
Woohoo! Qlik Community has won “Best in Class Community” in the 2024 Khoros Kudos awards!
Announcements
Nov. 20th, Qlik Insider - Lakehouses: Driving the Future of Data & AI - PICK A SESSION
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Incremental load

I read that i can use the exist () phrase for a qvd load and still the qlikview will load the qvd as "optimized Load".


So if i'll replace the "where DateField > $(VDate)" part in an incremental load statment with "where exist (AlreadyLoadDateField)" i will get better performance ?


did someone use it or try it out ?

thanks for your help.

7 Replies
Not applicable
Author

Can someone help me with this ?

gandalfgray
Specialist II
Specialist II

Yes I think you would.

But You should rather use Not.

...

Where Not Exists(DateField);

/gg

Not applicable
Author

i dont think you will get better performance! because if you load from a source (except a qvd-file) with the "not exists"-statement, the script has to load all the data from the source and then filter the already existing datasets out of it. if you load with an "where DateField > $(VDate)"  statement, then the source maybe can filter the datasets for itself and only submit the correct Data to your script. Therfore not so much data has to be streamed.

MFG

gandalfgray
Specialist II
Specialist II

... well I thought "a qvd load" meant that Matan was loading from a qvd...

If it's a load from a relational database, the performance will depend on the indexes in the database etc.

/g

Not applicable
Author

Thanks for your comments.

I probably didnt make my self clear.

 

I know that the part in the script that load the new data from relational database will not be affected by using exist () insted of "where DateField > $(VDate)" .

in the second part - the loading of the saved data (stored in qvd) i want to load the old data without the last saved day.

for that part i considered of using the exist () phrase.

so...will it work better ?

gandalfgray
Specialist II
Specialist II

If you have first loaded data (by sql or something) into a field DateField

you could then load from a qvd using

Where Not Exists(DateField)

to avoid loading records with DateField-values that is already loaded into your qvw.

I believe it should be faster than

Where DateField>$(VDate)

hth/gg

Not applicable
Author

Thank a lot