Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
sasikanth
Master
Master

Incremental load problem

Hi all,

Can some one please explain me about optimization of incremental load.....

what i am doing is .. creating a QVD and fetching again data from that QVD into temp table for sorting of timestamp column .

if that qvd is 1 GB then if you run the script then ... two times u need to load these 1 GB of records ..Thats very bad ...

How can i avoid this ...

Thanks in advance

5 Replies
saumyashah90
Specialist
Specialist


you can use

Table1:

Load * abc

from qvd;

concatenate

Load * xyz

from Excel/qvd Where not exists(Timestamp);

Store table Table1 into qvd(QVD);

This will load only those from qvd whose timestamps are different from the other qvd.

Not applicable

ome examples of things that will cause a non-optimised load are:

- Adding new fields to the table
- Deriving new values from a field in the QVD
- Retrieving a field twice
- Most WHERE conditions
- Joining to an existing in memory table
- Loading data into a mapping table

In contrast the things you are allowed to do are:

- Rename fields
- Omit fields
- Do a simple one field WHERE EXISTS on a field returned in the record set

once qvd optimized then go for resident and do any operation that will improve your utilization and performance.

thanks and regards

shiv

sasikanth
Master
Master
Author

Again you are fetching data from Qvd ...my question is again your are fetching data from 1GB of records then it may decrease performence .

step1:

Tab:

load * from

..

..

Store Tab into .....BC.qvd(qvd);

Step2:

Tab1:

load * from BC.qvd(qvd)

i want to avoid step2 ...how can i do this ... is this posible or not ...

Can we use peek() function at the time of data loading or not ..

Thank you soumya

sasikanth
Master
Master
Author

Thats fine siva but i am asking about incremental load optimization ...

Not applicable

Hi,

You can refer to this:

http://community.qlik.com/message/318991

Might be of help!