Qlik Community

QlikView Scripting

Discussion Board for collaboration on QlikView Scripting.

sasikanth
Valued Contributor III

Incremental load problem

Hi all,

Can some one please explain me about optimization of incremental load.....

what i am doing is .. creating a QVD and fetching again data from that QVD into temp table for sorting of timestamp column .

if that qvd is 1 GB then if you run the script then ... two times u need to load these 1 GB of records ..Thats very bad ...

How can i avoid this ...

Thanks in advance

5 Replies
saumyashah90
Valued Contributor

Re: Incremental load problem


you can use

Table1:

Load * abc

from qvd;

concatenate

Load * xyz

from Excel/qvd Where not exists(Timestamp);

Store table Table1 into qvd(QVD);

This will load only those from qvd whose timestamps are different from the other qvd.

Not applicable

Re: Incremental load problem

ome examples of things that will cause a non-optimised load are:

- Adding new fields to the table
- Deriving new values from a field in the QVD
- Retrieving a field twice
- Most WHERE conditions
- Joining to an existing in memory table
- Loading data into a mapping table

In contrast the things you are allowed to do are:

- Rename fields
- Omit fields
- Do a simple one field WHERE EXISTS on a field returned in the record set

once qvd optimized then go for resident and do any operation that will improve your utilization and performance.

thanks and regards

shiv

sasikanth
Valued Contributor III

Re: Incremental load problem

Again you are fetching data from Qvd ...my question is again your are fetching data from 1GB of records then it may decrease performence .

step1:

Tab:

load * from

..

..

Store Tab into .....BC.qvd(qvd);

Step2:

Tab1:

load * from BC.qvd(qvd)

i want to avoid step2 ...how can i do this ... is this posible or not ...

Can we use peek() function at the time of data loading or not ..

Thank you soumya

sasikanth
Valued Contributor III

Re: Incremental load problem

Thats fine siva but i am asking about incremental load optimization ...

Not applicable

Re: Incremental load problem

Hi,

You can refer to this:

http://community.qlik.com/message/318991

Might be of help!

Community Browser