topic Re: Incremental load problem in QlikView

Incremental load problem

sasikanth — Fri, 29 Nov 2013 06:44:58 GMT

Hi all,

Can some one please explain me about optimization of incremental load.....

what i am doing is .. creating a QVD and fetching again data from that QVD into temp table for sorting of timestamp column .

if that qvd is 1 GB then if you run the script then ... two times u need to load these 1 GB of records ..Thats very bad ...

How can i avoid this ...

Thanks in advance

Re: Incremental load problem

saumyashah90 — Fri, 29 Nov 2013 07:18:27 GMT

you can use

Table1:

Load * abc

from qvd;

concatenate

Load * xyz

from Excel/qvd Where not exists(Timestamp);

Store table Table1 into qvd(QVD);

This will load only those from qvd whose timestamps are different from the other qvd.

Re: Incremental load problem

Fri, 29 Nov 2013 07:28:08 GMT

ome examples of things that will cause a non-optimised load are:

- Adding new fields to the table
- Deriving new values from a field in the QVD
- Retrieving a field twice
- Most WHERE conditions
- Joining to an existing in memory table
- Loading data into a mapping table

In contrast the things you are allowed to do are:

- Rename fields
- Omit fields
- Do a simple one field WHERE EXISTS on a field returned in the record set

once qvd optimized then go for resident and do any operation that will improve your utilization and performance.

thanks and regards

shiv

Re: Incremental load problem

sasikanth — Fri, 29 Nov 2013 07:39:46 GMT

Again you are fetching data from Qvd ...my question is again your are fetching data from 1GB of records then it may decrease performence .

step1:

Tab:

load * from

Store Tab into .....BC.qvd(qvd);

Step2:

Tab1:

load * from BC.qvd(qvd)

i want to avoid step2 ...how can i do this ... is this posible or not ...

Can we use peek() function at the time of data loading or not ..

Thank you soumya

Re: Incremental load problem

sasikanth — Fri, 29 Nov 2013 07:40:48 GMT

Thats fine siva but i am asking about incremental load optimization ...

Re: Incremental load problem

Fri, 29 Nov 2013 08:12:17 GMT

Hi,

You can refer to this:

http://community.qlik.com/message/318991

Might be of help!