topic Re. :Re: Re. :'Where' Performance reading from QVD; in QlikView

'Where' Performance reading from QVD;

Wed, 06 Apr 2011 15:12:17 GMT

Hello All!

Need some advice on the best way to approach this,

I'm trying to load data from a (Optimised) QVD,

I have around 15-16 individual load statements with separate 'Where' clauses, all concatenated together.

As the load times are significantly slower than i expected - please can anyone offer any advice on how to improve performance.

Example

Controls:

LOAD

'Opp' as [Type],

id,

Date

FROM Ticket.QVD (Qvd)

WHERE

[Task 1 Due] <= $(vToday);

Concatenate

LOAD

'Achieved' as [Type],

id,

Date

FROM Ticket.QVD (Qvd)

WHERE

[Task 1 Due] <= $(vToday) AND

[Task 1 Due] >= [Task 1 Done];

....................

Re. :'Where' Performance reading from QVD;

vincent_ardiet — Wed, 06 Apr 2011 15:27:48 GMT

Hi,

Have you try to use a resident table ?

First load Ticket.QVD with no filter then perform several resident load.

Regards,

Vincent

Re. :'Where' Performance reading from QVD;

Wed, 06 Apr 2011 15:55:33 GMT

Hey,

Yes, i have tried but as we have a very large data set, holding that much data in memory (resident) also was having difficulties.

Thanks

Re. :Re: Re. :'Where' Performance reading from QVD;

vincent_ardiet — Wed, 06 Apr 2011 17:08:32 GMT

Ouch !

And if you try to mutualize some loading, for example something like :

Controls:

LOAD

'Opp' as [Type],

id,

Date,

[Task 1 Due],

[Task 1 Done]

FROM Ticket.QVD (Qvd)

WHERE

[Task 1 Due] <= $(vToday);

Concatenate

LOAD

'Achieved' as [Type],

id,

Date,

[Task 1 Due],

[Task 1 Done]

RESIDENT Controls

WHERE

[Task 1 Due] <= $(vToday) AND

[Task 1 Due] >= [Task 1 Done];

Then you can drop the unused columns.

Regards,

Vincent