Qlik Community

QlikView App Development

Discussion Board for collaboration related to QlikView App Development.

Announcements
Save the Date: QlikWorld Online, June 24-25, 2020. Free global virtual event for data integration and data analytic gurus. Register Today
Highlighted
Not applicable

How to make this load more quickly

Hi All,

I have one big qvd about 1GB size.

And I use load script to extract data from that qvd.

However,  when I use the where clause to filter the records to get what I want, the whole process takes some much time.

How can I speed up that process like adding the index in the SQL table?

Thanks.

15 Replies
Highlighted
Honored Contributor

Re: How to make this load more quickly

Hey Isaac,

Here's a blog post explaining what will and what will not break the optimized QVD load:

http://www.quickintelligence.co.uk/qlikview-optimised-qvd-loads/

Highlighted
Not applicable

Re: How to make this load more quickly

HI ,

       You can create the View in the DataBase with the filter what ever you have.

And then try to Export that view into Qvd .

So you can use that Exported Qvd wihtout any clause ( i.e By Optimised Load) .

Please let me know if there is anything .

Highlighted
Honored Contributor III

Re: How to make this load more quickly

try to resolve the Synthetic keys if they are more!!

Highlighted
Partner
Partner

Re: How to make this load more quickly

Hi,

You can try out the below ways

1) use only required fields

2) Add where clause on resident statement

    eg:  

Test:

Load a,bc, from qvd;

Load * resident Test where a=9;

I hope this helps you out.

Regards

Deepak


Highlighted
Contributor III

Re: How to make this load more quickly

Hi, 

If you don't perform any calculations and replace your WHERE clause with:

WHERE Exists(Field) 

your load will be optimized, 

br Paul

Highlighted
Partner
Partner

Re: How to make this load more quickly

Adding a WHERE clause breaks the optimized load.  If you only have a single field in the WHERE clause you can replace it with WHERE EXISTS - please post your WHERE clasue and I'll help further.

Jason

Highlighted
MVP
MVP

Re: How to make this load more quickly

First, please try not to start multiple threads with the same topic, it makes it hard to follow  a topic and clutters the forum.

Coming back to your question: I assume your qvd load changes from optimized to non-optimized, when you added the where clause. This can lead to orders of magnitude slower load times.

I believe the only condition you can use in a where clause to keep getting an optimized qvd load is a single exists() statement.

So maybe you can remodel your load to consider that, maybe like suggested here:

http://community.qlik.com/message/41463#41463

Regards,

Stefan

Highlighted
MVP
MVP

Re: How to make this load more quickly

Hi

Using a where clause when loading from a QVD will prevent an optimised load. Therefore it will take much longer to load.

Regards

Jonathan

Logic will get you from a to b. Imagination will take you everywhere. - A Einstein
Highlighted
Luminary
Luminary

Re: How to make this load more quickly

Hi,

You need to use optimized load which is much faster than normal QVD load. I have used optimized load with 7GB QVD files and it loads in few minutes. Please use Where Exists() instead of simple Where Clause. If you have more than one field in where clause then please build a concatenated key preferably a number and not hashed key or text based key. Then use the where clause with this key along with the Exists() function and this way you always perform optimized load.

I hope this helps!

Cheers,

DV

www.QlikShare.com