topic Re: load issue in QlikView

load issue

arulsettu — Wed, 16 Jul 2014 06:44:49 GMT

i am having 20 gb of data. when i load only three fields from a table it taking 1+ hour to load. is there any way to make it fast?

what is the issue?

Re: load issue

alexandros17 — Wed, 16 Jul 2014 06:50:05 GMT

Too many data, probably you can split in historical data and actual data, in this scenario store the historical data in a qvd (it is faster to reload and concatenate the actual data loaded normally.

Let me know

Re: load issue

rbecher — Wed, 16 Jul 2014 06:51:29 GMT

Hi Arul,

what is the source? A csv file or a database?

- Ralf

Re: load issue

arulsettu — Wed, 16 Jul 2014 06:52:25 GMT

sql

Re: load issue

arulsettu — Wed, 16 Jul 2014 06:53:09 GMT

storing into optimized qvd?

Re: load issue

arulsettu — Wed, 16 Jul 2014 06:56:02 GMT

can you give a example?

Re: load issue

alexandros17 — Wed, 16 Jul 2014 07:07:46 GMT

The following script must be executed once.

History1:

Select * from mytable where year < 2014;

Store * from History1 into TAB_HIS1.qvd;

This is the script you must execute in your document:

FinalTable:

noconcatenate

Load * from TAB_HIS1.qvd;

concatenate

Load * from mytable where year >= 2014;

Re: load issue

arulsettu — Wed, 16 Jul 2014 07:19:39 GMT

i trying and let u know the result

Re: load issue

Wed, 16 Jul 2014 08:00:03 GMT

Hi,

My advice is to use QlikView Deploy Framework to create new qvd-extractor. You have to create QVD for all tables, you will spend a lot of time to do that, but you will do that once time. There are a lot of information about this solution.

Next when you create your first qvd files next qvd files will be created with "Initial Load" or "Delta Load". This will save a lot of time of you, and if you got optimized qvd file to concatenate the initial load , the time will be shorter.

There are a lot of ways for "Initial Load". There are also ready solution for this (for example Qlik Components) , so you have big choice and you have to choose most suitable in your situation.

Regards,

Venelin

Re: load issue

ashfaq_haseeb — Wed, 16 Jul 2014 08:08:11 GMT

Hi,

You can apply index at DB level to improve query response time.

Regards

ASHFAQ

Re: load issue

arulsettu — Wed, 16 Jul 2014 08:30:02 GMT

its taking 1+ hour still loading data. is there any other way to load quickly. its loading based on my ram size?

Re: load issue

ashfaq_haseeb — Wed, 16 Jul 2014 08:35:06 GMT

Hi,

You can look at performance by adding Index.

Regards

ASHFAQ

Re: load issue

rbecher — Wed, 16 Jul 2014 08:35:31 GMT

I guess you have to speed up your database query (ask your db admin) and your network connection..

- Ralf

Re: load issue

alexandros17 — Wed, 16 Jul 2014 08:40:37 GMT

Your Ram have great influence in your loading process.

Re: load issue

arulsettu — Wed, 16 Jul 2014 08:42:25 GMT

using qlikview desktop with 2gb of ram

Re: load issue

rbecher — Wed, 16 Jul 2014 08:42:39 GMT

No, not really. Either you have enough memory or not..

Re: load issue

arulsettu — Wed, 16 Jul 2014 08:44:41 GMT

so ram is the issue. any other way?

Re: load issue

jagan — Wed, 16 Jul 2014 08:46:15 GMT

Hi,

Are you using any conditions in Where? If yes, check whether there are indexes for those fields?

Then try to limit the data, restrict some unwanted data, suppose if users are using only last years data, then load only last 3 years data instead of loading entire data.

If the issue still not fixed try to implement incremental loading refer Qlikview help file (F1) for this topic "QVD files and Incremental Load" and try to implement.

Hope below link helps you.

Incremental Load

Regards,

Jagan.

Re: load issue

arulsettu — Wed, 16 Jul 2014 08:53:45 GMT

i just try to load three fields and stored into qvd. rows are 1534582.

only single table with three records.

Re: load issue

jagan — Wed, 16 Jul 2014 09:00:42 GMT

Hi Arul,

If you try the same query in SQL Server or Oracle, how much time it is taking to get the results?

Regards,

Jagan.