SQL table taking too long to store into QVD

[#1]

Hi,

I have two huge sql tables with Table1 has 4213039 rows

and Table2 has 12710440 rows

when I am loading the tables into QVDs by using

Store Table1 into C:\qlikviewpath\Table1.qvd;

Store Table2 into C:\qlikviewpath\Table2.qvd;

to load the scrit execution is taking more than 14 minutes. could anyone let me know is there anyway to load data into QVD very fast.

Thanks.

[rajeshvaswani77]

Hi Amelia,

Please see below link for Optimized load.

http://www.quickintelligence.co.uk/qlikview-optimised-qvd-loads/

Please see attachment for Incremental Load.

Thanks,

Rajesh Vaswani

[christian77]

Hi Amelia:

Time its not important, you can do tha't at night. What is important is the limit of memory.

You could 'slice' your data in years or months. Doing small packs could be better.

You may don't have to load last years or last months everytime you reload If records were not modify.

salutes.

[Clever_Anjos]

I would rewrite your SQL with a "WHERE" clause retrieving only records from current month

SELECT *

FROM "XX".schema."TRANSACTIONS"

where TransDate >= '$(vStartOfMonth);

and keep splitted qvd´s for each month

Transactions201301.qvd

Transactions201302.qvd

Transactions201303.qvd

...

[#5]

Hi,

Thanks and I need to take years from 2005 to 2013 . Do I need to store qvd's for all

Transactions201301.qvd

Transactions201302.qvd

Transactions201303.qvd

---

--

--

--

Transactions201309.qvd

Please let me know.

[#6]

Hi Amelia,

When you perform an incremental load on your script before it creates the qvd,your timewill be reduced considerably.So you wont really have to bother about memory management in that case.Im assuming you would perform a load of this data in the night . Try and perform an incremental load on a date field in your script.

If you don't want to load last years'/months' data,what you can do is,initially perform a full reload of your data from the master and then write a script for incremental load on your table pertaining to a date field(preferably modified date).

In that case,QV will first load all the data,and then append data thata that has been changed or insert records if they are new.You can schedule this activity in the night.It will not hamper your memory or other processes.

Hope this helps.

Rochelle

[#7]

Thanks and  could you let me know how to write the script to get this.

[Clever_Anjos]

Amelia,

Yes you must have qvd´s for each month.

Please note, that for example, 2012 and 2011 you only run once. You don´t need to reload old data everytime

[#9]

Hi Amelia,

Did you solve this problem? I'm having the same probem.