Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
samuel_brierley
Creator
Creator

Does saving to a qvd make a difference

Hi im loading in over 10 mil lines from various sheets and performance is explanably poor. will storing the load into a qvd then loading that file improve performance?

9 Replies
giakoum
Partner - Master II
Partner - Master II

Hi. QV will definitely load quicker a qvd file than any other source. However, to make it a qvd file, you need to load it from the database anyway, so the overall time might be the same. But you could use incremental loads and save a lot of time, especially if you have a unique key per record.

Carlos_Reyes
Partner - Specialist
Partner - Specialist

I think as , doing an incremental load its your best choice.

Ive seen that when you're transforming data, specifically with several millions of records, it is a good idea to save the intermediate tables into qvd's to free up space so that the others transformation processes have more space to work... and in my experience that does improves performance.

So.. I would say.. saving to qvd does makes a difference in certain tasks, but in your case it's a better choice to create an incremental load process.

Not applicable

Hi Samuel

The correct answer to your question is "Yes". The QVD files are meant to be a faster way to read data, but you have to keep in mind when you are loading the QVD file into you RAM memory you cannot add columns or make any calcultions because it will drop the benefit of loading faster, the only thing you are able to do without losing performance is to rename a column,

Wrong Examples:

Load

     *,

     1 as Column 2

From [youQVDfile.qvd](qvd);

//

Load

     Date(dateColumn, 'YYYYMMDD')  as DateColumn,

     sum(valueColumn) as ValueColumn

From [youQVDfile.qvd](qvd)

Group By

     dateColumn;

Correct Examples:

Load

     *

From [youQVDfile.qvd](qvd);

//

Load

     dateColumn  as DateColumn,

     valueColumn as ValueColumn

From [youQVDfile.qvd](qvd);

Not applicable

Hi,

Yes, it should be really faster to load from qvd than from the various sources. Especially since your qvd can be stored in a local file, while some of your sources are probably on network. Also, compared to csv for example, Qlikview doesn't have to read it all to reconstitute the cells.

Last, qvd takes usually less place on disk than other storages. Identical values are "grouped", let's say you have 1 million lines with an identical long string value, it will take 1 million times this size in csv, but not in qvd.

rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

"will storing the load into a qvd then loading that file improve performance?"

Are you asking about the performance of chart objects? -- Or duration of script execution.

-Rob

samuel_brierley
Creator
Creator
Author

Hi apologies for being away from the thread, im looking at improving the performance of the charts. theyre very slow.

tabletuner
Creator III
Creator III

then it wont make a difference.

Anonymous
Not applicable

Using a QVD to store data only improves the loading time of data and has nothing to do with the Chart performance.

If you want to improve the chart performance it is better to remove the complex calculations in the chart and do them in the script itself.

Hope that helps.

Peter_Cammaert
Partner - Champion III
Partner - Champion III

Or add more memory to your Server / Desktop.

Peter