Reducing the load time of a big table

[#1]

Hi all,

I have a Straight Table containing 600.000+ lines. The effect is loading this takes a while, and the application is unresponsive in the meantime.

Is there some way to partially load this and tell Qlikview to load the entire table upon explicit request only.

For example, load the first 100 lines, then use a button to start loading the remaining 600k+. I am explicitly looking for a solution to this problem in the front end (is possible), not in the load script.

Thanks!

[#2]

It's not working out. Seems like it's calculating this for the 600.000 rows only then to find out which rows are to be shown and which are not. So it loads the entire table only then to reduce it. It's not loading the reduced rows only.

[Colin-Albert]

Are the loads for the QVD file optimised?

Can you narrow down the slow loading to any specific expressions in your chart?

Can you disable the expressions except one of them and enable then one by one to see if the slow rendering is related to a particular expression.

Are you expressions using set expressions rather than "if" statements?

Are your set expressions based on integer or text values?

Perhaps posting some examples of your expression may help identify the problem.

[MarcoWedel]

can you post an example to show what exactly does not work?

[karthikeyan1504]

Hi Henk,

If you are using this report in qlikview just to extract it to excel, then please try to build the expressions in script and store the table in any shared location in form of excel file instead of qvd.

[#6]

Thanks for your help. I have indeed tried other/simpler expressions, but even one expression with a simple Count() statement makes that the table load takes 2 minutes. Probably because of the high number of dimensions.

[#7]

Hi Marco, can't post examples I am afraid because I am working with confidential data. However anything I have tried doesn't solve the load time (see all other suggestions in this thread). Maybe because the dimensions are derived from several different QVDs.

[#8]

For testing purposes this would work, but not for the production environment I have to work in. It did speed it up a bit though. I think the high number of QVDs may be the main cause for my issue.

[Colin-Albert]

You can scramble the data to protect any confidentiality and can reduce the data to minimise the file size.

The number of QVDs the dimensions are generated from has no bearing on the refresh time of the charts as all the data is in RAM. Are you using calculated dimensions?

[#10 · Accepted solution]

Hi all,

Thanks a bunch for your help and suggestions.

Eventually how I solved this case is by adding duplicate expressions, making both conditional. The first expression contains a set analysis for a certain period only. With a button users can switch to the other expressions which loads and shows data for all periods.

Henk-Jan