QVD file Performance

[#1]

Hello Team

   Good evening, i would like to know how to test the performance of QVD file while loading data at the dash board.

Say for example

If I am having 100 million records - how many hrs it takes

If i am having  1 GB of data how many hrs it take

if i am having 10 GB of data how many hrs it take.

Is there any object which help us to find or we need to do any thing manually(kindly share if  there any statement for it)

Thanks

Srini

[rbecher]

Hi Srini,

this is also  dependend from the data itself (cardinality). Why not create relevant test data in different size (1 GB, 10 GB etc.) and measure the loading time?

- Ralf

[#3]

I understood Ralf, but at the development it is not possible for us. when the dashboard moved to Production only we face this issue and more over  we need to provide the information to client about the statical information, I am just quizing like oracle /Sqlserve we are planner tool is available. just like that is there any way to find.

[sujeetsingh]

I too facing an issue that i am having a qvw of memory 37 mb but at a  straight table it comes out of memory .

[rajeshvaswani77]

Hi Srini,

You might have to look at the option of incremental load if it suits your context.

thanks,

Rajesh Vaswani

[Gysbert_Wassenaar]

Simply test. First load 100.000 records and measure the time:

LET vStart = num(now());

FIRST 100000

LOAD * from somewhere;

LET vRunTime = interval(now() - $(vStart),'mm:ss');

Then look at the content of the vRunTime variable to see how long it took. Next you can try a million rows or just extrapolate.

[#7]

But how you will collect the time

say for eg..

20 data 5 Sec

30 data  7 Sec etc

[#8]

Sure GW

   will check it out and come back to you

[#9]

yes even i too faced the same same out of memory at the  straight table but when i reload the qvw it is working fine. dont know why

[StefanBackstrand]

As G Wassenaar says; the only way is to measure the time to reload on a subset of the actual data on the same hardware and then draw conclusions by extrapolating.

Also make sure to frequently cross-match the script logs from the reloads, they will have timestamps on literally each script row executed. A skilled QlikView developer could easily make a qvw that consumes the script log and measure duration between each log row timestmap. That can then be compared to see trends, and even do forecasts.