Qlik Community

Ask a Question

QlikView App Dev

Discussion Board for collaboration related to QlikView App Development.

Announcements
Become an analytics expert with Qlik's new 15 week course: Applied Data Analytics using Qlik Sense. READ MORE
cancel
Showing results for 
Search instead for 
Did you mean: 
c_pannese
Partner
Partner

What is the best way to create qvd

Hello everybody.

I have this issue.

I need to load 3 years in my app and my sources are csv files, one for every month in particular.

Every csv has about 3 or 4 millions of rows.

I have the possibility to make sum for all the measures(about 60 different)  grouping by more than 30 dimension.

I  tried to created one qvds for each month and from here I created another qvd made by the sum of all measures and all dimension.

Finally I load all qvds in my app, but the result is vary bad.

I have this questions:

-is it correct make the sum/aggregations in the script when there are many measure and many dimension or is better create a simple qvd without aggregation only ?

-What is the best way to work in my case?

Thank you all.

8 Replies
kaushiknsolanki
Luminary Alumni
Luminary Alumni

Hi,

It depends on what kind of calculation is required.

If the KPI has simple sum functions then you should do it in frontend else better to move it to script.

When you say result is bad, what exactly is happening.

Regards,

Kaushik Solanki

Regards,
Kaushik
If reply is satisfactory, please "Like" the post.
If reply is solution, please mark as "Correct".
ali_hijazi
Partner
Partner

it's better to have the sum in the front end

how are you loading the QVDs ?

you can read csvs as follows:

my_table:

Load column1, column2, ...

from ...file1.csv;

concatenate(my_table)

Load column1, column2, ...

from ...file2.csv;

concatenate(my_table)

Load column1, column2, ...

from ...file3.csv;

store my_table into file.qvd

finally in the front end you can read the qvd and add an expression there

I can walk on water when it freezes
c_pannese
Partner
Partner
Author

Loading.png

The calculation  is not simple.

I stay for minutes in this situation. The sistem isn't reactive.

kaushiknsolanki
Luminary Alumni
Luminary Alumni

This can be the calculation issue and also the problem of low resource availability.

Make sure you have enough CPU and RAM for this calculations.

Regards,

Kaushik Solanki

Regards,
Kaushik
If reply is satisfactory, please "Like" the post.
If reply is solution, please mark as "Correct".
c_pannese
Partner
Partner
Author

So i have to create only one qvd or one for year??

And for future months??

c_pannese
Partner
Partner
Author

ok, i had already tought about it.

Can I do anything to improve the performance?

kaushiknsolanki
Luminary Alumni
Luminary Alumni

The best way to improve the performance is have the summary data (which is the aggregated data) and use the ODAG concept to show the detailed data if required.

This will help you to reduce large amount of data and improve the performance.

Regards,

Kaushik Solanki

Regards,
Kaushik
If reply is satisfactory, please "Like" the post.
If reply is solution, please mark as "Correct".
c_pannese
Partner
Partner
Author

Thank you Kaushik.

I will find out about it.

Regards

Chiara