Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
munin7665
Contributor
Contributor

Replace only one QVD of the lot by partial load.

Hi All,

I have a dashboard where I have to load four QVDs.

1)Full Year 2017 data.QVD

2)Full Year 2018 data.QVD

3)Budget 2018.QVD

4)Current Year to Date data.QVD

So each day i have to pull the data extract Current Year to Data (always year to date data as there would be changes ) create QVD and than load the entire set of QVDs.

Problem: it takes a lot of time as each qvs contains almost 90 millions rows. However the Current Year to Date data is lesser. Is there is a way i can replace only the Current Year to date.qvd from qlik memory and replace it with the new QVD every time i want to. I tried Replace only but it erases all the data and replace and keeps only the Current year data.

Thanks a lot for your help in advance. 

Munin

4 Replies
jyothish8807
Master II
Master II

Hi Munin,

 

May be like this:

Tab1:

Load Full Year 2017 data.QVD;

Tab2:

Load Full Year 2018 data.QVD;

Tab3:

Load Budget 2018.QVD;

Tab4:

Add Load Current Year to Date data.QVD;

 

Since you will have updated "Current Year to Date data.QVD" every time, while doing partial load the data will get updated.

Best Regards,
KC
jyothish8807
Master II
Master II

Hi Munin,

May be like this:

Tab1:

Load Full Year 2017 data.QVD;

Tab2:

Load Full Year 2018 data.QVD;

Tab3:

Load Budget 2018.QVD;

Tab4:

Add Load Current Year to Date data.QVD;

 

Since you will have updated "Current Year to Date data.QVD" every time, while doing partial load the data will get updated.

Best Regards,
KC
mrybalko
Creator II
Creator II

Hello @munin7665 


Problem: it takes a lot of time as each qvs contains almost 90 millions rows. However the Current Year to Date data is lesser


Do you use optimized qvd loading? Partial reload does not suit your case because you have to replace some fact table rows. 

jonathandienst
Partner - Champion III
Partner - Champion III

I assume that you are concatenating  these table on loading, so you want to replace he current year part of an existing resident table with the updated current year data?

I doubt you will achieve a meaningful time reduction as you would first need a resident load to drop all the current year records. This is very likely not measurably faster than an optimised QVD load, and may even be slower (as counter intuitive as that may sound).

To improve performance of the QVD loads, make sure that the loads are optimized loads. This means no transformations, and no filters other than single parameter Where Exists(). Do any transformations and filtering in the QVD generation phase, even if that means splitting it into two steps (Extract and Transform).

Secondly, store the QVDs on a local drive for the server to prevent network transport from limiting your performance.

Logic will get you from a to b. Imagination will take you everywhere. - A Einstein