How to reload latest QVD file only after click 'Load Data' button

[mudanhong]

Below is my incremental loading script:

Test:

Load

*

where primary key>'xxx';

select * FROM CSV (header on, delimiter ",", quote """") "abc"

WITH CONNECTION (Url "$(vUrl1)");

concatenate load

* 

from abc.qvd

where not exists(primary key);

This only works for avoiding duplicate calculating for the whole abc.qvd each now I start this app task. 

But if there is any way that I can only load latest qvd into app RAM rather than loading the whole year QVD file data into app RAM again? I means if there is a way that app could keep the previous run qvd data in RAM and only load required latest data onto the old data in app RAM?

Test:

Load

*

from abc_2021.02.02.qvd;//2021.02.02 should be defined as latest date

[micheledenardi]

How 2021.02.02 should be determinated? is it the yesterday date? Last reload date?

[mudanhong]

It is just a specified date. yesterday/today/last reload date whose data has not been loaded into abc.qvd file.

[micheledenardi]

In any case you've to load the entire dataset but you can use Binary statement to load data from your datamodel without load data from Qvd/csv and refresh only some tables.

Something like This:

Binary Test.qvf;

SET ThousandSep='.';
SET DecimalSep=',';
SET MoneyThousandSep='.';
SET MoneyDecimalSep=',';
SET MoneyFormat='#.##0,00 €;-#.##0,00 €';
SET TimeFormat='hh:mm:ss';
SET DateFormat='DD/MM/YYYY';
...
...
...
...
SET NumericalAbbreviation='3:k;6:M;9:G;12:T;15:P;18:E;21:Z;24:Y;-3:m;-6:μ;-9:n;-12:p;-15:f;-18:a;-21:z;-24:y';


concatenate(Test)
load * 
from abc_*.qvd
where not exists(primary key);

[mudanhong]

Hi Michele,

I have tried your way. But I got below error, should I give the absolute path?

Started loading data

The following error occurred:

No qualified path for file: ***

The error occurred here:

Binary QvdIncrementalTest.qvf

[mudanhong]

Thanks Michele, I have tried successfully with absolute path of another app. 

[mudanhong]

But it seems not quicken the app speed😂. I would try a simple app to test its effect on app performance. Thank you for your help.

[micheledenardi]

You're getting that error because your QVF file is not in the server root directory.

So, if you want to use custom path for your QVF you have to create a directory connection to your QVF file path and use it in your binary statement:

Binary lib://YourQvfPath/QvdIncrementalTest.qvf;

[mudanhong]

Thanks Michele, I will do that with our Admin team. 

But I tried a lot of times. Binary loading qvf file still costs a lot of time. might not be faster than Concatenating loading data from qvd file.

Do you suggest us use Binary loading than loading data from qvd file to improve the app performance?

[micheledenardi]

Read data from QVD Optimized is always the fastest solution but Binary is an alternative to refresh data without read data directly from QVDs.

I believe the bottleneck of your script is during the select statement and not during Concatenate(). Right?