Reload strategy Help !

amit_saini — Tue, 18 Aug 2015 06:29:22 GMT

Hi Folks,

Below is my script for storing QVD's for one of my application. We are triggering this task every 2 hours , but it has been many times that we are missing some data . So to avoid this I want to come up with an idea to load the data from beginning of the week over the weekends. No sure if this is possible but i need help on this.

For example : This week starts form 17th Aug (Monday) , below script will keep on running until Friday i.e. 21st Aug and on 22nd Aug (Weekend) I want to load complete data from 17th until 22nd .

Please suggest what could be the best approach to avoid loss of data.

LET vStartDate = date(makedate(2014,1), 'DD.MM.YYYY hh:mm:ss');

Set vSignal = '..\0_Signal';

Set vQVD = 'D:\qvprod\QV_QVD';

Let vQVDCurrentYear = 'D:\qvprod\QV_QVD\'& year(today());

//####################################################

$(Include=D:\qvprod\Operation\SCAQ\#Source\Misc\0_db_connections.qvs)

//********* Amit 20120508**********************

Let vStart = now();

Let vReloadStart = now();

//********************

///$tab MM_REV

MM_REV:

LOAD

'$(vPlant)' as Plant,

SQL SELECT

NEINHEIT,

NLFDMMNR,

NLFDMMSPEZNR

FROM $(vPlantAutor)."MM_REV";

//********* Amit 20120508**********************

LET vMM_REV_LOAD_DURATION = Interval(now() - vStart) ;

LET vStart = now();

//********************

store MM_REV into $(vQVD)\$(vPlant)_MM_REV.qvd;

//********* Amit 20120508**********************

LET vMM_REV_Size = num(filesize('$(vQVD)\$(vPlant)_MM_REV.qvd')/1024,'00.0') & ' KB';

LET vMM_REV_Records = QvdNoOfRecords ('$(vQVD)\$(vPlant)_MM_REV.qvd');

//********************

drop table MM_REV;

///$tab STATION

STATION:

LOAD

'$(vPlant)' as Plant,

SQL SELECT

NLFDSTATIONNR,

SSTATIONBEZ,

SSTATIONNR

FROM $(vPlantAutor).STATION;

//********* Amit 20120508**********************

LET vSTATION_LOAD_DURATION = Interval(now() - vStart) ;

LET vStart = now();

//********************

store STATION into $(vQVD)\$(vPlant)_STATION.qvd;

Thanks,

Re: Reload strategy Help !

Peter_Cammaert — Tue, 18 Aug 2015 07:04:07 GMT

I would suggest that you first investigate why your data is incomplete when your script does a complete load every two hours. Is it the data volume that is causing trouble?

Another question: if you load the complete data set every two hours (no WHERE clauses that reduce the data set to a

specific period), then what's the use of only loading one week in the weekend?

The best approach IMHO is to perform an Incremental LOAD during weekdays, and perform full LOADs in the weekend. But if your current Full LOADs have issues on weekdays, they'll will most probably have the same issues in the weekend.

Best,

Peter

topic Reload strategy Help ! in QlikView

Reload strategy Help !

Re: Reload strategy Help !