Skip to main content
Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE
cancel
Showing results for 
Search instead for 
Did you mean: 
amit_saini
Master III
Master III

Reload strategy Help !

Hi Folks,

Below is my script for storing QVD's for one of my application. We are triggering this task every 2 hours , but it  has been many times that we are missing some data . So to avoid this I want to come up with an idea to load the data from beginning of the week over the weekends. No sure if this is possible but i need help on this.

For example : This week starts form 17th Aug (Monday) , below script will keep on running until Friday i.e. 21st Aug and on 22nd Aug (Weekend) I want to load complete data from 17th until 22nd .

Please suggest what could be the best approach to avoid loss of data.

LET vStartDate = date(makedate(2014,1), 'DD.MM.YYYY hh:mm:ss');

Set vSignal = '..\0_Signal';

Set vQVD = 'D:\qvprod\QV_QVD';

Let vQVDCurrentYear = 'D:\qvprod\QV_QVD\'& year(today());

//####################################################

$(Include=D:\qvprod\Operation\SCAQ\#Source\Misc\0_db_connections.qvs)

//********* Amit 20120508**********************

Let vStart = now();

Let vReloadStart = now();

//********************

///$tab MM_REV

MM_REV:

LOAD

    '$(vPlant)' as Plant,

    *;

SQL SELECT

    NEINHEIT,

    NLFDMMNR,

    NLFDMMSPEZNR

FROM $(vPlantAutor)."MM_REV";

//********* Amit 20120508**********************

LET vMM_REV_LOAD_DURATION = Interval(now() - vStart) ;

LET vStart = now();

//********************

store MM_REV into $(vQVD)\$(vPlant)_MM_REV.qvd;

//********* Amit 20120508**********************

LET vMM_REV_Size = num(filesize('$(vQVD)\$(vPlant)_MM_REV.qvd')/1024,'00.0') & ' KB';

LET vMM_REV_Records = QvdNoOfRecords ('$(vQVD)\$(vPlant)_MM_REV.qvd');

//********************

drop table MM_REV;

///$tab STATION

STATION:

LOAD

    '$(vPlant)' as Plant,

    *;

SQL SELECT

    NLFDSTATIONNR,

    SSTATIONBEZ,

    SSTATIONNR

FROM $(vPlantAutor).STATION;

//********* Amit 20120508**********************

LET vSTATION_LOAD_DURATION =  Interval(now() - vStart) ;

LET vStart = now();

//********************

store STATION into $(vQVD)\$(vPlant)_STATION.qvd;



Thanks,

AS

1 Reply
Peter_Cammaert
Partner - Champion III
Partner - Champion III

I would suggest that you first investigate why your data is incomplete when your script does a complete load every two hours. Is it the data volume that is causing trouble?

Another question: if you load the complete data set every two hours (no WHERE clauses that reduce the data set to a

specific period), then what's the use of only loading one week in the weekend?


The best approach IMHO is to perform an Incremental LOAD during weekdays, and perform full LOADs in the weekend. But if your current Full LOADs have issues on weekdays, they'll will most probably have the same issues in the weekend.


Best,


Peter