Partial Reload accumulating data but based on the same source file changing

Wed, 27 Jul 2011 11:33:17 GMT

Hello,

i`m facing with the problem how to code accumulative partial reloads where source of new data is
the same file named 'src.csv' and is ovverwiritten by other process with new data every hour. QV application
should start partial reload every hour and appending new content of 'src.csv' (adding to data info what time
they were generated) but still keeping old data but not older than 14 days.

I`ve coded so far sth like below:

SET HistoricDays = 14;
LET ftime = FileTime('src.csv');
LET minDate = Date#((Date(today())-$(HistoricDays)),'YYYY-MM-DD'); // lowest historical date to keep in datamodel at parial reload
//--------------------------------------------------------------------------------------
// load incremental data from changing src.csv file
Departures_tmp:
add LOAD 
'$(ftime)' as D_CREATION_DATE_TIME,
  Date#(left('$(ftime)',10),'YYYY-MM-DD') as D_CREATION_DATE,
  data1,
  data2,
  ...
FROM
src.csv
(txt, codepage is 1250, no labels, delimiter is ';', msq);

// remove data older than 14 days (minDate) from the table at partial reload.
Departures_tmp2:
replace LOAD
D_CREATION_DATE_TIME,
D_CREATION_DATE,
data1,
data2,
...
RESIDENT Departures_tmp where Date#(D_CREATION_DATE) > $(minDate);
DROP TABLE Departures_tmp;
//-------------------------------------------------------------------------------------- 
When I trigger 'partial reload':
add LOAD works fine and appends new data into existing ones.
but 'replace LOAD' seems does not work , it looks like it loads only 'fresh' data from src.csv
(but oryginaly it bases on RESIDENT Departures_tmp table). I`ve tried replace 'replace LOAD' with 'add LOAD' 
 but it does not work eather.
 
Does any one have a good Idea to overcome this ?
 
 Cheers.

topic Partial Reload accumulating data but based on the same source file changing in QlikView

Partial Reload accumulating data but based on the same source file changing