loading of files incrementally with filenames

[shree909]

Hi folks,

i want to do the loading of files incrementally as i will be getting new .csv files daily which are identified by the file name..

i dont know whether this is doable or not...

1) i will create the qvd for the existingf files. then

2) i have to create a  new qvd file  which is not present in the previous qvd files and store that as a new qvd and append that to the old qvd.

3) so this has to identified by the file name which are  loading daily..

so i think we have to write a script in such a way that old filenames are  stored in a variable and use that variable to compare the new files in the folder and load the new data.

so how the script to be written with if condition or something else..

[Gysbert_Wassenaar]

Unless the size of the csv files makes this impractical I'd first try simply always reload all csv files

Table1:

BUFFER load ...

from *.csv

(txt, codepage is 1252, embedded labels);

STORE Table1 into MyData.qvd;

If that turns out to take too long then you could load all new cvs files and then move the files to a different directory so only the new files will get loaded.

Table1:

Load * from MyData.qvd;

concatenate(Table1)

BUFFER load ...

from *.csv

(txt, codepage is 1252, embedded labels);

STORE Table1 into MyData.qvd;

execute cmd.exe /c move d:\datafiles\source\*.csv d:\datafiles\processed\;