Automatically load new data from CSV

[Pingcheater]

Hi everyone,

hope this question is right here, sorry if not - I've just started working with Qlik.

So here is our current set-up:

We've got a system, that provides data over a period of time in a csv file. Currently, it's one line per 15 mins and each csv file only has the header and the current line. The csv files are stored in a FTP folder.

What we want is, Qlik to load those files into a QVD automatically and once loaded, to delete them in order for the QVD to be the main data source, not the csv files. The filenames include a date and timestamp (so can be predicted), but it would be even better, if Qlik would just load all files in the folder (e.g. in case the server is down for a moment, he would pick up all files, he missed) in order not to miss any data sets.

Is that understandable/ plausible?

Thank you in advance!

[jheasley]

Yes - if I am understanding, you have something like the following:

5 files - SomeFile_Some_Time.CSV

all have the same fields. 

If so, then your load statement would be:

MainData:
Load
   Field1,
   Field2,
   Field3
From [filelocation/SomeFile_*.CSV]

then write a store command to store MainData into a QVD. When reloading from the QVD, you may want to do a load distinct to prevent duplicates. 

[Pingcheater]

Thanks jheasley,

that's exactly, what I have.

Is there a way to delete the csv files after a successful load to avoid having unnecessary files on the store and speed up the load?