topic Incremental load based on filename in QlikView

Incremental load based on filename

Mon, 04 Nov 2013 14:35:18 GMT

Hello,

I need to have an incremental (only add) of some log files in a QVD.

The file name is of the format filename_date_time;csv. And there is a new file added every hour.

For the incremental load of the QVD how can I compare the last file loaded in the QVD and the latest log files in the folder so as to load only the files that have not been loaded?

Is it better to maintain a list of log files in a separate table?

Load *, filename() as name
From f:\Qlikview\*.csv

Load *

From f:\Qlikview\*.csv

where ...?

Thank you in advance,

Regards,

Re: Incremental load based on filename

Jason_Michaelides — Mon, 04 Nov 2013 14:44:52 GMT

You could call a cmd prompt to move *.csv into a subfolder. That way you can always use *.csv in the load.

LOAD

Filename() AS Name

FROM *.csv;

EXECUTE /c cmd Move *.csv \Loaded;

Hope this helps,

Jason

Re: Incremental load based on filename

Mon, 04 Nov 2013 16:33:37 GMT

Wow that works like magic! I was trying to do something more complicated.

Would you also be having an example of an incremental reload with insert / update?

Thank you again!