Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Incremental load based on filename

Hello,

I need to have an incremental (only add) of some log files in a QVD.

The file name is of the format filename_date_time;csv. And there is a new file added every hour.

For the incremental load of the QVD how can I compare the last file loaded in the QVD and the latest log files in the folder so as to load only the files that have not been loaded?

Is it better to maintain a list of log files in a separate table?

Load *, filename() as name
From f:\Qlikview\*.csv

Load *

From f:\Qlikview\*.csv

where ...?

Thank you in advance,

Regards,

2 Replies
Jason_Michaelides
Partner - Master II
Partner - Master II

You could call a cmd prompt to move *.csv into a subfolder. That way you can always use *.csv in the load.

LOAD

     Filename()     AS     Name

     ,*

FROM *.csv;

EXECUTE /c cmd Move *.csv \Loaded;

Hope this helps,

Jason

Not applicable
Author

Wow that works like magic! I was trying to do something more complicated.

Would you also be having an example of an incremental reload with insert / update?

Thank you again!