To summarize, what is the way to perform an Incremental Load on a folder full of csv files without Loading the data against the 'Where" condition?
I have a folder with daily files going back 10 years+. To save time, I ran an initial load and Stored everything into a QVD file.
Moving forward, I would like to add data from only New files, but on an ad-hoc basis.
When running an Incremental Load, I do not want to open ALL files to match the "Where" statement. I've set an artificial variable limit (Year portion of the Last_Modified_Date 2015) with a wildcard into the file name, so the Load would only look into the current year. This will be problematic at year-end.
The problem I'm trying to overcome is that, say after 200 days, that is 200 files to load for a match to my Where statement. This takes quite a bit of time. Is there a way to perform an Incremental Load on a folder full of CSV files without Loading the data against the 'Where" condition?
File Names have have a date stamp in the file name.
I'm a new QlikView user! A code example would help.