topic Re: Load only new files in QlikView

Load only new files

evansabres — Wed, 10 Oct 2018 13:40:03 GMT

I am using a QVD to load all files in a directory. The files are quite large and therefore the re-load time is long. Once a file is loaded once, there is no reason for it be loaded again as the data will not change. Is there a method to only load "new" files?

Re: Load only new files

m_woolf — Wed, 10 Oct 2018 14:03:10 GMT

Delta Load Template

Re: Load only new files

evansabres — Wed, 10 Oct 2018 14:56:34 GMT

My directory contains .xlsx files and is not connected to a SQL db

Re: Load only new files

m_woolf — Wed, 10 Oct 2018 15:07:15 GMT

Those files all have FileTimes. The logic for doing an incremental load is the same.

When you first create the QVD, you load all the files and store the max Filetime in a variable.

When you do the incremental load, you load all data from the QVD. Then you concantenate load all files from the folder that have a Filetime greater than the stored max Filetime (variable).

Re: Load only new files

rwunderlich — Wed, 10 Oct 2018 18:48:01 GMT

Just add the prefix:

BUFFER (incremental)

to your LOAD statement. That will load only new files since the last load. The old files will come form the buffer which will be very fast.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

http://www.easyqlik.com

Re: Load only new files

evansabres — Thu, 11 Oct 2018 14:09:52 GMT

This worked wonderfully, thank you!

Re: Load only new files

m_woolf — Thu, 11 Oct 2018 15:02:01 GMT

Glad to help.

If any of the replies addressed your problem, please mark as Correct. If any were Helpful, please mark as Helpful.