Skip to main content

QlikView App Dev

Discussion Board for collaboration related to QlikView App Development.

Announcements
QlikWorld 2023, a live, in-person thrill ride. Save $300 before February 6: REGISTER NOW!
cancel
Showing results for 
Search instead for 
Did you mean: 
evansabres
Specialist
Specialist

Load only new files

I am using a QVD to load all files in a directory. The files are quite large and therefore the re-load time is long. Once a file is loaded once, there is no reason for it be loaded again as the data will not change. Is there a method to only load "new" files?

1 Solution

Accepted Solutions
rwunderlich

Just add the prefix:

BUFFER (incremental)

to your LOAD statement.  That will load only new files since the last load.  The old files will come form the buffer which will be very fast.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

http://www.easyqlik.com

View solution in original post

6 Replies
m_woolf
Master II
Master II

evansabres
Specialist
Specialist
Author

My directory contains .xlsx files and is not connected to a SQL db

m_woolf
Master II
Master II

Those files all have FileTimes. The logic for doing an incremental load is the same.

When you first create the QVD, you load all the files and store the max Filetime in a variable.

When you do the incremental load, you load all data from the QVD. Then you concantenate load all files from the folder that have a Filetime greater than the stored max Filetime (variable).

rwunderlich

Just add the prefix:

BUFFER (incremental)

to your LOAD statement.  That will load only new files since the last load.  The old files will come form the buffer which will be very fast.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

http://www.easyqlik.com

evansabres
Specialist
Specialist
Author

This worked wonderfully, thank you!

m_woolf
Master II
Master II

Glad to help.

If any of the replies addressed your problem, please mark as Correct. If any were Helpful, please mark as Helpful.