Qlik Community

Ask a Question

QlikView App Dev

Discussion Board for collaboration related to QlikView App Development.

Announcements
On May 18th at 10AM EDT we will answer your QlikView questions live. REGISTER
cancel
Showing results for 
Search instead for 
Did you mean: 
stevelord
Specialist
Specialist

incremental load of files

Hi, I have many folders with datafeed files accumulating on a daily basis and a qlikview of the contents of those files.  As the files become more numerous, the qlikview reload time increases.

Presently I have the qlikview process files dated more recently with:

Load.. From... Where NUM(<FileDateExpression>) >= NUM(Now()-2)

Then it loads the qvd of previously processed data with:

Load... From... Where NUM(<FileDateExpression>) < NUM(Now()-2)

This does a nice job of having it only do the processing work on newer files, and bring the previously processed stuff from older files out of the qvd that was made previously.  HOWEVER, the qlikview is still parsing through all of the files to check inside and see if any data to load.

Can someone provide me the script piece for qlikview to parse the file names themselves and not even look into the files when loading?  Would that save any more processing time in addition to time saved from not trying to process stuff in the files?

The filedate expression I have in the file loads is built on the filenames themselves, so would work to identify the files in such a script to tell it which ones to parse.

1 Reply
SergeyMak
Partner
Partner

You can use variables as file names or part of file name.

Like

//load old data from qvd

[data]:

LOAD *

FROM olddata.qvd (qvd);

//load the new data

FOR i=0 to 2

Concatenate(data)

LET vDate=Date(Now()-$(i));

LOAD *

FROM

data$(vDate).xls

(biff, embedded labels, table is Sheet1$);

Next

STORE data to olddata.qvd;

Regards,
Sergey