Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Incremental Log file load

Howdy,

I'm kind of confused about incremental load from log files...

Is it as easy as...

Buffer (Incremental)

Load *

From

O:\DavidC\3CX\*.txt

(txt, codepage is 1252, no labels, delimiter is ',', msq, filters(

Rotate(left)));

The question is, why is this not speeding up my reload times. I'm loading phone call log files, which grow daily. Shouldn't this script ONLY load the new files? Every day...my reaload time grows and grows as my number of log files.

-David

5 Replies
rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

Yes, it is that easy.

You will see "loading" messages for the "old" files, but it is only checking the filesize and not actually loading any data from the files -- it gets that data form the buffered QVD.

What I'm not sure about is the effect of the filters(). Can you try it without the filter as a timing test?

-Rob

Not applicable
Author

Do you mean my "rotate(left" cross table part of the script?

rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

Yes, I meant remove the

filters(

Rotate(left));

I know it won't give you the correct result, but just as a test.

-Rob

Not applicable
Author

OK, i'll update after the test.

What's the different in the script execution window between records with the "(buffered)" and the ones without?

-DC

rwunderlich
Partner Ambassador/MVP
Partner Ambassador/MVP

I think you see a "buffered" message at the beginning of the load, but other than that the progress window looks the same.

-Rob