Some questions for the experts about Buffer LOAD

[mwallman]

Hello,

I have some questions about the Buffer function:

1) What exactly does Stale option do in Buffer? Does it delete the buffer file after x number of hours or days is specified?

2) Does the Incremental option for Buffer work for text files and database tables too? E.g. if new rows of data are added to a table in a database or a excel file etc and I use buffer, will it read the new rows?

3) Is there a way to change the file name QlikView gives to buffer QVD files? E.g. instead of using 160-bit hexadecimal hash I want to specify my own name for each buffer table?

4) I believe it is no possible to get the buffer QVD files to check on deleted and updated values from the original table. Is this true? Would a developer have to use other methods instead of Buffer?

[swuehl]

The HELP already gives some good hints here:

1)

The stale after option is typically used with DB sources where there is no simple timestamp on the original data. Instead one specifies how old the QVD snapshot can be to be used. A stale after clause simply states a time period from the creation time of the QVD buffer after which it will no longer be considered valid. Before that time the QVD buffer will be used as source for data and after that the original data source will be used. The QVD buffer file will then automatically be updated and a new period starts.

Hence it doesn't just delete the file. When QV encounters the stale buffer option during reload, it will check how old the buffer QVD is. If it's younger than the stale period, it's used, if it's older, the original source is used and the QVD will be updated.

2) It works for text files only, as indicated in the HELP:

The incremental option enables the ability to read only part of an underlying file. Previous size of the file is stored in the XML header in the QVD file. This is particularly useful with log files. All records loaded at a previous occasion are read from the QVD file whereas the following new records are read from the original source and finally an updated QVD-file is created. Note that the incremental option can only be used with load statements and text files and that incremental load cannot be used where old data is changed or deleted!

3) I don't think that you can rename the buffer file. Reason is also stated in the HELP:

The name of the QVD file is a calculated name (a 160-bit hexadecimal hash of the entire following load or select statement and other discriminating info) and is typically stored in the Application Data folder or another folder specified under User Preferences: Locations. This means that the QVD buffer will be rendered invalid by any change in the following load or select statement.

The script code of the LOAD / SELECT statement is encoded (checksummed) in the file name, so the file name itself serves a certain purpose (find the correct buffer / invalidate when codes changes).

4) I think you are looking for incremental load options (not to be confused with incremental buffer load option). Search for 'incremental load' here in the forum or have a look at the excellent sample code in Rob Wunderlich's QV cookbook:

http://qlikviewcookbook.com/download/delta-load-template/