topic Re: Buffer Load - Regarding in QlikView

Buffer Load - Regarding

kkkumar82 — Thu, 03 Mar 2016 05:05:35 GMT

All,

I see two options in settings menu regarding the qvds created using buffer command

1. Settings -> Document properties - > general tab - > Keep unreferenced Qvd Buffers

2. Settings -> User references -> Keep orphaned Qvd Buffers

Can any body clarify what is the difference between "Orphaned" and "Unreferenced" qvds in this context.

Thanks

Kiran

Re: Buffer Load - Regarding

jagan — Thu, 03 Mar 2016 05:17:47 GMT

Hi,

QVD buffers will normally be removed when no longer referenced anywhere throughout a complete script execution in the document that created it or when the document that created it no longer exists. This housekeeping can be turned off by checking the options Keep Unreferenced QVD Buffers in Document Properties: General (first requirement) and Keep Orphaned QVD Buffers in User Preferences: General (second requirement) property pages respectively, though the procedure cannot be generally recommended.

Regards,

Jagan.

Re: Buffer Load - Regarding

kkkumar82 — Thu, 03 Mar 2016 05:23:33 GMT

Hi jagan,

You mean to say both the options must be checked to house keep the qvds which are created by buffer command.

What I just want to know is the second option can be used independent of first one.

thanks

Kiran Kumar

Re: Buffer Load - Regarding

tresB — Thu, 03 Mar 2016 05:24:49 GMT

It guess it's like:

Keep unreferenced Qvd Buffers - QVD's would be purged if the qvw that created those doesn't use it any more

Keep orphaned Qvd Buffers - QVD's would be purged if the qvw that created those doesn't exist it any more

Re: Buffer Load - Regarding

kkkumar82 — Thu, 03 Mar 2016 05:29:33 GMT

Hi Tresco,

For the first one , lets assume I have run the script using buffer command and qvds got created in a location, after that I didn't re run the qvw for a while. My question here , "Is there any time QV identifies that qvw is didn't run and removes the qvds or how it knows we are not using the qvd.

The second point is clear because if the qvw is removed the qvd will be remove, just didn't understand how QV is counting time for the first option.

Thanks

Kiran Kumar

Re: Buffer Load - Regarding

tresB — Thu, 03 Mar 2016 05:37:46 GMT

I guess one reload without referencing the qvds would be required before its get purged. And does it know then? My reasoning would be - it must search for the path (default application folder or a customized one) once it found 'buffer' in earlier reload. There should some flag maintaining.

Re: Buffer Load - Regarding

kkkumar82 — Thu, 03 Mar 2016 06:18:37 GMT

Any more thoughts on this .

Thanks

kiran kumar

Re: Buffer Load - Regarding

avinashelite — Thu, 03 Mar 2016 12:11:23 GMT

Please have a look at the HELP:

Buffer

QVD files can be created and maintained automatically via the buffer prefix. This prefix can be used on mostLoad and Select (SQL) statements in script. It indicates that QVD Files are used to cache/buffer the result of the statement.

Numerous limitations exist, most notable is that there must be either a file load or a select statement at the core of any complex statement.

The name of the QVD file is a calculated name (a 160-bit hexadecimal hash of the entire following load or selectstatement and other discriminating info) and is typically stored in the Application Data folder or another folder specified under User Preferences: Locations. This means that the QVD buffer will be rendered invalid by any change in the following load or select statement.

QVD buffers will normally be removed when no longer referenced anywhere throughout a complete script execution in the document that created it or when the document that created it no longer exists. This housekeeping can be turned off by checking the options Keep Unreferenced QVD Buffers in Document Properties: General (first requirement) and Keep Orphaned QVD Buffers in User Preferences (second requirement) property pages respectively, though the procedure cannot be generally recommended.

The syntax is:

buffer[ (option [ , option] ) ] ( loadstatement | selectstatement )

where:

option ::= incremental | expiry

expiry::= stale [after]amount[ (days | hours)]

amount is a number specifying the time period. Decimals may be used. The unit is assumed to be days if omitted.

The incremental option enables the ability to read only part of an underlying file. Previous size of the file is stored in the XML header in the QVD file. This is particularly useful with log files. All records loaded at a previous occasion are read from the QVD file whereas the following new records are read from the original source and finally an updated QVD-file is created. Note that the incremental option can only be used with loadstatements and text files and that incremental load cannot be used where old data is changed or deleted!

The stale after option is typically used with DB sources where there is no simple timestamp on the original data. Instead one specifies how old the QVD snapshot can be to be used. A stale after clause simply states a time period from the creation time of the QVD buffer after which it will no longer be considered valid. Before that time the QVD buffer will be used as source for data and after that the original data source will be used. The QVD buffer file will then automatically be updated and a new period starts.

If no option is used, the QVD buffer created by the first execution of the script will be used indefinitely.

Examples:

buffer SQL SELECT * FROM MyTable;

buffer (stale after 7 days) SQL SELECT * FROM MyTable;

buffer (incremental) LOAD * FROM MyLog.log;