topic Re: qvd's in QlikView

qvd's

Mon, 04 Aug 2014 13:14:28 GMT

Hi, community

I'm new to qlikview.what's the exact benefit of to creating qvd's.Can anyone explain on this in detail.

Request to help me in this.

Regards,

Sattish.

Re: qvd's

joshabbott — Mon, 04 Aug 2014 13:19:33 GMT

1 - QVD's can be loaded into a QVW 10-100 times faster then other data sources.

2 - It is good for data sources shared between multiple qvw's. You can share the QVD between multiple QVW's.

3 - Good for preceding loads. If you have data you constantly want to append to and grow, using QVD's will optimize this performance.

These are just a few of the benefits. Basically, in my opinion, you want to plan and not just build for the now. If this is a situation where you think you may use the dataset again, or need to append data, use a QVD. If you just have a simple spreadsheet you may only use once, there isn't much benefit in using a QVD because it would add another complexity and wouldn't give much benefit in return.

Re: qvd's

vardhancse — Mon, 04 Aug 2014 13:30:20 GMT

The important purpose of creating QVD was:

1.If the data load was happening from DB directly, then there will be performance issue

2.If there are any issues in DB, then there will be no data in the QV dashboard

3.We can store the data in QVD

4. the load time from QVD is much faster than direct DB

5. Decrease the reload time

Re: qvd's

avinashelite — Mon, 04 Aug 2014 13:45:56 GMT

Hi Sattish,

Please refer below:

QVD Files

A QVD (QlikView Data) file is a file containing a table of data exported from QlikView. QVD is a native QlikView format and can only be written to and read by QlikView. The file format is optimized for speed when reading data from a QlikView script but it is still very compact. Reading data from a QVD file is typically 10-100 times faster than reading from other data sources.

QVD files can be read in two modes, standard (fast) and super-fast. The selected mode is determined automatically by the QlikView script engine. Super-fast mode can be utilized only when all fields or a subset thereof are read without any transformations (formulas acting upon the fields), though the renaming of fields is allowed.

A QVD file holds exactly one data table and consists of three parts:

A well formed XML header (in UTF-8 char set) describing the fields in the table, the layout of the subsequent information and some other meta-data.
Symbol tables in a byte stuffed format.
Actual table data in a bit-stuffed format.

Purpose of QVD Files

QVD files can be used for many purposes. At least four major uses can be easily identified. More than one may apply in any given situation:

Increasing Load Speed

By buffering non-changing or slowly changing blocks of input data in QVD files, script execution becomes considerably faster for large data sets.

Decreasing Load on Database Servers

The amount of data fetched from external data sources can also be greatly reduced. This reduces work load on external databases and network traffic. Furthermore, when several QlikView scripts share the same data it is only necessary to load it once from the source database. The other applications can make use of the same data via a QVD file.

Consolidating Data from Multiple QlikView Applications

With the Binary script statement it is limited to loading data from a single QlikView application into another one, but with QVD files a QlikView script can combine data from any number of QlikView applications. This opens up possibilities e.g. for applications consolidating similar data from different business units etc.

Incremental Load

In many common cases the QVD functionality can be used for facilitating incremental load, i.e. exclusively loading new records from a growing database.To learn more about the use of QVD files and Incremental Load, follow the link below:Using QVD Files for Incremental Load

Create QVD Files

A QVD file can be created by one of three different methods:

Explicit creation and naming from script by means of the Store command. Simply state in the script that a previously read table or part thereof is to be exported to an explicitly named file at a location of your choice.
Automatic creation and maintenance from script. By preceding a load or select statement with theBufferprefix, QlikView will automatically create a QVD file which under certain conditions can be used instead of the original data source when reloading data.
Explicit, manual creation and naming from layout or via Internal Macro Interpreter. Data can be exported from the QlikView layout via GUI commands or Automation macros. In the GUI you will find QVD as one of the possible export formats under the Export... command, found on the object menu of most sheet objects.

There is no difference between the resulting QVD files, e.g. with regard to reading speed etc.

Reading Data from QVD Files

A QVD file can be read into or accessed by QlikView by the following methods:

Loading a QVD file as an explicit data source. QVD files can be referenced by a load statement in the QlikView script just like any other type of text files (csv, fix, dif, biff etc). The File Wizard: Type handles QVD files by the same principles.
Examples:
load * from xyz.qvd (qvd);load Name, RegNo from xyz.qvd (qvd);load Name as a, RegNo as b from xyz.qvd (qvd);
Automatic loading of buffered QVD files. When using the Buffer prefix on load or select statements, no explicit statements for reading are necessary. QlikView will determine the extent to which it will use data from the QVD file as opposed to acquiring data via the original load or select statement.
Accessing QVD files via the script. A number of script functions (all beginning with qvd) can be used for retrieving various information on the data found in the XML header of a QVD file. These functions are presented under the File Functions in Script functions.

Re: qvd's

Mon, 04 Aug 2014 13:55:42 GMT

If I can add to this.

In alot of curcumstances you are not allowed to pull data from source during office hours and therefore you right qvd's every night.

If you need to reload your model for whatever reason or if you need to use the same data set you can simply click the reload button.

Hope this helped.

Regards,
Marius

Re: qvd's

Mon, 04 Aug 2014 14:54:49 GMT

Thanks to every one to sharing very useful information.

Re: Re: qvd's

giakoum — Mon, 04 Aug 2014 14:57:42 GMT

Enjoy!

Re: qvd's

Mon, 04 Aug 2014 16:06:16 GMT

Thakq so much loannis! Very useful document.

Re: qvd's

Anonymous — Sat, 23 Aug 2014 20:47:00 GMT

Hi All

A nice post, but may I have my 2 cents.

We've been working on eliminating Incremental Loads altogether and removing DB access to Qlikview, and the speeds are a vast improvement on the Incremental Loads.

We had a 35M row 40 column DB table, to rebuild the QV table would take 3.5 hours, to rebuild from prebuilt CSVs takes 6 minutes.

Please read the following.

Incremental Loads are dead, long live the CSV

Re: qvd's

giakoum — Mon, 25 Aug 2014 08:46:35 GMT

Hi Dave.

I am sure that speed has improved, but there are many good reasons why people have moved from file based to database based solutions for storing data. The advantages of using a database over a file are so many and so important, making it difficult to ignore them.

I would try optimizing my database load instead, but of course, every case is unique, so csv may be the best solution in your case.

BR, Ioannis.