How to load a QVD of 200 GB in Qlikview.

[Anonymous]

Hi ,

I am new to Qlikview and in an interview i was asked how i would load a single QVD file of about 200GB.

Can you tell me the best possible method to perform this task.

Thank you !

[oknotsen]

Exactly the same way as you would load a 1MB QVD file.

[Bill_Britt]

Hi Mamtha,

This is a very large file and if I were to use it I would use it in Optimized mode. There are two modes you can run a QVD in.

1. Standard which means you have Transformation a file when you load the QVD Like day(TempDate) as Day. This will changed the load into Standard mode which is slower than the Optimized mode.

2. Optimized means you are doing a straight load from the QVD and nothing is changed.

If you have to transform the file do it on the QVD creation load.

Bill

[evan_kurowski]

Hello Mamtha,

Make sure you have a server north of 250GB of RAM and a lot of patience.  While optimized loads are really quick, and in-memory analysis is very responsive..  a 200GB QVD might take a while to search/calculate.  Do a Cartesian!

[SreeniJD]

Hi Mamta

I can understand why the questions was asked in your interview..

This was to test your knowledge on prerequisites of loading QVD files and how much you are familiar on system configuration.

As per Evan's suggestion, One has to check the Server RAM before loading 200GB of data and of course a lot of patience. Without any second thought you have only one option to go is "Optimized Loads" which is you are giving additional burden to your system to fetch the data, in simple words.. its just dumping the data from your source without applying any filters or transformations. You can have where exists() clause if required.

Also, please read the following articles on Optimized and Un-optimized loads!

Optimized and Non Optimized loading

Optimized and unoptimized QVD Load Situations? thanks in advance

optimize and non-optimze load in qvd

www.quickintelligence.co.uk/qlikview-optimised-qvd-loads/

Happy Learning!

Sreeni

[Anonymous]

Thank you Sreeni!

[Anonymous]

Thanks Bill !!

[qlikviewwizard]

You can use in normal way.It is better to load required fields for better performance and we need enough RAM in machine.

[Anonymous]

Hi mamatha,

Simple answer its based on the Ram it should hold 200 GB size of  Qvd and such may be used as ,

UN-Transformed - fully optimized But you can use where clause if required

Transformed - It will be slow . Tests patience.

But better way is to avoid such qvd's to load at Once ,

*For performance of the Application

*Just Pick the required information for a particular application.

              EG:- Current year and previous year data

*where if the user requires the complete data means it will done using Dynamic Qlikview application .

Thanks ,

Bunny

[#10]

Hi Mamatha,

QlikView doesn't have any limitations on the data to load instead it depends on the RAM size you have.

Generally, the amount of RAM consumption is 4 times the data size and also depends on the compression ratio applied.

Optimal techniques is to perform Optimal QVD which means there are no constrains when creating the QVD.

The best solutions are to perform incremental refresh of data, try to perform distribution of data by using document distribution.

Section Access can be helpful which limits the data to be accessible only to privileged users.

Hope it helps!

Thanks

SV