Faster QVD ? Optimised or Unoptimised QVD

[pradeep92]

Hi,@

I have a question with, say 2 scenarios :

1.Large data set projects

2.minimal data set projects eg: 1lakh records

Which load is faster and better with its performance ?

I am actually trying to load from qvds,perform join and take a resident of it with performing some calculations in it and use that table in data model. Is that right ?

table1:

load *

from tab1  // optimised

leftjoin

load *

from tab2; //optimised

tab 3:

load *

unit*price as quantity;

resident

table1;

drop table table1;

[its_anandrjs]

Few of the main points to be taken for understand the architecture of the Qlikview.

1. Follow the Qlikview Architecture.

2. Loading data from the QVD is faster that any load. Ex:- Try to create the Qvds of the Main Tables (Transactions, Incremental, Masters etc).

3. What ever no. of rows you have qlikview read millions of the data in very fast and quickly depends on the hardware arch also here.

4. Maintain proper data modelling concept (STAR, SNOWFLAKE) what ever is required use that.

5. Always keep meaning full fields, Keys and drop those are not required to you.

Ex:-

table1:

load * from tab1  // optimised

leftjoin(table1)

load * from tab2; //optimised

tab_3:

load * unit * price as quantity resident table1;

drop table table1;

[pradeep92]

Thanks Anand. Can you just tell me which QVDS will get loaded faster ?

[Colin-Albert · Accepted solution]

Hi Pradeep,

It depends which qvd the unit and price fields are in.

I tend to avoid using load* in a script and prefer to use the full list of field names, listing all fields makes debugging code much easier and gives you the option to easily remove unwanted fields by commenting them out.

If unit and price are in the same table, it may be better to do the calculation directly from the qvd, thus making the load not optimised, but this saves doing the extra resident load and drop table.

Also depending on how many fields you need for the table you are joining - using applymap instead of join may perform better.  See this post Don't join - use Applymap instead  In some cases I will use 5 or more applymap commands rather than joins, often with several mapping tables being created from the same base table.

You cannot give a direct answer to your question that A is better than B in this case - Test the options against your data, or a subset of the data, and see what performs best noting the timings from the document log.

[pradeep92]

Yeah I got it. Instead of creating resident , I can use apply map function where ever required.

Thanks Man!