Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,@
I have a question with, say 2 scenarios :
1.Large data set projects
2.minimal data set projects eg: 1lakh records
Which load is faster and better with its performance ?
I am actually trying to load from qvds,perform join and take a resident of it with performing some calculations in it and use that table in data model. Is that right ?
table1:
load *
from tab1 // optimised
leftjoin
load *
from tab2; //optimised
tab 3:
load *
unit*price as quantity;
resident
table1;
drop table table1;
Hi Pradeep,
It depends which qvd the unit and price fields are in.
I tend to avoid using load* in a script and prefer to use the full list of field names, listing all fields makes debugging code much easier and gives you the option to easily remove unwanted fields by commenting them out.
If unit and price are in the same table, it may be better to do the calculation directly from the qvd, thus making the load not optimised, but this saves doing the extra resident load and drop table.
Also depending on how many fields you need for the table you are joining - using applymap instead of join may perform better. See this post Don't join - use Applymap instead In some cases I will use 5 or more applymap commands rather than joins, often with several mapping tables being created from the same base table.
You cannot give a direct answer to your question that A is better than B in this case - Test the options against your data, or a subset of the data, and see what performs best noting the timings from the document log.
Few of the main points to be taken for understand the architecture of the Qlikview.
1. Follow the Qlikview Architecture.
2. Loading data from the QVD is faster that any load. Ex:- Try to create the Qvds of the Main Tables (Transactions, Incremental, Masters etc).
3. What ever no. of rows you have qlikview read millions of the data in very fast and quickly depends on the hardware arch also here.
4. Maintain proper data modelling concept (STAR, SNOWFLAKE) what ever is required use that.
5. Always keep meaning full fields, Keys and drop those are not required to you.
Ex:-
table1:
load * from tab1 // optimised
leftjoin(table1)
load * from tab2; //optimised
tab_3:
load * unit * price as quantity resident table1;
drop table table1;
Thanks Anand. Can you just tell me which QVDS will get loaded faster ?
Hi Pradeep,
It depends which qvd the unit and price fields are in.
I tend to avoid using load* in a script and prefer to use the full list of field names, listing all fields makes debugging code much easier and gives you the option to easily remove unwanted fields by commenting them out.
If unit and price are in the same table, it may be better to do the calculation directly from the qvd, thus making the load not optimised, but this saves doing the extra resident load and drop table.
Also depending on how many fields you need for the table you are joining - using applymap instead of join may perform better. See this post Don't join - use Applymap instead In some cases I will use 5 or more applymap commands rather than joins, often with several mapping tables being created from the same base table.
You cannot give a direct answer to your question that A is better than B in this case - Test the options against your data, or a subset of the data, and see what performs best noting the timings from the document log.
Yeah I got it. Instead of creating resident , I can use apply map function where ever required.
Thanks Man!