3 Replies Latest reply: Feb 1, 2013 11:02 AM by Michael Tarallo RSS

    Best practice and performance (Aggregation)



      Following on from a previous post on row merge and logic (http://community.qlik.com/message/299379#299379) I am looking at this further and wondering about what the QVE best practice is for performance when introducing larger volumes, both rows and columns.


      In the original solution we used the aggregation to process pairs of records which needed some logic applied to decide on relevant outputs, along with some sum's.


      If this is now expaned to say 10 different sets of logic to provide different outputs what is the best approach for performance, best practice and maintainability;


      1) Use one aggregation artifact with one function rule, include all logic within that.


      2) Use one aggregation artifact, with multiple function rules to process each set of logic seperately


      3) Use multiple aggregation artifacts with one function rule.


      My concern with 3 is that incoming dataset would need to be copied x number of times to each aggregation, and would probably add load to the workflow, 2 seems to be the best for managing each set of logic and requires one copy of the data set to be fed in.


      1 doesn't appear to be all that different from 2, except that you are only processing one function rule, how much burden does processing say 10 function rules within the same aggregation cause, or is it not that much?


      Is there any documentation on QVE best practice for data flow design?