How to use Buffer Load when Script has Group by and Resident Tables.

[diego1991]

Good day to all of you,

I built an application in Qlik Sense Desktop which is using data from txt files that contain an average of 2 million registers per month, this app has to be updated every month containing the data from the actual and the previous months also.

My problem is that currently the app has too many data and is working really slow, so what I thought to do is to load the info using SUM and COUNT formulas and grouping the data with 'group by' and 'resident' tables but I noticed that Buffer Incremental Load doesn't works when grouping data like that,

I'm still kinda new in Qlik Sense, and don't know if there are other ways to do this, maybe you can recommend me tutorials for loading data properly or any other kind of advice, doesn't matter if the explanations are advanced I will take the necessary time to understand it.

Thanks a lot for reading.

Cordially,

Diego Vélez

[JordyWegman]

Hi Diego,

You can try the following script for a grouping.

Table:
Load
  Field1,
  Field2,
  Sum(Field3) as Field3
From [YourSource] (txt)
Group by Field1, Field2
;

You can expand this when you have more field. You need to add all the fields that are not aggregated (Counted/Summed etc.) to the Group by.

Jordy

Climber

[RadovanOresky]

Hi, I think you should create incremental load using QVD files and make the grouping on top of them.

This is a nice basic summary for incremental loads:

https://youtu.be/wJ0XMmK_5Z4

For a more complex advise you need to post more details or a simplified example of your script.

Hope this helps,

-Radovan-