Qlik Community

App Development

Discussion board where members can learn more about Qlik Sense App Development and Usage.

Announcements
Customer & Partners, DEC. 9, 11 AM ET: Qlik Product & Strategy Roadmap Session: Data Analytics REGISTER NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Tony
Partner
Partner

Impact of Large Dataset with Group by in Load Statement

Hello,

While loading a dataset of 100 million records consisting of 5 columns, (one column is doing a Max aggregation function, and the other 4 columns are part of a Group by clause,) the computer crashes. No blue screen. When It crashed with the Group by the whole screen just froze completely, and then it rebooted.

Performing a load without the Group by, then using a Resident load with a Group by, all window objects failed. The desktop just died. I had to force a restart by holding down the power button for 30 seconds. But never had the blue screen in any of this.

Any thoughts on limitations with the Group by that may be contributing to this issue would be most appreciated.

Thank you,

Tony

Labels (1)
2 Replies
dilipranjith
Partner
Partner

Group by is more expensive. But i have not heard of system crashes

You can optimize group by sorting the data (order by) by the same group and then applying group by. check below

https://community.qlik.com/t5/QlikView-Documents/Optimize-Group-By-Performance/tac-p/1481478

hope it helps

 

Tony
Partner
Partner
Author

Thank you, dilipranjith, for your very helpful reply.  I'll try out your suggestion,

Tony