Qlik Community

Qlik Sense App Development

Discussion board where members can learn more about Qlik Sense App Development and Usage.

Announcements
BARC’s The BI Survey 19 makes it official. BI users love Qlik. GET REPORT
Partner
Partner

Impact of Large Dataset with Group by in Load Statement

Hello,

While loading a dataset of 100 million records consisting of 5 columns, (one column is doing a Max aggregation function, and the other 4 columns are part of a Group by clause,) the computer crashes. No blue screen. When It crashed with the Group by the whole screen just froze completely, and then it rebooted.

Performing a load without the Group by, then using a Resident load with a Group by, all window objects failed. The desktop just died. I had to force a restart by holding down the power button for 30 seconds. But never had the blue screen in any of this.

Any thoughts on limitations with the Group by that may be contributing to this issue would be most appreciated.

Thank you,

Tony

Labels (1)
2 Replies
Partner
Partner

Re: Impact of Large Dataset with Group by in Load Statement

Group by is more expensive. But i have not heard of system crashes

You can optimize group by sorting the data (order by) by the same group and then applying group by. check below

https://community.qlik.com/t5/QlikView-Documents/Optimize-Group-By-Performance/tac-p/1481478

hope it helps

 

Partner
Partner

Re: Impact of Large Dataset with Group by in Load Statement

Thank you, dilipranjith, for your very helpful reply.  I'll try out your suggestion,

Tony