Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Tony
Partner - Contributor
Partner - Contributor

Impact of Large Dataset with Group by in Load Statement

Hello,

While loading a dataset of 100 million records consisting of 5 columns, (one column is doing a Max aggregation function, and the other 4 columns are part of a Group by clause,) the computer crashes. No blue screen. When It crashed with the Group by the whole screen just froze completely, and then it rebooted.

Performing a load without the Group by, then using a Resident load with a Group by, all window objects failed. The desktop just died. I had to force a restart by holding down the power button for 30 seconds. But never had the blue screen in any of this.

Any thoughts on limitations with the Group by that may be contributing to this issue would be most appreciated.

Thank you,

Tony

Labels (1)
2 Replies
dplr-rn
Partner - Master III
Partner - Master III

Group by is more expensive. But i have not heard of system crashes

You can optimize group by sorting the data (order by) by the same group and then applying group by. check below

https://community.qlik.com/t5/QlikView-Documents/Optimize-Group-By-Performance/tac-p/1481478

hope it helps

 

Tony
Partner - Contributor
Partner - Contributor
Author

Thank you, dilipranjith, for your very helpful reply.  I'll try out your suggestion,

Tony