The rapid growth of memory after the amount of data up to 814 million line

[jixiaoquan]

Hi everyone,

    I have a  128 G memory computer.Recently I use this computer to load a qvd files,the qvd files have 1 billion 250 million line data.During the load process,it stopped when load 814 million ,before it stopped load the use of memory is 40 G ,when it stopped the use of memory grow to 100% in rapidly.

    About 10 minutes later,it go on to load the rest of data,but the use of memory keep in 100% until the end of load.

    Is it normal in this phenomenon?

    If it is abnormal,could you tell me why it appear this kind of imagination? And how to solve this problem?

[marcus_sommer]

I assume it has what do with the way qlikview processed the data internal with Symbol Tables and Bit-Stuffed Pointers if the amount of data exceeds some point and needed to be extend. Have a look here:

http://community.qlik.com/blogs/qlikviewdesignblog/2012/11/20/symbol-tables-and-bit-stuffed-pointers

- Marcus