6 Replies Latest reply: Jul 21, 2010 12:23 PM by Tim Webber RSS

    Memory Usage Issue

    Tim Webber

      I have a document that is 40megs. How come when I load it into memory it takes over 200megs of memory? It takes this much space if i load it locally (qv.exe) or remotely through a thin client on the server (qvs.exe).

      Build is 9.0.7440.8

       

      Thanks!

        • Memory Usage Issue
          Stefan Bäckstrand

          Tim, the qvw document size on disk is in very few ways a measure for it's size in memory. Depending on data structure and foremost your chart objects design, it will consume different amounts of memory. The raw "table" data is compressed on disk and when unaggregated in memory.

          A table that defines multiple calculations on, let's say, millions of rows of data is bound to extract some volume in memory, since the compressed (or "optimized") data will then be aggregated and calculated, and consequently also cached in memory for future use.

          This is by design. You can of course affect the size in memory (to most extent) by cutting down data amounts, decrease data uniquety and build smarter chart objects/tabs.

          • Memory Usage Issue
            Scott Moon

            Tim - When the application is closed, it is only using the space required to store the code that creates the visible app and any compressed data that is saved with the application. When you open it, the sheets and objects use memory, the data (I assume) gets expanded to some extent, and more memory is used to display the app. In other words, there's a lot more going on after you open the app than when it's sitting idle on the hard drive.

            I'm sure there are other reasons, but these are the ones that are common to most applications - QlikView or otherwise.

            Scott Moon

              • Memory Usage Issue
                Tim Webber

                Thanks folks. Both answers make sense. Here's a question for you. My application is about 4 times as big in memory as it is on disk..

                As my application grows in size due to just new data, would it be fair to say that the app will continue to be "about" 4 times the size in memory (ie if I dont add any more charts/lists)? I am thinking the data will end up driving the final size of an application that has not had any changes.

                  • Memory Usage Issue
                    Stefan Bäckstrand

                    I depends very much on the uniquety (spelling?) of the data, I would say. It's hard to say for sure.

                      • Memory Usage Issue
                        Scott Moon

                        Keep in mind that QlikView doesn't store data like a traditional relational database. Most databases store each individual piece of data, regardless of whether that same value is already stored in another record. QlikView stores each unique value the first time that value is encountered and then uses pointers to that value everywhere else. So with no other changes to the application, the size should only grow appreciably with the addition of new unique values. If you keep pumping in the same values in different combinations, it will only grow by the addition of pointers, which use much less space than most data.

                        Scott Moon

                        • Memory Usage Issue
                          Tim Webber

                          For sure. the data model behind this app is dimensional and the dimensions do not grow very much so data size is driven by mesures in fact tables, the primary key of the fact table and then as you point out, the calculations that i have that are done on the fly.

                          Each measures are probably pretty unique though some are counts that will be repeated. Thanks folks.